---
title: "Kimi-Audio Technical Report — paper by KimiTeam et al. (2025)"
description: "They release checkpoints, code and an evaluation toolkit that makes model comparisons fairer. 89 references, cited by 55 papers here."
authors: ["KimiTeam", "Ding Ding", "Zeqian Ju", "Yichong Leng", "Songxiang Liu", "Tong Liu", "Zeyu Shang", "Kai Shen", "Wei Song", "Xu Tan", "Heyi Tang", "Zhengtao Wang", "Chu Wei", "Yifei Xin", "Xinran Xu", "Jianwei Yu", "Yutao Zhang", "Xinyu Zhou", "Y. Charles", "Jun Chen", "Yanru Chen", "Yulun Du", "Weiran He", "Zhenxing Hu", "Guokun Lai", "Qingcheng Li", "Yangyang Liu", "Weidong Sun", "Jianzhou Wang", "Yuzhi Wang", "Yuefeng Wu", "Yuxin Wu", "Dongchao Yang", "Hao Yang", "Ying Yang", "Zhilin Yang", "Aoxiong Yin", "Ruibin Yuan", "Yutong Zhang", "Zaida Zhou"]
year: 2025
venue: "arXiv:2504.18425"
source: "https://arxiv.org/abs/2504.18425"
doi: "10.48550/arxiv.2504.18425"
arxiv: "2504.18425"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/kimiteam-2025-kimi-audio-technical-report/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/kimiteam-2025-kimi-audio-technical-report/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/) / Kimi-Audio Technical Report

# Kimi-Audio Technical Report

**Authors:** KimiTeam, Ding Ding, [Zeqian Ju](https://md.research.elamur.ai/free/en/authors/zeqian-ju/), [Yichong Leng](https://md.research.elamur.ai/free/en/authors/yichong-leng/), Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen, Wei Song, [Xu Tan](https://md.research.elamur.ai/free/en/authors/xu-tan/), Heyi Tang, Zhengtao Wang, Chu Wei, Yifei Xin, Xinran Xu, Jianwei Yu, Yutao Zhang, Xinyu Zhou, [Y. Charles](https://md.research.elamur.ai/free/en/authors/y-charles/), [Jun Chen](https://md.research.elamur.ai/free/en/authors/jun-chen/), Yanru Chen, Yulun Du, Weiran He, Zhenxing Hu, Guokun Lai, Qingcheng Li, Yangyang Liu, Weidong Sun, Jianzhou Wang, Yuzhi Wang, Yuefeng Wu, Yuxin Wu, Dongchao Yang, [Hao Yang](https://md.research.elamur.ai/free/en/authors/hao-yang/), Ying Yang, [Zhilin Yang](https://md.research.elamur.ai/free/en/authors/zhilin-yang/), Aoxiong Yin, Ruibin Yuan, Yutong Zhang and Zaida Zhou · **Year:** 2025 · **Venue:** arXiv:2504.18425 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2504.18425) · [PDF](https://arxiv.org/pdf/2504.18425) · [DOI 10.48550/arxiv.2504.18425](https://doi.org/10.48550/arxiv.2504.18425) · [arXiv 2504.18425](https://arxiv.org/abs/2504.18425) · [OpenAlex](https://openalex.org/W4416381162)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · [55](https://md.research.elamur.ai/free/en/papers/kimiteam-2025-kimi-audio-technical-report/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Earlier audio language models tend to target one task family, skip serious audio pre-training, or ship without code and weights. Kimi-Audio (Kimi Team, Moonshot AI) is an open-source foundation model that covers speech recognition, audio understanding, generation and spoken dialogue in one system. Its input merges discrete semantic tokens from the GLM-4-Voice tokenizer (a quantized Whisper encoder at 12.5Hz) with continuous Whisper features downsampled to the same rate. An LLM initialized from a pre-trained text model shares its lower layers, then branches into a text head and an audio head. A chunk-wise streaming flow-matching detokenizer with look-ahead renders speech at low latency. Continual pre-training uses over 13 million hours of speech, sound and music, combining text-only, audio-only, audio-to-text and interleaved tasks. Fine-tuning relies mostly on openly available data. The authors report state-of-the-art results on ASR, audio understanding, audio QA and speech conversation. They release checkpoints, code and an evaluation toolkit that makes model comparisons fairer.

## Key points
- **Pre-training corpus:** more than 13M hours of audio spanning speech, sound and music. It was cleaned with a pipeline covering speech enhancement, diarization, transcription and filtering.
- **Hybrid 12.5Hz input:** discrete semantic tokens (GLM-4-Voice, single codebook) are added to Whisper features. An adaptor brings these features from 50Hz down to 12.5Hz.
- **Dual-head audio LLM:** shared transformer layers and the text head reuse the pre-trained text LLM's weights, while the audio head starts from random initialization.
- **Detokenizer:** it follows the MoonCast design, with flow matching from 12.5Hz tokens to 50Hz mel-spectrograms followed by a BigVGAN vocoder. It streams in chunks of about one second, using causal chunk masking and look-ahead to avoid boundary artifacts.
- **Evaluation and release:** the model is benchmarked against Qwen2-Audio, Baichuan-Audio, Step-Audio and Qwen2.5-Omni. Code, weights and the evaluation toolkit are available at github.com/MoonshotAI/Kimi-Audio.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [🔊 Signal Processing](https://md.research.elamur.ai/free/en/shelves/computer-science/signal-processing/) · 213 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 1,344 works · [🎙️ Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) · 233 works

## Cite

Cite as: KimiTeam, Ding Ding, Zeqian Ju, Yichong Leng, Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen, Wei Song, Xu Tan, Heyi Tang, Zhengtao Wang, Chu Wei, Yifei Xin, Xinran Xu, Jianwei Yu, Yutao Zhang, Xinyu Zhou, Y. Charles, Jun Chen, Yanru Chen, Yulun Du, Weiran He, Zhenxing Hu, Guokun Lai, Qingcheng Li, Yangyang Liu, Weidong Sun, Jianzhou Wang, Yuzhi Wang, Yuefeng Wu, Yuxin Wu, Dongchao Yang, Hao Yang, Ying Yang, Zhilin Yang, Aoxiong Yin, Ruibin Yuan, Yutong Zhang and Zaida Zhou. 2025. Kimi-Audio Technical Report. arXiv:2504.18425.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{kimiteam2025kimi,
  title = {{Kimi-Audio Technical Report}},
  author = {KimiTeam, Ding Ding, Zeqian Ju, Yichong Leng, Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen, Wei Song, Xu Tan, Heyi Tang, Zhengtao Wang, Chu Wei, Yifei Xin, Xinran Xu, Jianwei Yu, Yutao Zhang, Xinyu Zhou, Y. Charles, Jun Chen, Yanru Chen, Yulun Du, Weiran He, Zhenxing Hu, Guokun Lai, Qingcheng Li, Yangyang Liu, Weidong Sun, Jianzhou Wang, Yuzhi Wang, Yuefeng Wu, Yuxin Wu, Dongchao Yang, Hao Yang, Ying Yang, Zhilin Yang, Aoxiong Yin, Ruibin Yuan, Yutong Zhang and Zaida Zhou},
  year = {2025},
  doi = {10.48550/arxiv.2504.18425},
  eprint = {2504.18425},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2504.18425}
}
```

- [← previous MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention](https://md.research.elamur.ai/free/en/papers/minimax-2025-minimax-m1-scaling-test/)
- [next → DeepSeek-OCR: Contexts Optical Compression](https://md.research.elamur.ai/free/en/papers/wei-2025-deepseek-ocr-contexts-optical/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/2/)

## References

89 references, 22 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rosana Ardila et al. “[Common voice: A massively-multilingual speech corpus](https://md.research.elamur.ai/free/en/papers/ardila-2019-common-voice-massively-multilingual-speech/)”. In: arXiv preprint arXiv:1912.06670 (2019).
2. Beijing Academy of Artificial Intelligence (BAAI). “Infinity Instruct”. In: arXiv preprint arXiv:2406.XXXX (2024). [find](https://scholar.google.com/scholar?q=Beijing+Academy+of+Artificial+Intelligence+%28BAAI%29%2E+%E2%80%9CInfinity+Instruct%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2406%2EXXXX+%282024%29%2E)
3. Zalán Borsos et al. “[Audiolm: a language modeling approach to audio generation](https://md.research.elamur.ai/free/en/papers/borsos-2022-audiolm-language-modeling-approach-audio/)”. In: IEEE/ACM transactions on audio, speech, and language processing 31 (2023), pp. 2523–2533.
4. Hui Bu et al. “Aishell-1: An open-source mandarin speech corpus and a speech recognition baseline”. In: 2017 20th conference of the oriental chapter of the international coordinating committee on speech databases and speech I/O systems and assessment (O-COCOSDA). IEEE. 2017, pp. 1–5. [find](https://scholar.google.com/scholar?q=Hui+Bu+et+al%2E+%E2%80%9CAishell%2D1%3A+An+open%2Dsource+mandarin+speech+corpus+and+a+speech+recognition+baseline%E2%80%9D%2E+In%3A+2017+20th+conference+of+the+oriental+chapter+of+the+international+coordinating+committee+on+spee)
5. Guoguo Chen et al. “Gigaspeech: An evolving, multi-domain asr corpus with 10,000 hours of transcribed audio”. In: arXiv preprint arXiv:2106.06909 (2021). — [arXiv:2106.06909](https://arxiv.org/abs/2106.06909) [find](https://scholar.google.com/scholar?q=Guoguo+Chen+et+al%2E+%E2%80%9CGigaspeech%3A+An+evolving%2C+multi%2Ddomain+asr+corpus+with+10%2C000+hours+of+transcribed+audio%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2106%2E06909+%282021%29%2E+%E2%80%94+%5BarXiv%3A2106%2E06909%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F210)
6. Honglie Chen et al. “Vggsound: A large-scale audio-visual dataset”. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. 2020, pp. 721–725. [find](https://scholar.google.com/scholar?q=Honglie+Chen+et+al%2E+%E2%80%9CVggsound%3A+A+large%2Dscale+audio%2Dvisual+dataset%E2%80%9D%2E+In%3A+ICASSP+2020%2D2020+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2E+IEEE%2E+2020%2C+pp%2E+721%E2%80%93725%2E)
7. Qian Chen et al. “Minmo: A multimodal large language model for seamless voice interaction”. In: arXiv preprint arXiv:2501.06282 (2025). — [arXiv:2501.06282](https://arxiv.org/abs/2501.06282) [find](https://scholar.google.com/scholar?q=Qian+Chen+et+al%2E+%E2%80%9CMinmo%3A+A+multimodal+large+language+model+for+seamless+voice+interaction%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2501%2E06282+%282025%29%2E+%E2%80%94+%5BarXiv%3A2501%2E06282%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2501%2E06282%29)
8. Yiming Chen et al. “[VoiceBench: Benchmarking LLM-Based Voice Assistants](https://md.research.elamur.ai/free/en/papers/chen-2026-voicebench-benchmarking-llm-based-voice/)”. In: arXiv preprint arXiv:2410.17196 (2024).
9. Zesen Cheng et al. “Videollama 2: Advancing spatial-temporal modeling and audio understanding in video-llms”. In: arXiv preprint arXiv:2406.07476 (2024). [find](https://scholar.google.com/scholar?q=Zesen+Cheng+et+al%2E+%E2%80%9CVideollama+2%3A+Advancing+spatial%2Dtemporal+modeling+and+audio+understanding+in+video%2Dllms%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2406%2E07476+%282024%29%2E)
10. Yunfei Chu et al. “[Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models](https://md.research.elamur.ai/free/en/papers/chu-2023-qwen-audio-advancing-universal/)”. In: arXiv preprint arXiv:2311.07919 (2023).
11. Yunfei Chu et al. “[Qwen2-audio technical report](https://md.research.elamur.ai/free/en/papers/chu-2024-qwen2-audio-technical-report/)”. In: arXiv preprint arXiv:2407.10759 (2024).
12. Alexis Conneau et al. “[Fleurs: Few-shot learning evaluation of universal representations of speech](https://md.research.elamur.ai/free/en/papers/conneau-2022-fleurs-few-shot-learning-evaluation/)”. In: 2022 IEEE Spoken Language Technology Workshop (SLT). IEEE. 2023, pp. 798–805.
13. DeepSeek-AI. DeepSeek-V3 Technical Report. 2024. arXiv: 2412.19437 [cs.CL]. URL : https://arxiv.org/abs/2412.19437. [↗](https://arxiv.org/abs/2412.19437)
14. Alexandre Défossez et al. “[Moshi: a speech-text foundation model for real-time dialogue](https://md.research.elamur.ai/free/en/papers/defossez-2024-moshi-speech-text-foundation-model/)”. In: arXiv preprint arXiv:2410.00037 (2024).
15. Chandeepa Dissanayake et al. OpenBezoar: Small, Cost-Effective and Open Models Trained on Mixes of Instruction Data. 2024. arXiv: 2404.12195 [cs.CL]. — [arXiv:2404.12195](https://arxiv.org/abs/2404.12195) [find](https://scholar.google.com/scholar?q=Chandeepa+Dissanayake+et+al%2E+OpenBezoar%3A+Small%2C+Cost%2DEffective+and+Open+Models+Trained+on+Mixes+of+Instruction+Data%2E+2024%2E+arXiv%3A+2404%2E12195+%5Bcs%2ECL%5D%2E+%E2%80%94+%5BarXiv%3A2404%2E12195%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2404%2E12)
16. Konstantinos Drossos, Samuel Lipping, and Tuomas Virtanen. “Clotho: An audio captioning dataset”. In: ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. 2020, pp. 736–740. [find](https://scholar.google.com/scholar?q=Konstantinos+Drossos%2C+Samuel+Lipping%2C+and+Tuomas+Virtanen%2E+%E2%80%9CClotho%3A+An+audio+captioning+dataset%E2%80%9D%2E+In%3A+ICASSP+2020%2D2020+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29)
17. Jiayu Du et al. “Aishell-2: Transforming mandarin asr research into industrial scale”. In: arXiv preprint arXiv:1808.10583 (2018). [find](https://scholar.google.com/scholar?q=Jiayu+Du+et+al%2E+%E2%80%9CAishell%2D2%3A+Transforming+mandarin+asr+research+into+industrial+scale%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A1808%2E10583+%282018%29%2E)
18. Qingkai Fang et al. “Llama-omni: Seamless speech interaction with large language models”. In: arXiv preprint arXiv:2409.06666 (2024). [find](https://scholar.google.com/scholar?q=Qingkai+Fang+et+al%2E+%E2%80%9CLlama%2Domni%3A+Seamless+speech+interaction+with+large+language+models%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2409%2E06666+%282024%29%2E)
19. Eduardo Fonseca et al. “Fsd50k: an open dataset of human-labeled sound events”. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing 30 (2021), pp. 829–852. [find](https://scholar.google.com/scholar?q=Eduardo+Fonseca+et+al%2E+%E2%80%9CFsd50k%3A+an+open+dataset+of+human%2Dlabeled+sound+events%E2%80%9D%2E+In%3A+IEEE%2FACM+Transactions+on+Audio%2C+Speech%2C+and+Language+Processing+30+%282021%29%2C+pp%2E+829%E2%80%93852%2E)
20. Zhifu Gao et al. “Paraformer: Fast and accurate parallel transformer for non-autoregressive end-to-end speech recognition”. In: arXiv preprint arXiv:2206.08317 (2022). — [arXiv:2206.08317](https://arxiv.org/abs/2206.08317) [find](https://scholar.google.com/scholar?q=Zhifu+Gao+et+al%2E+%E2%80%9CParaformer%3A+Fast+and+accurate+parallel+transformer+for+non%2Dautoregressive+end%2Dto%2Dend+speech+recognition%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2206%2E08317+%282022%29%2E+%E2%80%94+%5BarXiv%3A2206%2E08317%5D%28https%3A%2F%2Farx)
21. Xuelong Geng et al. “OSUM: Advancing Open Speech Understanding Models with Limited Resources in Academia”. In: arXiv preprint arXiv:2501.13306 (2025). — [arXiv:2501.13306](https://arxiv.org/abs/2501.13306) [find](https://scholar.google.com/scholar?q=Xuelong+Geng+et+al%2E+%E2%80%9COSUM%3A+Advancing+Open+Speech+Understanding+Models+with+Limited+Resources+in+Academia%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2501%2E13306+%282025%29%2E+%E2%80%94+%5BarXiv%3A2501%2E13306%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2501%2E1)
22. Sreyan Ghosh et al. “Gama: A large audio-language model with advanced audio understanding and complex reasoning abilities”. In: arXiv preprint arXiv:2406.11768 (2024). [find](https://scholar.google.com/scholar?q=Sreyan+Ghosh+et+al%2E+%E2%80%9CGama%3A+A+large+audio%2Dlanguage+model+with+advanced+audio+understanding+and+complex+reasoning+abilities%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2406%2E11768+%282024%29%2E)
23. Yuan Gong, Jin Yu, and James Glass. “Vocalsound: A Dataset for Improving Human Vocal Sounds Recognition”. In: ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). 2022, pp. 151–155. DOI : 10.1109/ICASSP43922.2022.9746828. — [doi:10.1109/icassp43922.2022.9746828](https://doi.org/10.1109/icassp43922.2022.9746828) [find](https://scholar.google.com/scholar?q=Yuan+Gong%2C+Jin+Yu%2C+and+James+Glass%2E+%E2%80%9CVocalsound%3A+A+Dataset+for+Improving+Human+Vocal+Sounds+Recognition%E2%80%9D%2E+In%3A+ICASSP+2022+%2D+2022+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processin)
24. Aaron Grattafiori et al. “The llama 3 herd of models”. In: arXiv preprint arXiv:2407.21783 (2024). [find](https://scholar.google.com/scholar?q=Aaron+Grattafiori+et+al%2E+%E2%80%9CThe+llama+3+herd+of+models%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2407%2E21783+%282024%29%2E)
25. Haorui He et al. “Emilia: A Large-Scale, Extensive, Multilingual, and Diverse Dataset for Speech Generation”. In: arXiv preprint arXiv:2501.15907 (2025). — [arXiv:2501.15907](https://arxiv.org/abs/2501.15907) [find](https://scholar.google.com/scholar?q=Haorui+He+et+al%2E+%E2%80%9CEmilia%3A+A+Large%2DScale%2C+Extensive%2C+Multilingual%2C+and+Diverse+Dataset+for+Speech+Generation%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2501%2E15907+%282025%29%2E+%E2%80%94+%5BarXiv%3A2501%2E15907%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F250)
26. Haorui He et al. “Emilia: An extensive, multilingual, and diverse speech dataset for large-scale speech generation”. In: 2024 IEEE Spoken Language Technology Workshop (SLT). IEEE. 2024, pp. 885–890. [find](https://scholar.google.com/scholar?q=Haorui+He+et+al%2E+%E2%80%9CEmilia%3A+An+extensive%2C+multilingual%2C+and+diverse+speech+dataset+for+large%2Dscale+speech+generation%E2%80%9D%2E+In%3A+2024+IEEE+Spoken+Language+Technology+Workshop+%28SLT%29%2E+IEEE%2E+2024%2C+pp%2E+885%E2%80%93890%2E)
27. T. Heittola et al. TAU Urban Acoustic Scenes 2022 Mobile, Development dataset. Zenodo. Mar. 2022. DOI : 10.5281/zenodo.6337421. — [doi:10.5281/zenodo.6337421](https://doi.org/10.5281/zenodo.6337421) [find](https://scholar.google.com/scholar?q=T%2E+Heittola+et+al%2E+TAU+Urban+Acoustic+Scenes+2022+Mobile%2C+Development+dataset%2E+Zenodo%2E+Mar%2E+2022%2E+DOI+%3A+10%2E5281%2Fzenodo%2E6337421%2E+%E2%80%94+%5Bdoi%3A10%2E5281%2Fzenodo%2E6337421%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E5281%2Fzenodo%2E6337421%29)
28. Ailin Huang et al. “Step-audio: Unified understanding and generation in intelligent speech interaction”. In: arXiv preprint arXiv:2502.11946 (2025). [find](https://scholar.google.com/scholar?q=Ailin+Huang+et+al%2E+%E2%80%9CStep%2Daudio%3A+Unified+understanding+and+generation+in+intelligent+speech+interaction%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2502%2E11946+%282025%29%2E)
29. Aaron Hurst et al. “[Gpt-4o system card](https://md.research.elamur.ai/free/en/papers/openai-2024-gpt-4o-system-card/)”. In: arXiv preprint arXiv:2410.21276 (2024).
30. Philip Jackson and Sana ul haq. Surrey Audio-Visual Expressed Emotion (SAVEE) database. Apr. 2011. [find](https://scholar.google.com/scholar?q=Philip+Jackson+and+Sana+ul+haq%2E+Surrey+Audio%2DVisual+Expressed+Emotion+%28SAVEE%29+database%2E+Apr%2E+2011%2E)
31. Il-Young Jeong and Jeongsoo Park. “Cochlscene: Acquisition of acoustic scene data using crowdsourcing”. In: 2022 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). IEEE. 2022, pp. 17–21. [find](https://scholar.google.com/scholar?q=Il%2DYoung+Jeong+and+Jeongsoo+Park%2E+%E2%80%9CCochlscene%3A+Acquisition+of+acoustic+scene+data+using+crowdsourcing%E2%80%9D%2E+In%3A+2022+Asia%2DPacific+Signal+and+Information+Processing+Association+Annual+Summit+and+Conference)
32. Zeqian Ju et al. “MoonCast: High-Quality Zero-Shot Podcast Generation”. In: arXiv preprint arXiv:2503.14345 (2025). — [arXiv:2503.14345](https://arxiv.org/abs/2503.14345) [find](https://scholar.google.com/scholar?q=Zeqian+Ju+et+al%2E+%E2%80%9CMoonCast%3A+High%2DQuality+Zero%2DShot+Podcast+Generation%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2503%2E14345+%282025%29%2E+%E2%80%94+%5BarXiv%3A2503%2E14345%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2503%2E14345%29)
33. Wei Kang et al. “Libriheavy: A 50,000 hours ASR corpus with punctuation casing and context”. In: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. 2024, pp. 10991–10995. [find](https://scholar.google.com/scholar?q=Wei+Kang+et+al%2E+%E2%80%9CLibriheavy%3A+A+50%2C000+hours+ASR+corpus+with+punctuation+casing+and+context%E2%80%9D%2E+In%3A+ICASSP+2024%2D2024+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2E+IEE)
34. Chris Dongjoo Kim et al. “[Audiocaps: Generating captions for audios in the wild](https://md.research.elamur.ai/free/en/papers/kim-2019-audiocaps-generating-captions-audios-wild/)”. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). 2019, pp. 119–132.
35. Zhifeng Kong et al. “Audio flamingo: A novel audio language model with few-shot learning and dialogue abilities”. In: arXiv preprint arXiv:2402.01831 (2024). — [arXiv:2402.01831](https://arxiv.org/abs/2402.01831) [find](https://scholar.google.com/scholar?q=Zhifeng+Kong+et+al%2E+%E2%80%9CAudio+flamingo%3A+A+novel+audio+language+model+with+few%2Dshot+learning+and+dialogue+abilities%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2402%2E01831+%282024%29%2E+%E2%80%94+%5BarXiv%3A2402%2E01831%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs)
36. Nathan Lambert et al. “T\\" ulu 3: Pushing frontiers in open language model post-training”. In: arXiv preprint arXiv:2411.[15124 (2024)](https://md.research.elamur.ai/free/en/papers/lambert-2024-tulu-3-pushing-frontiers-open/).
37. Matthew Le et al. “Voicebox: Text-guided multilingual universal speech generation at scale”. In: Advances in neural information processing systems 36 (2023), pp. 14005–14034. [find](https://scholar.google.com/scholar?q=Matthew+Le+et+al%2E+%E2%80%9CVoicebox%3A+Text%2Dguided+multilingual+universal+speech+generation+at+scale%E2%80%9D%2E+In%3A+Advances+in+neural+information+processing+systems+36+%282023%29%2C+pp%2E+14005%E2%80%9314034%2E)
38. Sang-gil Lee et al. “Bigvgan: A universal neural vocoder with large-scale training”. In: arXiv preprint arXiv:2206.04658 (2022). — [arXiv:2206.04658](https://arxiv.org/abs/2206.04658) [find](https://scholar.google.com/scholar?q=Sang%2Dgil+Lee+et+al%2E+%E2%80%9CBigvgan%3A+A+universal+neural+vocoder+with+large%2Dscale+training%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2206%2E04658+%282022%29%2E+%E2%80%94+%5BarXiv%3A2206%2E04658%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2206%2E04658%29)
39. Guangyao Li et al. “Learning to answer questions in dynamic audio-visual scenarios”. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2022, pp. 19108–19118. [find](https://scholar.google.com/scholar?q=Guangyao+Li+et+al%2E+%E2%80%9CLearning+to+answer+questions+in+dynamic+audio%2Dvisual+scenarios%E2%80%9D%2E+In%3A+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Pattern+Recognition%2E+2022%2C+pp%2E+19108%E2%80%9319118%2E)
40. Jia Li et al. “Numinamath: The largest public dataset in ai4maths with 860k pairs of competition math problems and solutions”. In: Hugging Face repository 13 (2024), p. 9. [find](https://scholar.google.com/scholar?q=Jia+Li+et+al%2E+%E2%80%9CNuminamath%3A+The+largest+public+dataset+in+ai4maths+with+860k+pairs+of+competition+math+problems+and+solutions%E2%80%9D%2E+In%3A+Hugging+Face+repository+13+%282024%29%2C+p%2E+9%2E)
41. Tianpeng Li et al. “Baichuan-audio: A unified framework for end-to-end speech interaction”. In: arXiv preprint arXiv:2502.17239 (2025). [find](https://scholar.google.com/scholar?q=Tianpeng+Li+et+al%2E+%E2%80%9CBaichuan%2Daudio%3A+A+unified+framework+for+end%2Dto%2Dend+speech+interaction%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2502%2E17239+%282025%29%2E)
42. Wing Lian et al. OpenOrca: An Open Dataset of GPT Augmented FLAN Reasoning Traces. https://https://huggingface.co/datasets/Open-Orca/OpenOrca. 2023. [find](https://scholar.google.com/scholar?q=Wing+Lian+et+al%2E+OpenOrca%3A+An+Open+Dataset+of+GPT+Augmented+FLAN+Reasoning+Traces%2E+https%3A%2F%2Fhttps%3A%2F%2Fhuggingface%2Eco%2Fdatasets%2FOpen%2DOrca%2FOpenOrca%2E+2023%2E)
43. Samuel Lipping et al. “Clotho-aqa: A crowdsourced dataset for audio question answering”. In: 2022 30th European Signal Processing Conference (EUSIPCO). IEEE. 2022, pp. 1140–1144. [find](https://scholar.google.com/scholar?q=Samuel+Lipping+et+al%2E+%E2%80%9CClotho%2Daqa%3A+A+crowdsourced+dataset+for+audio+question+answering%E2%80%9D%2E+In%3A+2022+30th+European+Signal+Processing+Conference+%28EUSIPCO%29%2E+IEEE%2E+2022%2C+pp%2E+1140%E2%80%931144%2E)
44. Jingyuan Liu et al. Muon is Scalable for LLM Training. 2025. arXiv: 2502.16982 [cs.LG]. URL : https://arxiv.org/abs/2502.16982. [↗](https://arxiv.org/abs/2502.16982)
45. Rui-Bo Liu et al. “Convincing Audio Generation Based on LLM and Speech Tokenization”. In: 2024 IEEE 14th International Symposium on Chinese Spoken Language Processing (ISCSLP). IEEE. 2024, pp. 591–595. [find](https://scholar.google.com/scholar?q=Rui%2DBo+Liu+et+al%2E+%E2%80%9CConvincing+Audio+Generation+Based+on+LLM+and+Speech+Tokenization%E2%80%9D%2E+In%3A+2024+IEEE+14th+International+Symposium+on+Chinese+Spoken+Language+Processing+%28ISCSLP%29%2E+IEEE%2E+2024%2C+pp%2E+591%E2%80%93595)
46. Songting Liu. “Zero-shot Voice Conversion with Diffusion Transformers”. In: arXiv preprint arXiv:2411.09943 (2024). — [arXiv:2411.09943](https://arxiv.org/abs/2411.09943) [find](https://scholar.google.com/scholar?q=Songting+Liu%2E+%E2%80%9CZero%2Dshot+Voice+Conversion+with+Diffusion+Transformers%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2411%2E09943+%282024%29%2E+%E2%80%94+%5BarXiv%3A2411%2E09943%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2411%2E09943%29)
47. Steven R Livingstone and Frank A Russo. “The Ryerson Audio-Visual Database of Emotional Speech and Song (RAVDESS): A dynamic, multimodal set of facial and vocal expressions in North American English”. In: PloS one 13.5 (2018), e0196391. [find](https://scholar.google.com/scholar?q=Steven+R+Livingstone+and+Frank+A+Russo%2E+%E2%80%9CThe+Ryerson+Audio%2DVisual+Database+of+Emotional+Speech+and+Song+%28RAVDESS%29%3A+A+dynamic%2C+multimodal+set+of+facial+and+vocal+expressions+in+North+American+English%E2%80%9D%2E)
48. Ilya Loshchilov and Frank Hutter. “Decoupled weight decay regularization”. In: arXiv preprint arXiv:1711.05101 (2017). [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+%E2%80%9CDecoupled+weight+decay+regularization%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A1711%2E05101+%282017%29%2E)
49. Yi Luo and Jianwei Yu. “Music Source Separation With Band-Split RNN”. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing 31 (2023), pp. 1893–1901. DOI : 10.1109/TASLP.2023.3271145. — [doi:10.1109/taslp.2023.3271145](https://doi.org/10.1109/taslp.2023.3271145) [find](https://scholar.google.com/scholar?q=Yi+Luo+and+Jianwei+Yu%2E+%E2%80%9CMusic+Source+Separation+With+Band%2DSplit+RNN%E2%80%9D%2E+In%3A+IEEE%2FACM+Transactions+on+Audio%2C+Speech%2C+and+Language+Processing+31+%282023%29%2C+pp%2E+1893%E2%80%931901%2E+DOI+%3A+10%2E1109%2FTASLP%2E2023%2E3271145%2E+%E2%80%94+)
50. Linhan Ma et al. “Wenetspeech4tts: A 12,800-hour mandarin tts corpus for large speech generation model benchmark”. In: arXiv preprint arXiv:2406.05763 (2024). — [arXiv:2406.05763](https://arxiv.org/abs/2406.05763) [find](https://scholar.google.com/scholar?q=Linhan+Ma+et+al%2E+%E2%80%9CWenetspeech4tts%3A+A+12%2C800%2Dhour+mandarin+tts+corpus+for+large+speech+generation+model+benchmark%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2406%2E05763+%282024%29%2E+%E2%80%94+%5BarXiv%3A2406%2E05763%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
51. Irene Martín-Morató and Annamaria Mesaros. “What is the ground truth? reliability of multi-annotator data for audio tagging”. In: 2021 29th European Signal Processing Conference (EUSIPCO). IEEE. 2021, pp. 76–80. [find](https://scholar.google.com/scholar?q=Irene+Mart%C3%ADn%2DMorat%C3%B3+and+Annamaria+Mesaros%2E+%E2%80%9CWhat+is+the+ground+truth%3F+reliability+of+multi%2Dannotator+data+for+audio+tagging%E2%80%9D%2E+In%3A+2021+29th+European+Signal+Processing+Conference+%28EUSIPCO%29%2E+IEEE%2E+2021%2C)
52. Xinhao Mei et al. “Wavcaps: A chatgpt-assisted weakly-labelled audio captioning dataset for audio-language multimodal research”. In: IEEE/ACM Transactions on Audio, Speech, and Language Processing (2024). [find](https://scholar.google.com/scholar?q=Xinhao+Mei+et+al%2E+%E2%80%9CWavcaps%3A+A+chatgpt%2Dassisted+weakly%2Dlabelled+audio+captioning+dataset+for+audio%2Dlanguage+multimodal+research%E2%80%9D%2E+In%3A+IEEE%2FACM+Transactions+on+Audio%2C+Speech%2C+and+Language+Processing+%2820)
53. Annamaria Mesaros, Toni Heittola, and Tuomas Virtanen. “TUT Database for Acoustic Scene Classification and Sound Event Detection”. In: 24th European Signal Processing Conference 2016 (EUSIPCO 2016). Budapest, Hungary, 2016. [find](https://scholar.google.com/scholar?q=Annamaria+Mesaros%2C+Toni+Heittola%2C+and+Tuomas+Virtanen%2E+%E2%80%9CTUT+Database+for+Acoustic+Scene+Classification+and+Sound+Event+Detection%E2%80%9D%2E+In%3A+24th+European+Signal+Processing+Conference+2016+%28EUSIPCO+2016%29%2E+B)
54. Vassil Panayotov et al. “Librispeech: an asr corpus based on public domain audio books”. In: 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP). IEEE. 2015, pp. 5206–5210. [find](https://scholar.google.com/scholar?q=Vassil+Panayotov+et+al%2E+%E2%80%9CLibrispeech%3A+an+asr+corpus+based+on+public+domain+audio+books%E2%80%9D%2E+In%3A+2015+IEEE+international+conference+on+acoustics%2C+speech+and+signal+processing+%28ICASSP%29%2E+IEEE%2E+2015%2C+pp%2E+520)
55. Karol J. Piczak. “ESC: Dataset for Environmental Sound Classification”. In: Proceedings of the 23rd Annual ACM Conference on Multimedia. Brisbane, Australia: ACM Press, Oct. 13, 2015, pp. 1015–1018. ISBN : 978-1-4503-3459-4. DOI : 10.1145/2733373.2806390. URL : [http://dl.acm.org/citation.cfm?doid=27…](http://dl.acm.org/citation.cfm?doid=27…) — [doi:10.1145/2733373.2806390](https://doi.org/10.1145/2733373.2806390) [↗](http://dl.acm.org/citation.cfm?doid=27…)
56. Soujanya Poria et al. “[Meld: A multimodal multi-party dataset for emotion recognition in conversations](https://md.research.elamur.ai/free/en/papers/poria-2018-meld-multimodal-multi-party-dataset/)”. In: arXiv preprint arXiv:1810.02508 (2018).
57. Vineel Pratap et al. “[MLS: A Large-Scale Multilingual Dataset for Speech Research](https://md.research.elamur.ai/free/en/papers/pratap-2020-mls-large-scale-multilingual-dataset/)”. In: ArXiv abs/2012.03411 (2020).
58. Alec Radford et al. “[Robust speech recognition via large-scale weak supervision](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/)”. In: International conference on machine learning. PMLR. 2023, pp. 28492–28518.
59. Muhammad Mamunur Rashid, Guiqing Li, and Chengrui Du. “Nonspeech7k dataset: Classification and analysis of human non-speech sound”. In: IET Signal Processing 17.6 (2023), e12233. [find](https://scholar.google.com/scholar?q=Muhammad+Mamunur+Rashid%2C+Guiqing+Li%2C+and+Chengrui+Du%2E+%E2%80%9CNonspeech7k+dataset%3A+Classification+and+analysis+of+human+non%2Dspeech+sound%E2%80%9D%2E+In%3A+IET+Signal+Processing+17%2E6+%282023%29%2C+e12233%2E)
60. S Sakshi et al. “Mmau: A massive multi-task audio understanding and reasoning benchmark”. In: arXiv preprint arXiv:2410.19168 (2024). [find](https://scholar.google.com/scholar?q=S+Sakshi+et+al%2E+%E2%80%9CMmau%3A+A+massive+multi%2Dtask+audio+understanding+and+reasoning+benchmark%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2410%2E19168+%282024%29%2E)
61. Justin Salamon, Christopher Jacoby, and Juan Pablo Bello. “A dataset and taxonomy for urban sound research”. In: Proceedings of the 22nd ACM international conference on Multimedia. 2014, pp. 1041–1044. [find](https://scholar.google.com/scholar?q=Justin+Salamon%2C+Christopher+Jacoby%2C+and+Juan+Pablo+Bello%2E+%E2%80%9CA+dataset+and+taxonomy+for+urban+sound+research%E2%80%9D%2E+In%3A+Proceedings+of+the+22nd+ACM+international+conference+on+Multimedia%2E+2014%2C+pp%2E+1041%E2%80%931044)
62. Yao Shi et al. “Aishell-3: A multi-speaker mandarin tts corpus and the baselines”. In: arXiv preprint arXiv:2010.11567 (2020). — [arXiv:2010.11567](https://arxiv.org/abs/2010.11567) [find](https://scholar.google.com/scholar?q=Yao+Shi+et+al%2E+%E2%80%9CAishell%2D3%3A+A+multi%2Dspeaker+mandarin+tts+corpus+and+the+baselines%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2010%2E11567+%282020%29%2E+%E2%80%94+%5BarXiv%3A2010%2E11567%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2010%2E11567%29)
63. Changli Tang et al. “[Salmonn: Towards generic hearing abilities for large language models](https://md.research.elamur.ai/free/en/papers/tang-2023-salmonn-towards-generic-hearing-abilities/)”. In: arXiv preprint arXiv:2310.13289 (2023).
64. Zhiyuan Tang et al. “Kespeech: An open source speech dataset of mandarin and its eight subdialects”. In: Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). 2021. [find](https://scholar.google.com/scholar?q=Zhiyuan+Tang+et+al%2E+%E2%80%9CKespeech%3A+An+open+source+speech+dataset+of+mandarin+and+its+eight+subdialects%E2%80%9D%2E+In%3A+Thirty%2Dfifth+Conference+on+Neural+Information+Processing+Systems+Datasets+and+Benchmarks+Track+)
65. Kimi Team et al. [Kimi k1.5: Scaling Reinforcement Learning with LLMs](https://md.research.elamur.ai/free/en/papers/kimi-team-2025-kimi-k1-5-scaling-reinforcement/). 2025. arXiv: 2501.12599 [cs.AI]. URL : https://arxiv.org/abs/2501.12599. [↗](https://arxiv.org/abs/2501.12599)
66. Teknium. OpenHermes 2.5: An Open Dataset of Synthetic Data for Generalist LLM Assistants. 2023. URL : [https://huggingface.co/datasets/teknium/OpenHermes-2.5](https://huggingface.co/datasets/teknium/OpenHermes-2.5). [↗](https://huggingface.co/datasets/teknium/OpenHermes-2.5)
67. Migel Tissera. Synthia-70b-v1.2: Synthetic intelligent agent. Hugging Face. 2023. URL : [https://huggingface.co/migtissera/Synthia-13B](https://huggingface.co/migtissera/Synthia-13B). [↗](https://huggingface.co/migtissera/Synthia-13B)
68. Samarth Tripathi, Sarthak Tripathi, and Homayoon Beigi. “Multi-modal emotion recognition on iemocap dataset using deep learning”. In: arXiv preprint arXiv:1804.05788 (2018). — [arXiv:1804.05788](https://arxiv.org/abs/1804.05788) [find](https://scholar.google.com/scholar?q=Samarth+Tripathi%2C+Sarthak+Tripathi%2C+and+Homayoon+Beigi%2E+%E2%80%9CMulti%2Dmodal+emotion+recognition+on+iemocap+dataset+using+deep+learning%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A1804%2E05788+%282018%29%2E+%E2%80%94+%5BarXiv%3A1804%2E05788%5D%28https)
69. Changhan Wang et al. “[VoxPopuli: A large-scale multilingual speech corpus for representation learning, semi-supervised learning and interpretation](https://md.research.elamur.ai/free/en/papers/wang-2021-voxpopuli-large-scale-multilingual-speech/)”. In: arXiv preprint arXiv:2101.00390 (2021).
70. Chengyi Wang et al. “[Neural codec language models are zero-shot text to speech synthesizers](https://md.research.elamur.ai/free/en/papers/wang-2023-neural-codec-language-models-zero/)”. In: arXiv preprint arXiv:2301.02111 (2023).
71. Xiong Wang et al. “Freeze-omni: A smart and low latency speech-to-speech dialogue model with frozen llm”. In: arXiv preprint arXiv:2411.00774 (2024). [find](https://scholar.google.com/scholar?q=Xiong+Wang+et+al%2E+%E2%80%9CFreeze%2Domni%3A+A+smart+and+low+latency+speech%2Dto%2Dspeech+dialogue+model+with+frozen+llm%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2411%2E00774+%282024%29%2E)
72. Zhifei Xie and Changqiao Wu. “Mini-omni: Language models can hear, talk while thinking in streaming”. In: arXiv preprint arXiv:2408.16725 (2024). [find](https://scholar.google.com/scholar?q=Zhifei+Xie+and+Changqiao+Wu%2E+%E2%80%9CMini%2Domni%3A+Language+models+can+hear%2C+talk+while+thinking+in+streaming%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2408%2E16725+%282024%29%2E)
73. Jin Xu et al. “Qwen2. 5-omni technical report”. In: arXiv preprint arXiv:2503.[20215 (2025)](https://md.research.elamur.ai/free/en/papers/xu-2025-qwen2-5-omni-technical/).
74. Jin Xu et al. [Qwen2.5-Omni Technical Report](https://md.research.elamur.ai/free/en/papers/xu-2025-qwen2-5-omni-technical/). 2025. arXiv: 2503.20215 [cs.CL]. URL : https://arxiv.org/abs/2503.20215. [↗](https://arxiv.org/abs/2503.20215)
75. Zhangchen Xu et al. “[Magpie: Alignment data synthesis from scratch by prompting aligned llms with nothing](https://md.research.elamur.ai/free/en/papers/xu-2024-magpie-alignment-data-synthesis-scratch/)”. In: arXiv preprint arXiv:2406.08464 (2024).
76. An Yang et al. “Qwen2.5 Technical Report”. In: arXiv preprint arXiv:2412.15115 (2024). [find](https://scholar.google.com/scholar?q=An+Yang+et+al%2E+%E2%80%9CQwen2%2E5+Technical+Report%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2412%2E15115+%282024%29%2E)
77. Dongchao Yang et al. “Uniaudio: An audio foundation model toward universal audio generation”. In: arXiv preprint arXiv:2310.00704 (2023). — [arXiv:2310.00704](https://arxiv.org/abs/2310.00704) [find](https://scholar.google.com/scholar?q=Dongchao+Yang+et+al%2E+%E2%80%9CUniaudio%3A+An+audio+foundation+model+toward+universal+audio+generation%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2310%2E00704+%282023%29%2E+%E2%80%94+%5BarXiv%3A2310%2E00704%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2310%2E00704%29)
78. Pinci Yang et al. “Avqa: A dataset for audio-visual question answering on videos”. In: Proceedings of the 30th ACM international conference on multimedia. 2022, pp. 3480–3491. [find](https://scholar.google.com/scholar?q=Pinci+Yang+et+al%2E+%E2%80%9CAvqa%3A+A+dataset+for+audio%2Dvisual+question+answering+on+videos%E2%80%9D%2E+In%3A+Proceedings+of+the+30th+ACM+international+conference+on+multimedia%2E+2022%2C+pp%2E+3480%E2%80%933491%2E)
79. Zehui Yang et al. “Open source magicdata-ramc: A rich annotated mandarin conversational (ramc) speech dataset”. In: arXiv preprint arXiv:2203.16844 (2022). — [arXiv:2203.16844](https://arxiv.org/abs/2203.16844) [find](https://scholar.google.com/scholar?q=Zehui+Yang+et+al%2E+%E2%80%9COpen+source+magicdata%2Dramc%3A+A+rich+annotated+mandarin+conversational+%28ramc%29+speech+dataset%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2203%2E16844+%282022%29%2E+%E2%80%94+%5BarXiv%3A2203%2E16844%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2)
80. Zhen Ye et al. “Llasa: Scaling Train-Time and Inference-Time Compute for Llama-based Speech Synthesis”. In: arXiv preprint arXiv:2502.04128 (2025). — [arXiv:2502.04128](https://arxiv.org/abs/2502.04128) [find](https://scholar.google.com/scholar?q=Zhen+Ye+et+al%2E+%E2%80%9CLlasa%3A+Scaling+Train%2DTime+and+Inference%2DTime+Compute+for+Llama%2Dbased+Speech+Synthesis%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2502%2E04128+%282025%29%2E+%E2%80%94+%5BarXiv%3A2502%2E04128%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2502%2E0412)
81. Jianwei Yu et al. “Autoprep: An automatic preprocessing framework for in-the-wild speech data”. In: ICASSP 2024-2024 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. 2024, pp. 1136–1140. [find](https://scholar.google.com/scholar?q=Jianwei+Yu+et+al%2E+%E2%80%9CAutoprep%3A+An+automatic+preprocessing+framework+for+in%2Dthe%2Dwild+speech+data%E2%80%9D%2E+In%3A+ICASSP+2024%2D2024+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2E+)
82. Jianwei Yu et al. “High fidelity speech enhancement with band-split rnn”. In: arXiv preprint arXiv:2212.00406 (2022). — [arXiv:2212.00406](https://arxiv.org/abs/2212.00406) [find](https://scholar.google.com/scholar?q=Jianwei+Yu+et+al%2E+%E2%80%9CHigh+fidelity+speech+enhancement+with+band%2Dsplit+rnn%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2212%2E00406+%282022%29%2E+%E2%80%94+%5BarXiv%3A2212%2E00406%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2212%2E00406%29)
83. Heiga Zen et al. “[Libritts: A corpus derived from librispeech for text-to-speech](https://md.research.elamur.ai/free/en/papers/zen-2019-libritts-corpus-derived-librispeech-text/)”. In: arXiv preprint arXiv:1904.02882 (2019).
84. Aohan Zeng et al. “[Glm-4-voice: Towards intelligent and human-like end-to-end spoken chatbot](https://md.research.elamur.ai/free/en/papers/zeng-2024-glm-4-voice-intelligent/)”. In: arXiv preprint arXiv:2412.02612 (2024).
85. Binbin Zhang et al. “Wenetspeech: A 10000+ hours multi-domain mandarin corpus for speech recognition”. In: ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE. 2022, pp. 6182–6186. [find](https://scholar.google.com/scholar?q=Binbin+Zhang+et+al%2E+%E2%80%9CWenetspeech%3A+A+10000%2B+hours+multi%2Ddomain+mandarin+corpus+for+speech+recognition%E2%80%9D%2E+In%3A+ICASSP+2022%2D2022+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28IC)
86. Qinglin Zhang et al. “Omniflatten: An end-to-end gpt model for seamless voice conversation”. In: arXiv preprint arXiv:2410.17799 (2024). [find](https://scholar.google.com/scholar?q=Qinglin+Zhang+et+al%2E+%E2%80%9COmniflatten%3A+An+end%2Dto%2Dend+gpt+model+for+seamless+voice+conversation%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2410%2E17799+%282024%29%2E)
87. Yu Zhang et al. “Google usm: Scaling automatic speech recognition beyond 100 languages”. In: arXiv preprint arXiv:2303.01037 (2023). — [arXiv:2303.01037](https://arxiv.org/abs/2303.01037) [find](https://scholar.google.com/scholar?q=Yu+Zhang+et+al%2E+%E2%80%9CGoogle+usm%3A+Scaling+automatic+speech+recognition+beyond+100+languages%E2%80%9D%2E+In%3A+arXiv+preprint+arXiv%3A2303%2E01037+%282023%29%2E+%E2%80%94+%5BarXiv%3A2303%2E01037%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2303%2E01037%29)
88. Hang Zhao et al. “MINT: Boosting Audio-Language Model via Multi-Target Pre-Training and Instruction Tuning”. In: Interspeech 2024. 2024, pp. 52–56. DOI : 10.21437/Interspeech.2024-1863. — [doi:10.21437/interspeech.2024-1863](https://doi.org/10.21437/interspeech.2024-1863) [find](https://scholar.google.com/scholar?q=Hang+Zhao+et+al%2E+%E2%80%9CMINT%3A+Boosting+Audio%2DLanguage+Model+via+Multi%2DTarget+Pre%2DTraining+and+Instruction+Tuning%E2%80%9D%2E+In%3A+Interspeech+2024%2E+2024%2C+pp%2E+52%E2%80%9356%2E+DOI+%3A+10%2E21437%2FInterspeech%2E2024%2D1863%2E+%E2%80%94+%5Bdoi%3A10%2E2143)
89. Kun Zhou et al. “Emotional voice conversion: Theory, databases and ESD”. In: Speech Communication 137 (2022), pp. 1–18. [find](https://scholar.google.com/scholar?q=Kun+Zhou+et+al%2E+%E2%80%9CEmotional+voice+conversion%3A+Theory%2C+databases+and+ESD%E2%80%9D%2E+In%3A+Speech+Communication+137+%282022%29%2C+pp%2E+1%E2%80%9318%2E)

*Updated 24 October 2026*