---
title: "HearSay Benchmark: Do Audio LLMs Leak What They Hear?"
description: "To test this, they built HearSay, a benchmark of more than 22,000 real recordings. 92.89% accuracy. 65 references."
authors: ["Jin Wang", "Kaiwen Luo", "Liang Lin", "Weiliu Wang", "Yitian Chen", "Moayad Aloqaily", "Xuehai Tang", "Zhenhong Zhou", "Kun Wang", "Li Sun", "Qingsong Wen"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.964/"
doi: "10.18653/v1/2026.findings-acl.964"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wang-2026-hearsay-benchmark-do-audio-llms/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wang-2026-hearsay-benchmark-do-audio-llms/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/32/) / HearSay Benchmark: Do Audio LLMs Leak What They Hear?

# HearSay Benchmark: Do Audio LLMs Leak What They Hear?

**Authors:** [Jin Wang](https://md.research.elamur.ai/free/en/authors/jin-wang/), Kaiwen Luo, [Liang Lin](https://md.research.elamur.ai/free/en/authors/liang-lin/), Weiliu Wang, Yitian Chen, Moayad Aloqaily, [Xuehai Tang](https://md.research.elamur.ai/free/en/authors/xuehai-tang/), [Zhenhong Zhou](https://md.research.elamur.ai/free/en/authors/zhenhong-zhou/), [Kun Wang](https://md.research.elamur.ai/free/en/authors/kun-wang/), [Li Sun](https://md.research.elamur.ai/free/en/authors/li-sun/) and [Qingsong Wen](https://md.research.elamur.ai/free/en/authors/qingsong-wen/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.964/) · [PDF](https://aclanthology.org/2026.findings-acl.964.pdf) · [DOI 10.18653/v1/2026.findings-acl.964](https://doi.org/10.18653/v1/2026.findings-acl.964) · [OpenAlex](https://openalex.org/W7166805001)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Audio Large Language Models (ALLMs) are trained on huge amounts of speech. The authors ask whether this training lets them work out private facts about a speaker from the voice alone, with no help from what is actually said. To test this, they built HearSay, a benchmark of more than 22,000 real recordings. Most clips come from public lecture videos paired with speaker background profiles, and some come from existing speech datasets. Labels for eight attributes were produced through automated profiling and then checked by humans, so each one rests on factual records. These eight cover a speaker's Accent and Gender, their Weight, Age and Health Status, and their Income, Social Stratum and Education.

The benchmark measures three things:
- how accurately models infer each attribute (IAR),
- how often they decline to answer (ARR),
- whether they ignore the audio and fall back on prior guesses (BBR).

Across 13 models, attribute inference is strong and refusals are rare. Chain-of-Thought prompting makes the leakage worse. A simple defensive system prompt helps some models but does little for physiological traits. Together, these results show that ALLMs need privacy alignment aimed specifically at what they can learn from acoustic signals.

## Key points
- **Data sources:** HearSay draws on public lectures plus the NISP, VocalSound and VoxCeleb2 datasets. The speech is mostly in English, and the code and data are released for research use only.
- **Accuracy:** Gender is inferred with 92.89% accuracy on average, and overall audio-based inference beats random baselines by 46.4%. Qwen3-Omni-Flash identifies Income correctly 61.19% of the time.
- **Refusals:** Most open-source models almost never refuse, and Gender refusal stays near 0% for every model. GPT-4o-Audio is the exception, reaching 100% ARR on Weight.
- **Chain-of-Thought:** On stronger models, CoT prompting improves Accent inference by 22.1%.
- **Prompt defense:** A defensive system prompt raises ARR to 61.42% for MiniCPM-o-2.6 and 59.85% for Kimi-Audio. MERaLION barely responds to it, and Gender inference remains largely unaffected.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,220 works · [🎙️ Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) · 480 works

## Cite

Cite as: Jin Wang, Kaiwen Luo, Liang Lin, Weiliu Wang, Yitian Chen, Moayad Aloqaily, Xuehai Tang, Zhenhong Zhou, Kun Wang, Li Sun and Qingsong Wen. 2026. HearSay Benchmark: Do Audio LLMs Leak What They Hear?. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wang2026hearsay,
  title = {{HearSay Benchmark: Do Audio LLMs Leak What They Hear?}},
  author = {Jin Wang, Kaiwen Luo, Liang Lin, Weiliu Wang, Yitian Chen, Moayad Aloqaily, Xuehai Tang, Zhenhong Zhou, Kun Wang, Li Sun and Qingsong Wen},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.964},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.964}
}
```

- [← previous Headlines You Won't Forget: Can Pronoun Insertion Increase Memorability?](https://md.research.elamur.ai/free/en/papers/meyer-2026-headlines-you-won-t-forget/)
- [next → HEBATRON: A Hebrew-Specialized Open-Weight Mixture-of-Experts Language Model](https://md.research.elamur.ai/free/en/papers/kayzer-2026-hebatron-hebrew-specialized-open-weight/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/32/)

## References

65 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, and 1 others. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
2. Caridad Arroyo Arevalo, Sayedeh Leila Noorbakhsh, Yun Dong, Yuan Hong, and Binghui Wang. 2024. Task-agnostic privacy-preserving representation learning for federated learning against attribute inference attacks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 10909… [find](https://scholar.google.com/scholar?q=Caridad+Arroyo+Arevalo%2C+Sayedeh+Leila+Noorbakhsh%2C+Yun+Dong%2C+Yuan+Hong%2C+and+Binghui+Wang%2E+2024%2E+Task%2Dagnostic+privacy%2Dpreserving+representation+learning+for+federated+learning+against+attribute+inferen)
3. Tom Bäckström. 2025. Privacy in speech technology. Proceedings of the IEEE. [find](https://scholar.google.com/scholar?q=Tom+B%C3%A4ckstr%C3%B6m%2E+2025%2E+Privacy+in+speech+technology%2E+Proceedings+of+the+IEEE%2E)
4. Nicholas Carlini, Daphne Ippolito, Matthew Jagielski, Katherine Lee, Florian Tramèr, and Chiyuan Zhang. 2022. Quantifying memorization across neural language models. ArXiv, abs/2202.07646. [find](https://scholar.google.com/scholar?q=Nicholas+Carlini%2C+Daphne+Ippolito%2C+Matthew+Jagielski%2C+Katherine+Lee%2C+Florian+Tram%C3%A8r%2C+and+Chiyuan+Zhang%2E+2022%2E+Quantifying+memorization+across+neural+language+models%2E+ArXiv%2C+abs%2F2202%2E07646%2E)
5. Nicholas Carlini, Florian Tramer, Eric Wallace, Matthew Jagielski, Ariel Herbert-Voss, Katherine Lee, Adam Roberts, Tom Brown, Dawn Song, Ulfar Erlingsson, and 1 others. 2021. [Extracting training data from large language models](https://md.research.elamur.ai/free/en/papers/carlini-2020-extracting-training-data-large-language/). In 30th USENIX security symposium (USENIX Security 21), pages 2633–2650…
6. Yunfei Chu, Jin Xu, Qian Yang, Haojie Wei, Xipin Wei, Zhifang Guo, Yichong Leng, Yuanjun Lv, Jinzheng He, Junyang Lin, and 1 others. 2024. [Qwen2-audio technical report](https://md.research.elamur.ai/free/en/papers/chu-2024-qwen2-audio-technical-report/). arXiv preprint arXiv:2407.10759.
7. Joon Son Chung, Arsha Nagrani, and Andrew Zisserman. 2018. Voxceleb2: Deep speaker recognition. arXiv preprint arXiv:1806.05622. [find](https://scholar.google.com/scholar?q=Joon+Son+Chung%2C+Arsha+Nagrani%2C+and+Andrew+Zisserman%2E+2018%2E+Voxceleb2%3A+Deep+speaker+recognition%2E+arXiv+preprint+arXiv%3A1806%2E05622%2E)
8. Keqi Deng, Wenxi Chen, Xie Chen, and Phil Woodland. 2025. Simuls2s-llm: Unlocking simultaneous inference of speech llms for speech-to-speech translation. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16718–16734. [find](https://scholar.google.com/scholar?q=Keqi+Deng%2C+Wenxi+Chen%2C+Xie+Chen%2C+and+Phil+Woodland%2E+2025%2E+Simuls2s%2Dllm%3A+Unlocking+simultaneous+inference+of+speech+llms+for+speech%2Dto%2Dspeech+translation%2E+In+Proceedings+of+the+63rd+Annual+Meeting+of+t)
9. Junhao Dong, Piotr Koniusz, Xinghua Qu, and Yew-Soon Ong. 2025a. Stabilizing modality gap & lowering gradient norms improve zero-shot adversarial robustness of vlms. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 1, pages 236–247. [find](https://scholar.google.com/scholar?q=Junhao+Dong%2C+Piotr+Koniusz%2C+Xinghua+Qu%2C+and+Yew%2DSoon+Ong%2E+2025a%2E+Stabilizing+modality+gap+%26+lowering+gradient+norms+improve+zero%2Dshot+adversarial+robustness+of+vlms%2E+In+Proceedings+of+the+31st+ACM+SIG)
10. Junhao Dong, Cong Zhang, Xinghua Qu, Zejun Ma, Piotr Koniusz, and Yew-Soon Ong. 2025b. Robust superalignment: Weak-to-strong robustness generalization for vision-language models. In The Thirty-ninth Annual Conference on Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Junhao+Dong%2C+Cong+Zhang%2C+Xinghua+Qu%2C+Zejun+Ma%2C+Piotr+Koniusz%2C+and+Yew%2DSoon+Ong%2E+2025b%2E+Robust+superalignment%3A+Weak%2Dto%2Dstrong+robustness+generalization+for+vision%2Dlanguage+models%2E+In+The+Thirty%2Dninth+A)
11. Yihong Dong, Xue Jiang, Jiaru Qian, Tian Wang, Kechi Zhang, Zhi Jin, and Ge Li. 2025c. A survey on code generation with llm-based agents. arXiv preprint arXiv:2508.00083. — [arXiv:2508.00083](https://arxiv.org/abs/2508.00083) [find](https://scholar.google.com/scholar?q=Yihong+Dong%2C+Xue+Jiang%2C+Jiaru+Qian%2C+Tian+Wang%2C+Kechi+Zhang%2C+Zhi+Jin%2C+and+Ge+Li%2E+2025c%2E+A+survey+on+code+generation+with+llm%2Dbased+agents%2E+arXiv+preprint+arXiv%3A2508%2E00083%2E+%E2%80%94+%5BarXiv%3A2508%2E00083%5D%28https%3A%2F%2F)
12. Yexing Du, Ziyang Ma, Yifan Yang, Keqi Deng, Xie Chen, Bo Yang, Yang Xiang, Ming Liu, and Bing Qin. 2024a. Cot-st: Enhancing llm-based speech translation with multimodal chain-of-thought. arXiv preprint arXiv:2409.19510. — [arXiv:2409.19510](https://arxiv.org/abs/2409.19510) [find](https://scholar.google.com/scholar?q=Yexing+Du%2C+Ziyang+Ma%2C+Yifan+Yang%2C+Keqi+Deng%2C+Xie+Chen%2C+Bo+Yang%2C+Yang+Xiang%2C+Ming+Liu%2C+and+Bing+Qin%2E+2024a%2E+Cot%2Dst%3A+Enhancing+llm%2Dbased+speech+translation+with+multimodal+chain%2Dof%2Dthought%2E+arXiv+prepri)
13. Zhihao Du, Yuxuan Wang, Qian Chen, Xian Shi, Xiang Lv, Tianyu Zhao, Zhifu Gao, Yexin Yang, Changfeng Gao, Hui Wang, and 1 others. 2024b. Cosyvoice 2: Scalable streaming speech synthesis with large language models. arXiv preprint arXiv:2412.10117. [find](https://scholar.google.com/scholar?q=Zhihao+Du%2C+Yuxuan+Wang%2C+Qian+Chen%2C+Xian+Shi%2C+Xiang+Lv%2C+Tianyu+Zhao%2C+Zhifu+Gao%2C+Yexin+Yang%2C+Changfeng+Gao%2C+Hui+Wang%2C+and+1+others%2E+2024b%2E+Cosyvoice+2%3A+Scalable+streaming+speech+synthesis+with+large+lan)
14. Guy Fagherazzi, Aurélie Fischer, Muhannad Ismael, and Vladimir Despotovic. 2021. Voice for health: the use of vocal biomarkers from research to clinical practice. Digital biomarkers, 5(1):78–88. [find](https://scholar.google.com/scholar?q=Guy+Fagherazzi%2C+Aur%C3%A9lie+Fischer%2C+Muhannad+Ismael%2C+and+Vladimir+Despotovic%2E+2021%2E+Voice+for+health%3A+the+use+of+vocal+biomarkers+from+research+to+clinical+practice%2E+Digital+biomarkers%2C+5%281%29%3A78%E2%80%9388%2E)
15. Wenjie Fu, Huandong Wang, Chen Gao, Guanghua Liu, Yong Li, and Tao Jiang. 2024. Membership inference attacks against fine-tuned large language models via self-prompt calibration. Advances in Neural Information Processing Systems, 37:134981–135010. [find](https://scholar.google.com/scholar?q=Wenjie+Fu%2C+Huandong+Wang%2C+Chen+Gao%2C+Guanghua+Liu%2C+Yong+Li%2C+and+Tao+Jiang%2E+2024%2E+Membership+inference+attacks+against+fine%2Dtuned+large+language+models+via+self%2Dprompt+calibration%2E+Advances+in+Neural+In)
16. Yuan Gong, Jin Yu, and James Glass. 2022. Vocalsound: A dataset for improving human vocal sounds recognition. In ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 151–155. [find](https://scholar.google.com/scholar?q=Yuan+Gong%2C+Jin+Yu%2C+and+James+Glass%2E+2022%2E+Vocalsound%3A+A+dataset+for+improving+human+vocal+sounds+recognition%2E+In+ICASSP+2022+%2D+2022+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Proces)
17. Yuxin Guo, Shuailei Ma, Shijie Ma, Xiaoyi Bao, Chen-Wei Xie, Kecheng Zheng, Tingyu Weng, Siyang Sun, Yun Zheng, and Wei Zou. 2025. Aligned better, listen better for audio-visual large language models. arXiv preprint arXiv:2504.02061. — [arXiv:2504.02061](https://arxiv.org/abs/2504.02061) [find](https://scholar.google.com/scholar?q=Yuxin+Guo%2C+Shuailei+Ma%2C+Shijie+Ma%2C+Xiaoyi+Bao%2C+Chen%2DWei+Xie%2C+Kecheng+Zheng%2C+Tingyu+Weng%2C+Siyang+Sun%2C+Yun+Zheng%2C+and+Wei+Zou%2E+2025%2E+Aligned+better%2C+listen+better+for+audio%2Dvisual+large+language+models%2E)
18. Emman Haider, Daniel Perez-Becker, Thomas Portet, Piyush Madan, Amit Garg, Atabak Ashfaq, David Majercak, Wen Wen, Dongwoo Kim, Ziyi Yang, and 1 others. 2024. [Phi-3 safety post-training: Aligning language models with a" break-fix" cycle](https://md.research.elamur.ai/free/en/papers/haider-2024-phi-3-safety-post-training/). arXiv preprint arXiv:2407.13833.
19. Valentin Hartmann, Anshuman Suri, Vincent Bindschaedler, David Evans, Shruti Tople, and Robert West. 2023. Sok: Memorization in general-purpose large language models. arXiv preprint arXiv:2310.18362. — [arXiv:2310.18362](https://arxiv.org/abs/2310.18362) [find](https://scholar.google.com/scholar?q=Valentin+Hartmann%2C+Anshuman+Suri%2C+Vincent+Bindschaedler%2C+David+Evans%2C+Shruti+Tople%2C+and+Robert+West%2E+2023%2E+Sok%3A+Memorization+in+general%2Dpurpose+large+language+models%2E+arXiv+preprint+arXiv%3A2310%2E18362%2E+)
20. Luxi He, Xiangyu Qi, Michel Liao, Inyoung Cheong, Prateek Mittal, Danqi Chen, and Peter Henderson. 2025. The deployment of end-to-end audio language models should take into account the principle of least privilege. arXiv e-prints, pages arXiv–2503. [find](https://scholar.google.com/scholar?q=Luxi+He%2C+Xiangyu+Qi%2C+Michel+Liao%2C+Inyoung+Cheong%2C+Prateek+Mittal%2C+Danqi+Chen%2C+and+Peter+Henderson%2E+2025%2E+The+deployment+of+end%2Dto%2Dend+audio+language+models+should+take+into+account+the+principle+of+le)
21. Yingxu He, Zhuohan Liu, Shuo Sun, Bin Wang, Wenyu Zhang, Xunlong Zou, Nancy F Chen, and Ai Ti Aw. 2024. Meralion-audiollm: Bridging audio and language with large language models. arXiv preprint arXiv:2412.09818. — [arXiv:2412.09818](https://arxiv.org/abs/2412.09818) [find](https://scholar.google.com/scholar?q=Yingxu+He%2C+Zhuohan+Liu%2C+Shuo+Sun%2C+Bin+Wang%2C+Wenyu+Zhang%2C+Xunlong+Zou%2C+Nancy+F+Chen%2C+and+Ai+Ti+Aw%2E+2024%2E+Meralion%2Daudiollm%3A+Bridging+audio+and+language+with+large+language+models%2E+arXiv+preprint+arXiv%3A)
22. Qi Hu and Yangqiu Song. 2024. User consented federated recommender system against personalized attribute inference attack. In Proceedings of the 17th ACM International Conference on Web Search and Data Mining, pages 276–285. [find](https://scholar.google.com/scholar?q=Qi+Hu+and+Yangqiu+Song%2E+2024%2E+User+consented+federated+recommender+system+against+personalized+attribute+inference+attack%2E+In+Proceedings+of+the+17th+ACM+International+Conference+on+Web+Search+and+Dat)
23. Shengding Hu, Yuge Tu, Xu Han, Chaoqun He, Ganqu Cui, Xiang Long, Zhi Zheng, Yewei Fang, Yuxiang Huang, Weilin Zhao, and 1 others. 2024. [Minicpm: Unveiling the potential of small language models with scalable training strategies](https://md.research.elamur.ai/free/en/papers/hu-2024-minicpm-unveiling-potential-small-language/). arXiv preprint arXiv:2404.06395.
24. Ailin Huang, Boyong Wu, Bruce Wang, Chao Yan, Chen Hu, Chengli Feng, Fei Tian, Feiyu Shen, Jingbei Li, Mingrui Chen, and 1 others. 2025. [Step-audio: Unified understanding and generation in intelligent speech interaction](https://md.research.elamur.ai/free/en/papers/huang-2025-step-audio-unified-understanding/). arXiv preprint arXiv:2502.11946.
25. Zhifang Guo Jin Xu and Jinzheng Hen et al. 2025. [Qwen2.5-omni technical report](https://md.research.elamur.ai/free/en/papers/xu-2025-qwen2-5-omni-technical/). arXiv preprint arXiv:2503.20215.
26. Muhammad Mohsin Kabir, Muhammad Firoz Mridha, Jungpil Shin, Israt Jahan, and Abu Quwsar Ohi. 2021. A survey of speaker recognition: Fundamental theories, recognition methods and opportunities. Ieee Access, 9:79236–79263. [find](https://scholar.google.com/scholar?q=Muhammad+Mohsin+Kabir%2C+Muhammad+Firoz+Mridha%2C+Jungpil+Shin%2C+Israt+Jahan%2C+and+Abu+Quwsar+Ohi%2E+2021%2E+A+survey+of+speaker+recognition%3A+Fundamental+theories%2C+recognition+methods+and+opportunities%2E+Ieee+Ac)
27. Shareef Babu Kalluri, Deepu Vijayasenan, Sriram Ganapathy, Prashant Krishnan, and 1 others. 2021. Nisp: A multi-lingual multi-accent dataset for speaker profiling. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6953–6957. IEEE. [find](https://scholar.google.com/scholar?q=Shareef+Babu+Kalluri%2C+Deepu+Vijayasenan%2C+Sriram+Ganapathy%2C+Prashant+Krishnan%2C+and+1+others%2E+2021%2E+Nisp%3A+A+multi%2Dlingual+multi%2Daccent+dataset+for+speaker+profiling%2E+In+ICASSP+2021%2D2021+IEEE+Internation)
28. Mintong Kang, Chejian Xu, and Bo Li. 2024. Advwave: Stealthy adversarial jailbreak attack against large audio-language models. arXiv preprint arXiv:2412.08608. the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Vo… — [arXiv:2412.08608](https://arxiv.org/abs/2412.08608) [find](https://scholar.google.com/scholar?q=Mintong+Kang%2C+Chejian+Xu%2C+and+Bo+Li%2E+2024%2E+Advwave%3A+Stealthy+adversarial+jailbreak+attack+against+large+audio%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2412%2E08608%2E+the+2025+Conference+of+the+Nations+of+th)
29. KimiTeam, Ding Ding, Zeqian Ju, Yichong Leng, Songxiang Liu, Tong Liu, Zeyu Shang, Kai Shen, Wei Song, Xu Tan, Heyi Tang, Zhengtao Wang, Chu Wei, Yifei Xin, Xinran Xu, Jianwei Yu, Yutao Zhang, Xinyu Zhou, Y. Charles, and 21 others. 2025. [Kimi-audio technical report](https://md.research.elamur.ai/free/en/papers/kimiteam-2025-kimi-audio-technical-report/). Preprint, arXiv:2504.18425.
30. Ziyang Ma, Guanrou Yang, Yifan Yang, Zhifu Gao, Jiaming Wang, Zhihao Du, Fan Yu, Qian Chen, Siqi Zheng, Shiliang Zhang, and 1 others. 2025. Speech recognition meets large language model: Benchmarking, models, and exploration. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 3… [find](https://scholar.google.com/scholar?q=Ziyang+Ma%2C+Guanrou+Yang%2C+Yifan+Yang%2C+Zhifu+Gao%2C+Jiaming+Wang%2C+Zhihao+Du%2C+Fan+Yu%2C+Qian+Chen%2C+Siqi+Zheng%2C+Shiliang+Zhang%2C+and+1+others%2E+2025%2E+Speech+recognition+meets+large+language+model%3A+Benchmarking%2C)
31. Robert Kirk, Ishita Mediratta, Christoforos Nalmpantis, Jelena Luketina, Eric Hambro, Edward Grefenstette, and Roberta Raileanu. 2023. [Understanding the effects of rlhf on llm generalisation and diversity](https://md.research.elamur.ai/free/en/papers/kirk-2023-understanding-effects-rlhf-llm-generalisation/). arXiv preprint arXiv:2310.06452.
32. Tamera Lanham, Anna Chen, Ansh Radhakrishnan, Benoit Steiner, Carson Denison, Danny Hernandez, Dustin Li, Esin Durmus, Evan Hubinger, Jackson Kernion, and 1 others. 2023. [Measuring faithfulness in chain-of-thought reasoning](https://md.research.elamur.ai/free/en/papers/lanham-2023-measuring-faithfulness-chain-thought-reasoning/). arXiv preprint arXiv:2307.13702.
33. Haoyang Li, Lixu Wang, Xiaofeng Wang, Xinfeng Li, Dong Yang, Wei Dong, and Kaixiang Yao. 2025a. The man behind the sound: Demystifying audio private attribute profiling via multimodal large language model agents. Preprint, arXiv:2507.10016. — [arXiv:2507.10016](https://arxiv.org/abs/2507.10016) [find](https://scholar.google.com/scholar?q=Haoyang+Li%2C+Lixu+Wang%2C+Xiaofeng+Wang%2C+Xinfeng+Li%2C+Dong+Yang%2C+Wei+Dong%2C+and+Kaixiang+Yao%2E+2025a%2E+The+man+behind+the+sound%3A+Demystifying+audio+private+attribute+profiling+via+multimodal+large+language+m)
34. Zeping Min and Jinbo Wang. 2023. Exploring the integration of large language models into automatic speech recognition systems: An empirical study. In International Conference on Neural Information Processing, pages 69–84. Springer. [find](https://scholar.google.com/scholar?q=Zeping+Min+and+Jinbo+Wang%2E+2023%2E+Exploring+the+integration+of+large+language+models+into+automatic+speech+recognition+systems%3A+An+empirical+study%2E+In+International+Conference+on+Neural+Information+Pro)
35. Shervin Minaee, Amirali Abdolrashidi, Hang Su, Mohammed Bennamoun, and David Zhang. 2023. Biometrics recognition using deep learning: A survey. Artificial Intelligence Review, 56(8):8647–8695. [find](https://scholar.google.com/scholar?q=Shervin+Minaee%2C+Amirali+Abdolrashidi%2C+Hang+Su%2C+Mohammed+Bennamoun%2C+and+David+Zhang%2E+2023%2E+Biometrics+recognition+using+deep+learning%3A+A+survey%2E+Artificial+Intelligence+Review%2C+56%288%29%3A8647%E2%80%938695%2E)
36. Daye Nam, Andrew Macvean, Vincent Hellendoorn, Bogdan Vasilescu, and Brad Myers. 2024. Using an llm to help with code understanding. In Proceedings of the IEEE/ACM 46th International Conference on Software Engineering, pages 1–13. [find](https://scholar.google.com/scholar?q=Daye+Nam%2C+Andrew+Macvean%2C+Vincent+Hellendoorn%2C+Bogdan+Vasilescu%2C+and+Brad+Myers%2E+2024%2E+Using+an+llm+to+help+with+code+understanding%2E+In+Proceedings+of+the+IEEE%2FACM+46th+International+Conference+on+Sof)
37. Junhao Li, Jiahao Chen, Zhou Feng, and Chunyi Zhou. 2025b. Auditing m-llms for privacy risks: A synthetic benchmark and evaluation framework. arXiv preprint arXiv:2511.03248. — [arXiv:2511.03248](https://arxiv.org/abs/2511.03248) [find](https://scholar.google.com/scholar?q=Junhao+Li%2C+Jiahao+Chen%2C+Zhou+Feng%2C+and+Chunyi+Zhou%2E+2025b%2E+Auditing+m%2Dllms+for+privacy+risks%3A+A+synthetic+benchmark+and+evaluation+framework%2E+arXiv+preprint+arXiv%3A2511%2E03248%2E+%E2%80%94+%5BarXiv%3A2511%2E03248%5D%28http)
38. Bosen Rao, Jiale Zhang, Di Wu, Chengcheng Zhu, Xiaobing Sun, and Bing Chen. 2024. Privacy inference attack and defense in centralized and federated learning: A comprehensive survey. IEEE Transactions on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Bosen+Rao%2C+Jiale+Zhang%2C+Di+Wu%2C+Chengcheng+Zhu%2C+Xiaobing+Sun%2C+and+Bing+Chen%2E+2024%2E+Privacy+inference+attack+and+defense+in+centralized+and+federated+learning%3A+A+comprehensive+survey%2E+IEEE+Transactions+)
39. Kai Li, Can Shen, Yile Liu, Jirui Han, Kelong Zheng, Xuechao Zou, Zhe Wang, Shun Zhang, Xingjian Du, Hanjun Luo, and 1 others. 2025c. Au-diotrust: Benchmarking the multifaceted trustworthiness of audio large language models. arXiv preprint arXiv:2505.16211. — [arXiv:2505.16211](https://arxiv.org/abs/2505.16211) [find](https://scholar.google.com/scholar?q=Kai+Li%2C+Can+Shen%2C+Yile+Liu%2C+Jirui+Han%2C+Kelong+Zheng%2C+Xuechao+Zou%2C+Zhe+Wang%2C+Shun+Zhang%2C+Xingjian+Du%2C+Hanjun+Luo%2C+and+1+others%2E+2025c%2E+Au%2Ddiotrust%3A+Benchmarking+the+multifaceted+trustworthiness+of+audi)
40. Kyeongha Rho, Hyeongkeun Lee, Valentio Iverson, and Joon Son Chung. 2025. Lavcap: Llm-based audiovisual captioning using optimal transport. In ICASSP 2025-2025 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 1–5. IEEE. [find](https://scholar.google.com/scholar?q=Kyeongha+Rho%2C+Hyeongkeun+Lee%2C+Valentio+Iverson%2C+and+Joon+Son+Chung%2E+2025%2E+Lavcap%3A+Llm%2Dbased+audiovisual+captioning+using+optimal+transport%2E+In+ICASSP+2025%2D2025+IEEE+International+Conference+on+Acousti)
41. Xinfeng Li, Kai Li, Yifan Zheng, Chen Yan, Xiaoyu Ji, and Wenyuan Xu. 2024. Safeear: Content privacy-preserving audio deepfake detection. In Proceedings of the 2024 on ACM SIGSAC Conference on Computer and Communications Security, pages 3585– 3599. [find](https://scholar.google.com/scholar?q=Xinfeng+Li%2C+Kai+Li%2C+Yifan+Zheng%2C+Chen+Yan%2C+Xiaoyu+Ji%2C+and+Wenyuan+Xu%2E+2024%2E+Safeear%3A+Content+privacy%2Dpreserving+audio+deepfake+detection%2E+In+Proceedings+of+the+2024+on+ACM+SIGSAC+Conference+on+Compute)
42. Md Abu Sayed, Maliha Tayaba, MD Islam, Md Eyasin Ul Islam Pavel, Md Tuhin Mia, Eftekhar Hossain Ayon, Nur Nob, and Bishnu Padh Ghosh. 2023. Parkinson’s disease detection through vocal biomarkers and advanced machine learning algorithms. arXiv preprint arXiv:2311.05435. — [arXiv:2311.05435](https://arxiv.org/abs/2311.05435) [find](https://scholar.google.com/scholar?q=Md+Abu+Sayed%2C+Maliha+Tayaba%2C+MD+Islam%2C+Md+Eyasin+Ul+Islam+Pavel%2C+Md+Tuhin+Mia%2C+Eftekhar+Hossain+Ayon%2C+Nur+Nob%2C+and+Bishnu+Padh+Ghosh%2E+2023%2E+Parkinson%E2%80%99s+disease+detection+through+vocal+biomarkers+and+a)
43. Yifan Li, Yifan Du, Kun Zhou, Jinpeng Wang, Wayne Xin Zhao, and Ji-Rong Wen. 2023. [Evaluating object hallucination in large vision-language models](https://md.research.elamur.ai/free/en/papers/li-2023-evaluating-object-hallucination-large-vision/). arXiv preprint arXiv:2305.10355.
44. Brij Mohan Lal Srivastava, Aurélien Bellet, Marc Tommasi, and Emmanuel Vincent. 2019. Privacy-preserving adversarial representation learning in asr: Reality or illusion? arXiv preprint arXiv:1911.04913. — [arXiv:1911.04913](https://arxiv.org/abs/1911.04913) [find](https://scholar.google.com/scholar?q=Brij+Mohan+Lal+Srivastava%2C+Aur%C3%A9lien+Bellet%2C+Marc+Tommasi%2C+and+Emmanuel+Vincent%2E+2019%2E+Privacy%2Dpreserving+adversarial+representation+learning+in+asr%3A+Reality+or+illusion%3F+arXiv+preprint+arXiv%3A1911%2E0491)
45. Liang Lin, Miao Yu, Kaiwen Luo, Yibo Zhang, Lilan Peng, Dexian Wang, Xuehai Tang, Yuanhe Zhang, Xikang Yang, Zhenhong Zhou, and 1 others. 2025. Hidden in the noise: Unveiling backdoors in audio llms alignment through latent acoustic pattern triggers. arXiv preprint arXiv:2508.02175. — [arXiv:2508.02175](https://arxiv.org/abs/2508.02175) [find](https://scholar.google.com/scholar?q=Liang+Lin%2C+Miao+Yu%2C+Kaiwen+Luo%2C+Yibo+Zhang%2C+Lilan+Peng%2C+Dexian+Wang%2C+Xuehai+Tang%2C+Yuanhe+Zhang%2C+Xikang+Yang%2C+Zhenhong+Zhou%2C+and+1+others%2E+2025%2E+Hidden+in+the+noise%3A+Unveiling+backdoors+in+audio+llms+a)
46. Zheyuan Liu, Guangyao Dou, Mengzhao Jia, Zhaoxuan Tan, Qingkai Zeng, Yongle Yuan, and Meng Jiang. 2025. [Protecting privacy in multimodal large language models with mllmu-bench](https://md.research.elamur.ai/free/en/papers/liu-2025-protecting-privacy-multimodal-large-language/). In Proceedings of Robin Staab, Mark Vero, Mislav Balunovi’c, and Martin T. Vechev. 2023. Beyond memorization: Violating pri…
47. Panjun Sun, Shigen Shen, Yi Wan, Zongda Wu, Zhaoxi Fang, and Xiao-zhi Gao. 2024. A survey of iot privacy security: Architecture, technology, challenges, and trends. IEEE internet of things journal, 11(21):34567–34591. [find](https://scholar.google.com/scholar?q=Panjun+Sun%2C+Shigen+Shen%2C+Yi+Wan%2C+Zongda+Wu%2C+Zhaoxi+Fang%2C+and+Xiao%2Dzhi+Gao%2E+2024%2E+A+survey+of+iot+privacy+security%3A+Architecture%2C+technology%2C+challenges%2C+and+trends%2E+IEEE+internet+of+things+journal%2C+11)
48. Naohiro Tawara, Atsunori Ogawa, Yuki Kitagishi, and Hosana Kamiyama. 2021. Age-vox-celeb: Multimodal corpus for facial and speech estimation. In ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 6963–6967. IEEE. [find](https://scholar.google.com/scholar?q=Naohiro+Tawara%2C+Atsunori+Ogawa%2C+Yuki+Kitagishi%2C+and+Hosana+Kamiyama%2E+2021%2E+Age%2Dvox%2Dceleb%3A+Multimodal+corpus+for+facial+and+speech+estimation%2E+In+ICASSP+2021%2D2021+IEEE+International+Conference+on+Acous)
49. Gemini Team, Rohan Anil, Sebastian Borgeaud, Jean-Baptiste Alayrac, Jiahui Yu, Radu Soricut, Johan Schalkwyk, Andrew M Dai, Anja Hauth, Katie Millican, and 1 others. 2023. [Gemini: a family of highly capable multimodal models](https://md.research.elamur.ai/free/en/papers/gemini-team-2023-gemini-family-highly-capable-multimodal/). arXiv preprint arXiv:2312.11805.
50. Zhang, Hongkun Hao, Zishan Guo, and 19 others. 2025. [Qwen3-omni technical report](https://md.research.elamur.ai/free/en/papers/xu-2025-qwen3-omni-technical-report/). arXiv preprint arXiv:2509.17765.
51. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, and 1 others. 2025. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
52. Qwen Team. 2024. Qwen-omni-turbo. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+2024%2E+Qwen%2Domni%2Dturbo%2E)
53. Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. 2024a. [A survey on large language model (llm) security and privacy: The good, the bad, and the ugly](https://md.research.elamur.ai/free/en/papers/yao-2023-survey-large-language-model-llm/). High-Confidence Computing, 4(2):100211.
54. Francisco Teixeira, Karla Pizzi, Raphael Olivier, Alberto Abad, Bhiksha Raj, and Isabel Trancoso. 2026. Exploring features for membership inference in asr model auditing. Computer Speech & Language, 95:101812. [find](https://scholar.google.com/scholar?q=Francisco+Teixeira%2C+Karla+Pizzi%2C+Raphael+Olivier%2C+Alberto+Abad%2C+Bhiksha+Raj%2C+and+Isabel+Trancoso%2E+2026%2E+Exploring+features+for+membership+inference+in+asr+model+auditing%2E+Computer+Speech+%26+Language%2C+9)
55. Yuan Yao, Tianyu Yu, Ao Zhang, Chongyi Wang, Junbo Cui, Hongji Zhu, Tianchi Cai, Haoyu Li, Weilin Zhao, Zhihui He, and 1 others. 2024b. [Minicpm-v: A gpt-4v level mllm on your phone](https://md.research.elamur.ai/free/en/papers/yao-2024-minicpm-v-gpt-4v/). arXiv preprint arXiv:2408.01800.
56. Boshi Wang, Sewon Min, Xiang Deng, Jiaming Shen, You Wu, Luke Zettlemoyer, and Huan Sun. 2023a. [Towards understanding chain-of-thought prompting: An empirical study of what matters](https://md.research.elamur.ai/free/en/papers/wang-2023-understanding-chain-thought-prompting-empirical/). In Proceedings of the 61st annual meeting of the association for computational linguistics (volume 1: Long papers), pa…
57. Dongxu Zhang, Yiding Sun, Pengcheng Li, Yumou Liu, Hongqiang Lin, Haoran Xu, Xiaoxuan Mu, Liang Lin, Wenbiao Yan, Ning Yang, and 1 others. 2026a. Pointcot: A multi-modal benchmark for explicit 3d geometric reasoning. arXiv preprint arXiv:2602.23945. — [arXiv:2602.23945](https://arxiv.org/abs/2602.23945) [find](https://scholar.google.com/scholar?q=Dongxu+Zhang%2C+Yiding+Sun%2C+Pengcheng+Li%2C+Yumou+Liu%2C+Hongqiang+Lin%2C+Haoran+Xu%2C+Xiaoxuan+Mu%2C+Liang+Lin%2C+Wenbiao+Yan%2C+Ning+Yang%2C+and+1+others%2E+2026a%2E+Pointcot%3A+A+multi%2Dmodal+benchmark+for+explicit+3d+geom)
58. Kun Wang, Guibin Zhang, Zhenhong Zhou, Jiahao Wu, Miao Yu, Shiqian Zhao, Chenlong Yin, Jinhu Fu, Yibo Yan, Hanjun Luo, and 1 others. 2025. A comprehensive survey in llm (-agent) full stack safety: Data, training and deployment. arXiv preprint arXiv:2504.15585. — [arXiv:2504.15585](https://arxiv.org/abs/2504.15585) [find](https://scholar.google.com/scholar?q=Kun+Wang%2C+Guibin+Zhang%2C+Zhenhong+Zhou%2C+Jiahao+Wu%2C+Miao+Yu%2C+Shiqian+Zhao%2C+Chenlong+Yin%2C+Jinhu+Fu%2C+Yibo+Yan%2C+Hanjun+Luo%2C+and+1+others%2E+2025%2E+A+comprehensive+survey+in+llm+%28%2Dagent%29+full+stack+safety%3A+Dat)
59. Yuxia Wang, Haonan Li, Xudong Han, Preslav Nakov, and Timothy Baldwin. 2023b. [Do-notanswer: A dataset for evaluating safeguards in llms](https://md.research.elamur.ai/free/en/papers/wang-2023-answer-dataset-evaluating-safeguards-llms/). URL: https://arxiv.org/abs/2308.13387. [↗](https://arxiv.org/abs/2308.13387)
60. Taiba Majid Wani, Teddy Surya Gunawan, Syed Asif Ahmad Qadri, Mira Kartiwi, and Eliathamby Ambikairajah. 2021. A comprehensive review of speech emotion recognition systems. IEEE access, 9:47795–47814. [find](https://scholar.google.com/scholar?q=Taiba+Majid+Wani%2C+Teddy+Surya+Gunawan%2C+Syed+Asif+Ahmad+Qadri%2C+Mira+Kartiwi%2C+and+Eliathamby+Ambikairajah%2E+2021%2E+A+comprehensive+review+of+speech+emotion+recognition+systems%2E+IEEE+access%2C+9%3A47795%E2%80%9347814%2E)
61. Boyong Wu, Chao Yan, Chen Hu, Cheng Yi, Chengli Feng, Fei Tian, Feiyu Shen, Gang Yu, Haoyang Zhang, Jingbei Li, and 1 others. 2025. [Step-audio 2 technical report](https://md.research.elamur.ai/free/en/papers/wu-2025-step-audio-2-technical/). arXiv preprint arXiv:2507.16632.
62. Dongxu Zhang, Yiding Sun, Cheng Tan, Wenbiao Yan, Ning Yang, Jihua Zhu, and Haijun Zhang. 2026b. [Chain-of-thought compression should not be blind: V-skip for efficient multimodal reasoning via dual-path anchoring](https://md.research.elamur.ai/free/en/papers/zhang-2026-chain-thought-compression-should-not/). arXiv preprint arXiv:2601.13879.
63. Yibo Zhang and Liang Lin. 2025. Enj: Optimizing noise with genetic algorithms to jailbreak lsms. arXiv preprint arXiv:2509.11128. — [arXiv:2509.11128](https://arxiv.org/abs/2509.11128) [find](https://scholar.google.com/scholar?q=Yibo+Zhang+and+Liang+Lin%2E+2025%2E+Enj%3A+Optimizing+noise+with+genetic+algorithms+to+jailbreak+lsms%2E+arXiv+preprint+arXiv%3A2509%2E11128%2E+%E2%80%94+%5BarXiv%3A2509%2E11128%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2509%2E11128%29)
64. Zihao Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. [Calibrate before use: Improving few-shot performance of language models](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/). In International conference on machine learning, pages 12697–12706. PMLR.
65. Lianmin Zheng, Wei-Lin Chiang, Ying Sheng, Siyuan Zhuang, Zhanghao Wu, Yonghao Zhuang, Zi Lin, Zhuohan Li, Dacheng Li, Eric Xing, and 1 others. 2023. [Judging llm-as-a-judge with mt-bench and chatbot arena](https://md.research.elamur.ai/free/en/papers/zheng-2023-judging-llm-judge-mt-bench/). Advances in neural information processing systems, 36:46595–46623.

*Updated 1 November 2026*