---
title: "Multilingual Long-Form Speech Instruction — Ugan 2026"
description: "This paper describes the Karlsruhe Institute of Technology's entry to the unconstrained Long and Short Instruction Following tracks at IWSLT 2026."
authors: ["Enes Yavuz Ugan", "Maike Züfle", "Yuka Ko", "Supriti Sinhamahapatra", "Fabian Retkowski", "Seymanur Akti", "Jan Niehues", "Alexander Waibel"]
year: 2026
venue: "Proceedings of the 23rd International Conference on Spoken Language Translation (IWSLT 2026)"
source: "https://aclanthology.org/2026.iwslt-1.16/"
doi: "10.18653/v1/2026.iwslt-1.16"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/ugan-2026-multilingual-long-form-speech-instruction/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/ugan-2026-multilingual-long-form-speech-instruction/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2026](https://md.research.elamur.ai/free/papers/2026/page/3/) / Multilingual Long-Form Speech Instruction Following: KIT's Submission to IWSLT 2026

# Multilingual Long-Form Speech Instruction Following: KIT's Submission to IWSLT 2026

**Authors:** [Enes Yavuz Ugan](https://md.research.elamur.ai/free/authors/enes-yavuz-ugan/), [Maike Züfle](https://md.research.elamur.ai/free/authors/maike-zufle/), Yuka Ko, [Supriti Sinhamahapatra](https://md.research.elamur.ai/free/authors/supriti-sinhamahapatra/), [Fabian Retkowski](https://md.research.elamur.ai/free/authors/fabian-retkowski/), Seymanur Akti, [Jan Niehues](https://md.research.elamur.ai/free/authors/jan-niehues/), [Alexander Waibel](https://md.research.elamur.ai/free/authors/alexander-waibel/) · **Year:** 2026 · **Venue:** Proceedings of the 23rd International Conference on Spoken Language Translation (IWSLT 2026) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.iwslt-1.16/) · [PDF](https://aclanthology.org/2026.iwslt-1.16.pdf) · [DOI 10.18653/v1/2026.iwslt-1.16](https://doi.org/10.18653/v1/2026.iwslt-1.16) · [OpenAlex](https://openalex.org/W7166809944)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14) · 1 in our library, of the 140673 papers whose bibliographies we read

## Summary
This paper describes the Karlsruhe Institute of Technology's entry to the unconstrained Long and Short Instruction Following tracks at IWSLT 2026. The tracks cover English, German, Italian and Chinese. Systems must handle six tasks, including a surprise one, from natural-language prompts alone, and long-form inputs can last up to 15 minutes. Most available speech corpora contain short clips, so the authors built an augmentation pipeline with three steps. They concatenate segments while grouping by speaker, generate missing labels with LLMs, and translate English references with translategemma-12b-it. The result is a released corpus of over a million instances. Their main end-to-end model is Qwen2.5-Omni, fine-tuned with temperature-scaled task sampling, and they also tested cascaded setups. The analysis of re-ranking without known task identity is the most significant finding. Likelihood-based selection greatly helps ASR but harms semantic tasks, because it tends to favour outputs produced from segmented audio. Adding Minimum Bayes Risk scoring largely fixes this problem, which gives a practical default for multi-task speech systems.

## Key points
- The released dataset YapayNet/iwslt2026-if-augmented holds 1,048,158 instances. It draws on sources such as YTSeg, NUTSHELL, EuroParl-ST, LibriSpeech, CoVost, LibriSQA, MMSU and TowerBlocks.
- Temperature sampling with T = 2 rebalanced the task mix. For example, ASR rose from 1.84% to 6.12% of samples and SQA dropped from 45.31% to 30.41%.
- Conditioning on Chain-of-Thought task tokens failed. The model lost the ability to tell tasks apart when tasks were similar or unevenly represented.
- Plain likelihood re-ranking selected the SHAS-segmented candidate for 22.7% of SQA and 34.5% of SSUM cases. With Lik.+MBR these rates fell to 7.4% and 5.5%, and on English MCIF-long ASR the WER improved from 37.65 to 21.39.
- In the official results, neither submission was best on every task. The primary systems scored 0.0 accuracy on the unseen QE task, while the transcript-based contrastive system reached 0.722.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/shelves/machine-translation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ugan2026multilingual,
  title = {{Multilingual Long-Form Speech Instruction Following: KIT's Submission to IWSLT 2026}},
  author = {Ugan, Enes Yavuz and Züfle, Maike and Ko, Yuka and Sinhamahapatra, Supriti and Retkowski, Fabian and Akti, Seymanur and Niehues, Jan and Waibel, Alexander},
  booktitle = {Proceedings of the 23rd International Conference on Spoken Language Translation (IWSLT 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.iwslt-1.16},
  url = {https://doi.org/10.18653/v1/2026.iwslt-1.16}
}
```

- [← previous Multilingual Communication in the Asylum Context: Evaluating LLM-Based Machine Translation with Fuzzy Match Augmentation and Adaptive NMT across Resource Conditions under Low-Data Constraints](https://md.research.elamur.ai/free/papers/moerman-2026-multilingual-communication-asylum-context-evaluating/)
- [next → Multilingual Target-Stance Extraction](https://md.research.elamur.ai/free/papers/mines-2026-multilingual-target-stance-extraction/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/papers/2026/page/3/)

## References

40 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Marah Abdin, Jyoti Aneja, Harkirat Behl, Sébastien Bubeck, Ronen Eldan, Suriya Gunasekar, Michael Harrison, Russell J Hewett, Mojan Javaheripi, Piero Kauffmann, and 1 others. 2024. Phi-4 technical report. arXiv preprint arXiv:2412.08905. [find](https://scholar.google.com/scholar?q=Marah+Abdin%2C+Jyoti+Aneja%2C+Harkirat+Behl%2C+S%C3%A9bastien+Bubeck%2C+Ronen+Eldan%2C+Suriya+Gunasekar%2C+Michael+Harrison%2C+Russell+J+Hewett%2C+Mojan+Javaheripi%2C+Piero+Kauffmann%2C+and+1+others%2E+2024%2E+Phi%2D4+technical+rep) [our card](https://md.research.elamur.ai/free/papers/abdin-2024-phi-4-technical-report/)
2. David Ifeoluwa Adelani, Antonios Anastasopoulos, Victor Agostinelli, Luisa Bentivogli, Ondřej Bojar, Sebastien Bratières, Marine Carpuat, Roldano Cattoni, Mauro Cettolo, Lizhong Chen, Marcello Federico, Marco Gaido, Mahendra Gupta, HyoJung Han, Ali Hatami, David Javorský, Yejin Jeon, Marek Kaszteln… [find](https://scholar.google.com/scholar?q=David+Ifeoluwa+Adelani%2C+Antonios+Anastasopoulos%2C+Victor+Agostinelli%2C+Luisa+Bentivogli%2C+Ondr%CC%8Cej+Bojar%2C+Sebastien+Brati%C3%A8res%2C+Marine+Carpuat%2C+Roldano+Cattoni%2C+Mauro+Cettolo%2C+Lizhong+Chen%2C+Marcello+Federi)
3. Duarte M. Alves, José Pombal, Nuno M. Guerreiro, Pedro H. Martins, João Alves, Amin Farajian, Ben Peters, Ricardo Rei, Patrick Fernandes, Sweta Agrawal, Pierre Colombo, José G. C. de Souza, and André F. T. Martins. 2024. Tower: An open multilingual large language model for translation-related tasks.… [find](https://scholar.google.com/scholar?q=Duarte+M%2E+Alves%2C+Jos%C3%A9+Pombal%2C+Nuno+M%2E+Guerreiro%2C+Pedro+H%2E+Martins%2C+Jo%C3%A3o+Alves%2C+Amin+Farajian%2C+Ben+Peters%2C+Ricardo+Rei%2C+Patrick+Fernandes%2C+Sweta+Agrawal%2C+Pierre+Colombo%2C+Jos%C3%A9+G%2E+C%2E+de+Souza%2C+and+Andr%C3%A9+) [our card](https://md.research.elamur.ai/free/papers/alves-2024-tower-open-multilingual-large-language/)
4. Loïc Barrault, Yu-An Chung, Mariano Cora Meglioli, David Dale, Ning Dong, Paul-Ambroise Duquenne, Hady Elsahar, Hongyu Gong, Kevin Heffernan, John Hoffman, and 1 others. 2023. Seamlessm4t: Massively multilingual & multimodal machine translation. arXiv preprint arXiv:2308.11596. [find](https://scholar.google.com/scholar?q=Lo%C3%AFc+Barrault%2C+Yu%2DAn+Chung%2C+Mariano+Cora+Meglioli%2C+David+Dale%2C+Ning+Dong%2C+Paul%2DAmbroise+Duquenne%2C+Hady+Elsahar%2C+Hongyu+Gong%2C+Kevin+Heffernan%2C+John+Hoffman%2C+and+1+others%2E+2023%2E+Seamlessm4t%3A+Massively+m) [our card](https://md.research.elamur.ai/free/papers/communication-2023-seamlessm4t-massively-multilingual-multimodal-machine/)
5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. Bert: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies,… [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+Bert%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proceedings+of+the+2019+conference+of+the+N)
6. Mara Finkelstein, Isaac Caswell, Tobias Domhan, Jan-Thorsten Peter, Juraj Juraska, Parker Riley, Daniel Deutsch, Geza Kovacs, Cole Dilanni, Colin Cherry, and 1 others. 2026. Translategemma technical report. arXiv preprint arXiv:2601.09012. — [arXiv:2601.09012](https://arxiv.org/abs/2601.09012) [find](https://scholar.google.com/scholar?q=Mara+Finkelstein%2C+Isaac+Caswell%2C+Tobias+Domhan%2C+Jan%2DThorsten+Peter%2C+Juraj+Juraska%2C+Parker+Riley%2C+Daniel+Deutsch%2C+Geza+Kovacs%2C+Cole+Dilanni%2C+Colin+Cherry%2C+and+1+others%2E+2026%2E+Translategemma+technical+r)
7. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, and 1 others. 2022. Lora: Low-rank adaptation of large language models. Iclr, 1(2):3. [find](https://scholar.google.com/scholar?q=Edward+J+Hu%2C+Yelong+Shen%2C+Phillip+Wallis%2C+Zeyuan+Allen%2DZhu%2C+Yuanzhi+Li%2C+Shean+Wang%2C+Liang+Wang%2C+Weizhu+Chen%2C+and+1+others%2E+2022%2E+Lora%3A+Low%2Drank+adaptation+of+large+language+models%2E+Iclr%2C+1%282%29%3A3%2E)
8. Christian Huber, Tu Anh Dinh, Carlos Mullov, Ngoc-Quan Pham, Thai-Binh Nguyen, Fabian Retkowski, Stefan Constantin, Enes Ugan, Danni Liu, Zhaolin Li, and 1 others. 2023. End-to-end evaluation for low-latency simultaneous speech translation. In Proceedings of the 2023 Conference on Empirical Methods… [find](https://scholar.google.com/scholar?q=Christian+Huber%2C+Tu+Anh+Dinh%2C+Carlos+Mullov%2C+Ngoc%2DQuan+Pham%2C+Thai%2DBinh+Nguyen%2C+Fabian+Retkowski%2C+Stefan+Constantin%2C+Enes+Ugan%2C+Danni+Liu%2C+Zhaolin+Li%2C+and+1+others%2E+2023%2E+End%2Dto%2Dend+evaluation+for+low%2D)
9. Javier Iranzo-Sánchez, Joan Albert Silvestre-Cerdà, Javier Jorge, Nahuel Roselló, Adrià Giménez, Albert Sanchis, Jorge Civera, and Alfons Juan. 2020. [find](https://scholar.google.com/scholar?q=Javier+Iranzo%2DS%C3%A1nchez%2C+Joan+Albert+Silvestre%2DCerd%C3%A0%2C+Javier+Jorge%2C+Nahuel+Rosell%C3%B3%2C+Adri%C3%A0+Gim%C3%A9nez%2C+Albert+Sanchis%2C+Jorge+Civera%2C+and+Alfons+Juan%2E+2020%2E)
10. Europarl-st: A multilingual corpus for speech translation of parliamentary debates. In ICASSP 2020 -2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pages 8229– 8233. [find](https://scholar.google.com/scholar?q=Europarl%2Dst%3A+A+multilingual+corpus+for+speech+translation+of+parliamentary+debates%2E+In+ICASSP+2020+%2D2020+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29%2C+pages+8229%E2%80%93+)
11. Paul Kassianik, Baturay Saglam, Alexander Chen, Blaine Nelson, Anu Vellore, Massimo Aufiero, Fraser Burch, Dhruv Kedia, Avi Zohary, Sajana Weerawardhena, and 1 others. 2025. Llama-3.1-foundationai-securityllm-base-8b technical report. arXiv preprint arXiv:2504.21039. — [arXiv:2504.21039](https://arxiv.org/abs/2504.21039) [find](https://scholar.google.com/scholar?q=Paul+Kassianik%2C+Baturay+Saglam%2C+Alexander+Chen%2C+Blaine+Nelson%2C+Anu+Vellore%2C+Massimo+Aufiero%2C+Fraser+Burch%2C+Dhruv+Kedia%2C+Avi+Zohary%2C+Sajana+Weerawardhena%2C+and+1+others%2E+2025%2E+Llama%2D3%2E1%2Dfoundationai%2Dsec)
12. Sai Koneru, Fabian Retkowski, Christian Huber, Lukas Hilgert, Seymanur Akti, Enes Yavuz Ugan, Alex Waibel, and Jan Niehues. 2026. Boom: Beyond only one modality kit’s multimodal multilingual lecture companion. In Proceedings of the 19th Conference of the European Chapter of the Association for Compu… [find](https://scholar.google.com/scholar?q=Sai+Koneru%2C+Fabian+Retkowski%2C+Christian+Huber%2C+Lukas+Hilgert%2C+Seymanur+Akti%2C+Enes+Yavuz+Ugan%2C+Alex+Waibel%2C+and+Jan+Niehues%2E+2026%2E+Boom%3A+Beyond+only+one+modality+kit%E2%80%99s+multimodal+multilingual+lecture+c)
13. Sai Koneru, Maike Züfle, Thai-Binh Nguyen, Seymanur Akti, Jan Niehues, and Alex Waibel. 2025. Kit’s offline speech translation and instruction following submission for iwslt 2025. In Proceedings of the 22nd International Conference on Spoken Language Translation (IWSLT 2025), pages 232–244. [find](https://scholar.google.com/scholar?q=Sai+Koneru%2C+Maike+Z%C3%BCfle%2C+Thai%2DBinh+Nguyen%2C+Seymanur+Akti%2C+Jan+Niehues%2C+and+Alex+Waibel%2E+2025%2E+Kit%E2%80%99s+offline+speech+translation+and+instruction+following+submission+for+iwslt+2025%2E+In+Proceedings+of+th)
14. Oleksii Kuchaiev, Jason Li, Huyen Nguyen, Oleksii Hrinchuk, Ryan Leary, Boris Ginsburg, Samuel Kriman, Stanislav Beliaev, Vitaly Lavrukhin, Jack Cook, and 1 others. 2019. Nemo: a toolkit for building ai applications using neural modules. arXiv preprint arXiv:1909.09577. [find](https://scholar.google.com/scholar?q=Oleksii+Kuchaiev%2C+Jason+Li%2C+Huyen+Nguyen%2C+Oleksii+Hrinchuk%2C+Ryan+Leary%2C+Boris+Ginsburg%2C+Samuel+Kriman%2C+Stanislav+Beliaev%2C+Vitaly+Lavrukhin%2C+Jack+Cook%2C+and+1+others%2E+2019%2E+Nemo%3A+a+toolkit+for+building+)
15. Shankar Kumar and William Byrne. 2004. Minimum Bayes-risk decoding for statistical machine translation. In Proceedings of the Human Language Technology Conference of the North American Chapter of the Association for Computational Linguistics: HLT-NAACL 2004, pages 169–176, Boston, Massachusetts, USA… [find](https://scholar.google.com/scholar?q=Shankar+Kumar+and+William+Byrne%2E+2004%2E+Minimum+Bayes%2Drisk+decoding+for+statistical+machine+translation%2E+In+Proceedings+of+the+Human+Language+Technology+Conference+of+the+North+American+Chapter+of+the+) [our card](https://md.research.elamur.ai/free/papers/kumar-2004-minimum-bayes-risk-decoding-statistical/)
16. Lujun Li, Lama Sleem, Geoffrey Nichil, Radu State, and 1 others. 2025. Exploring the impact of temperature on large language models: Hot or cold? Procedia Computer Science, 264:242–251. [find](https://scholar.google.com/scholar?q=Lujun+Li%2C+Lama+Sleem%2C+Geoffrey+Nichil%2C+Radu+State%2C+and+1+others%2E+2025%2E+Exploring+the+impact+of+temperature+on+large+language+models%3A+Hot+or+cold%3F+Procedia+Computer+Science%2C+264%3A242%E2%80%93251%2E)
17. Vassil Panayotov, Guoguo Chen, Daniel Povey, and Sanjeev Khudanpur. 2015. Librispeech: an asr corpus based on public domain audio books. In 2015 IEEE international conference on acoustics, speech and signal processing (ICASSP), pages 5206–5210. IEEE. [find](https://scholar.google.com/scholar?q=Vassil+Panayotov%2C+Guoguo+Chen%2C+Daniel+Povey%2C+and+Sanjeev+Khudanpur%2E+2015%2E+Librispeech%3A+an+asr+corpus+based+on+public+domain+audio+books%2E+In+2015+IEEE+international+conference+on+acoustics%2C+speech+and+)
18. Sara Papi, Maike Züfle, Marco Gaido, Beatrice Savoldi, Danni Liu, Ioannis Douros, Luisa Bentivogli, and Jan Niehues. 2026. MCIF: Multimodal crosslingual instruction-following benchmark from scientific talks. In The Fourteenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Sara+Papi%2C+Maike+Z%C3%BCfle%2C+Marco+Gaido%2C+Beatrice+Savoldi%2C+Danni+Liu%2C+Ioannis+Douros%2C+Luisa+Bentivogli%2C+and+Jan+Niehues%2E+2026%2E+MCIF%3A+Multimodal+crosslingual+instruction%2Dfollowing+benchmark+from+scientific)
19. Ngoc-Quan Pham, Tuan-Nam Nguyen, Sebastian Stüker, and Alexander Waibel. 2021. Efficient weight factorization for multilingual speech recognition. arXiv preprint arXiv:2105.03010. — [arXiv:2105.03010](https://arxiv.org/abs/2105.03010) [find](https://scholar.google.com/scholar?q=Ngoc%2DQuan+Pham%2C+Tuan%2DNam+Nguyen%2C+Sebastian+St%C3%BCker%2C+and+Alexander+Waibel%2E+2021%2E+Efficient+weight+factorization+for+multilingual+speech+recognition%2E+arXiv+preprint+arXiv%3A2105%2E03010%2E+%E2%80%94+%5BarXiv%3A2105%2E03010%5D)
20. Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever. 2022. Robust speech recognition via large-scale weak supervision. Preprint, arXiv:2212.04356. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jong+Wook+Kim%2C+Tao+Xu%2C+Greg+Brockman%2C+Christine+McLeavey%2C+and+Ilya+Sutskever%2E+2022%2E+Robust+speech+recognition+via+large%2Dscale+weak+supervision%2E+Preprint%2C+arXiv%3A2212%2E04356%2E) [our card](https://md.research.elamur.ai/free/papers/radford-2022-robust-speech-recognition-large-scale/)
21. Ricardo Rei, Marcos Treviso, Nuno M Guerreiro, Chrysoula Zerva, Ana C Farinha, Christine Maroti, José GC De Souza, Taisiya Glushkova, Duarte Alves, Luisa Coheur, and 1 others. 2022. [find](https://scholar.google.com/scholar?q=Ricardo+Rei%2C+Marcos+Treviso%2C+Nuno+M+Guerreiro%2C+Chrysoula+Zerva%2C+Ana+C+Farinha%2C+Christine+Maroti%2C+Jos%C3%A9+GC+De+Souza%2C+Taisiya+Glushkova%2C+Duarte+Alves%2C+Luisa+Coheur%2C+and+1+others%2E+2022%2E)
22. Cometkiwi: Ist-unbabel 2022 submission for the quality estimation shared task. In Proceedings of the Seventh Conference on Machine Translation (WMT), pages 634– 645. [find](https://scholar.google.com/scholar?q=Cometkiwi%3A+Ist%2Dunbabel+2022+submission+for+the+quality+estimation+shared+task%2E+In+Proceedings+of+the+Seventh+Conference+on+Machine+Translation+%28WMT%29%2C+pages+634%E2%80%93+645%2E) [our card](https://md.research.elamur.ai/free/papers/rei-2022-cometkiwi-ist-unbabel-2022-submission/)
23. Fabian Retkowski and Alexander Waibel. 2024. From text segmentation to smart chaptering: A novel benchmark for structuring video transcriptions. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 406–419, St.… [find](https://scholar.google.com/scholar?q=Fabian+Retkowski+and+Alexander+Waibel%2E+2024%2E+From+text+segmentation+to+smart+chaptering%3A+A+novel+benchmark+for+structuring+video+transcriptions%2E+In+Proceedings+of+the+18th+Conference+of+the+European+C)
24. Fabian Retkowski, Maike Züfle, Andreas Sudmann, Dinah Pfau, Shinji Watanabe, Jan Niehues, and Alexander Waibel. 2025. Summarizing speech: A comprehensive survey. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 27275–27306, Suzhou, China. Association f… [find](https://scholar.google.com/scholar?q=Fabian+Retkowski%2C+Maike+Z%C3%BCfle%2C+Andreas+Sudmann%2C+Dinah+Pfau%2C+Shinji+Watanabe%2C+Jan+Niehues%2C+and+Alexander+Waibel%2E+2025%2E+Summarizing+speech%3A+A+comprehensive+survey%2E+In+Proceedings+of+the+2025+Conference+)
25. Fabian Retkowski, Maike Züfle, Thai Binh Nguyen, Jan Niehues, and Alexander Waibel. 2026. Beyond transcripts: A renewed perspective on audio chaptering. Preprint, arXiv:2602.08979. [find](https://scholar.google.com/scholar?q=Fabian+Retkowski%2C+Maike+Z%C3%BCfle%2C+Thai+Binh+Nguyen%2C+Jan+Niehues%2C+and+Alexander+Waibel%2E+2026%2E+Beyond+transcripts%3A+A+renewed+perspective+on+audio+chaptering%2E+Preprint%2C+arXiv%3A2602%2E08979%2E)
26. Supriti Sinhamahapatra and Jan Niehues. 2025. Do slides help? multi-modal context for automatic transcription of conference talks. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 16111–16121. [find](https://scholar.google.com/scholar?q=Supriti+Sinhamahapatra+and+Jan+Niehues%2E+2025%2E+Do+slides+help%3F+multi%2Dmodal+context+for+automatic+transcription+of+conference+talks%2E+In+Proceedings+of+the+2025+Conference+on+Empirical+Methods+in+Natural)
27. Changli Tang, Wenyi Yu, Guangzhi Sun, Xianzhao Chen, Tian Tan, Wei Li, Lu Lu, Zejun Ma, and Chao Zhang. 2024. SALMONN: towards generic hearing abilities for large language models. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenRev… [find](https://scholar.google.com/scholar?q=Changli+Tang%2C+Wenyi+Yu%2C+Guangzhi+Sun%2C+Xianzhao+Chen%2C+Tian+Tan%2C+Wei+Li%2C+Lu+Lu%2C+Zejun+Ma%2C+and+Chao+Zhang%2E+2024%2E+SALMONN%3A+towards+generic+hearing+abilities+for+large+language+models%2E+In+The+Twelfth+Inter) [our card](https://md.research.elamur.ai/free/papers/tang-2023-salmonn-towards-generic-hearing-abilities/)
28. Gemma Team, Aishwarya Kamath, Johan Ferret, Shreya Pathak, Nino Vieillard, Ramona Merhej, Sarah Perrin, Tatiana Matejovicova, Alexandre Ramé, Morgane Rivière, Louis Rouillard, Thomas Mesnard, Geoffrey Cideron, Jean bastien Grill, Sabela Ramos, Edouard Yvinec, Michelle Casbon, Etienne Pot, Ivo Penche… [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Aishwarya+Kamath%2C+Johan+Ferret%2C+Shreya+Pathak%2C+Nino+Vieillard%2C+Ramona+Merhej%2C+Sarah+Perrin%2C+Tatiana+Matejovicova%2C+Alexandre+Ram%C3%A9%2C+Morgane+Rivi%C3%A8re%2C+Louis+Rouillard%2C+Thomas+Mesnard%2C+Geoffrey)
29. Ioannis Tsiamas, Gerard I. Gállego, José A. R. Fonollosa, and Marta R. Costa-jussà. 2022. SHAS: Approaching optimal Segmentation for End-to-End Speech Translation. In Proc. [find](https://scholar.google.com/scholar?q=Ioannis+Tsiamas%2C+Gerard+I%2E+G%C3%A1llego%2C+Jos%C3%A9+A%2E+R%2E+Fonollosa%2C+and+Marta+R%2E+Costa%2Djuss%C3%A0%2E+2022%2E+SHAS%3A+Approaching+optimal+Segmentation+for+End%2Dto%2DEnd+Speech+Translation%2E+In+Proc%2E)
30. Interspeech 2022, pages 106–110. [find](https://scholar.google.com/scholar?q=Interspeech+2022%2C+pages+106%E2%80%93110%2E)
31. Enes Yavuz Ugan, Ngoc-Quan Pham, and Alexander Waibel. 2025. Weight factorization and centralization for continual learning in speech recognition. arXiv preprint arXiv:2506.16574. — [arXiv:2506.16574](https://arxiv.org/abs/2506.16574) [find](https://scholar.google.com/scholar?q=Enes+Yavuz+Ugan%2C+Ngoc%2DQuan+Pham%2C+and+Alexander+Waibel%2E+2025%2E+Weight+factorization+and+centralization+for+continual+learning+in+speech+recognition%2E+arXiv+preprint+arXiv%3A2506%2E16574%2E+%E2%80%94+%5BarXiv%3A2506%2E16574%5D)
32. Changhan Wang, Anne Wu, and Juan Pino. 2020. Covost 2: A massively multilingual speech-to-text translation corpus. Preprint, arXiv:2007.10310. [find](https://scholar.google.com/scholar?q=Changhan+Wang%2C+Anne+Wu%2C+and+Juan+Pino%2E+2020%2E+Covost+2%3A+A+massively+multilingual+speech%2Dto%2Dtext+translation+corpus%2E+Preprint%2C+arXiv%3A2007%2E10310%2E)
33. Dingdong Wang, Jincenzi Wu, Junan Li, Dongchao Yang, Xueyuan Chen, Tianhua Zhang, and Helen Meng. 2025. Mmsu: A massive multi-task spoken language understanding and reasoning benchmark. arXiv preprint arXiv:2506.04779. — [arXiv:2506.04779](https://arxiv.org/abs/2506.04779) [find](https://scholar.google.com/scholar?q=Dingdong+Wang%2C+Jincenzi+Wu%2C+Junan+Li%2C+Dongchao+Yang%2C+Xueyuan+Chen%2C+Tianhua+Zhang%2C+and+Helen+Meng%2E+2025%2E+Mmsu%3A+A+massive+multi%2Dtask+spoken+language+understanding+and+reasoning+benchmark%2E+arXiv+preprint)
34. Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, Bin Zhang, Xiong Wang, Yunfei Chu, and Junyang Lin. 2025a. Qwen2.5-omni technical report. arXiv preprint arXiv:2503.20215. [find](https://scholar.google.com/scholar?q=Jin+Xu%2C+Zhifang+Guo%2C+Jinzheng+He%2C+Hangrui+Hu%2C+Ting+He%2C+Shuai+Bai%2C+Keqin+Chen%2C+Jialin+Wang%2C+Yang+Fan%2C+Kai+Dang%2C+Bin+Zhang%2C+Xiong+Wang%2C+Yunfei+Chu%2C+and+Junyang+Lin%2E+2025a%2E+Qwen2%2E5%2Domni+technical+report%2E) [our card](https://md.research.elamur.ai/free/papers/xu-2025-qwen2-5-omni-technical/)
35. Jin Xu, Zhifang Guo, Jinzheng He, Hangrui Hu, Ting He, Shuai Bai, Keqin Chen, Jialin Wang, Yang Fan, Kai Dang, Bin Zhang, Xiong Wang, Yunfei Chu, and Junyang Lin. 2025b. Qwen2.5-omni technical report. Preprint, arXiv:2503.20215. [find](https://scholar.google.com/scholar?q=Jin+Xu%2C+Zhifang+Guo%2C+Jinzheng+He%2C+Hangrui+Hu%2C+Ting+He%2C+Shuai+Bai%2C+Keqin+Chen%2C+Jialin+Wang%2C+Yang+Fan%2C+Kai+Dang%2C+Bin+Zhang%2C+Xiong+Wang%2C+Yunfei+Chu%2C+and+Junyang+Lin%2E+2025b%2E+Qwen2%2E5%2Domni+technical+report%2E) [our card](https://md.research.elamur.ai/free/papers/xu-2025-qwen2-5-omni-technical/)
36. Jin Xu, Zhifang Guo, Hangrui Hu, Yunfei Chu, Xiong Wang, Jinzheng He, Yuxuan Wang, Xian Shi, Ting He, Xinfa Zhu, Yuanjun Lv, Yongqi Wang, Dake Guo, He Wang, Linhan Ma, Pei Zhang, Xinyu Zhang, Hongkun Hao, Zishan Guo, and 19 others. 2025c. Qwen3-omni technical report. Preprint, arXiv:2509.17765. [find](https://scholar.google.com/scholar?q=Jin+Xu%2C+Zhifang+Guo%2C+Hangrui+Hu%2C+Yunfei+Chu%2C+Xiong+Wang%2C+Jinzheng+He%2C+Yuxuan+Wang%2C+Xian+Shi%2C+Ting+He%2C+Xinfa+Zhu%2C+Yuanjun+Lv%2C+Yongqi+Wang%2C+Dake+Guo%2C+He+Wang%2C+Linhan+Ma%2C+Pei+Zhang%2C+Xinyu+Zhang%2C+Hongkun+) [our card](https://md.research.elamur.ai/free/papers/xu-2025-qwen3-omni-technical-report/)
37. Zihan Zhao, Yiyang Jiang, Heyang Liu, Yanfeng Wang, and Yu Wang. 2023. Librisqa: Advancing free-form and open-ended spoken question answering with a novel dataset and framework. arXiv preprint arXiv:2308.10390. — [arXiv:2308.10390](https://arxiv.org/abs/2308.10390) [find](https://scholar.google.com/scholar?q=Zihan+Zhao%2C+Yiyang+Jiang%2C+Heyang+Liu%2C+Yanfeng+Wang%2C+and+Yu+Wang%2E+2023%2E+Librisqa%3A+Advancing+free%2Dform+and+open%2Dended+spoken+question+answering+with+a+novel+dataset+and+framework%2E+arXiv+preprint+arXiv%3A2)
38. Yaowei Zheng, Richong Zhang, Junhao Zhang, Yanhan Ye, Zheyan Luo, Zhangchi Feng, and Yongqiang Ma. 2024. Llamafactory: Unified efficient fine-tuning of 100+ language models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 3: System Demonstrations),… [find](https://scholar.google.com/scholar?q=Yaowei+Zheng%2C+Richong+Zhang%2C+Junhao+Zhang%2C+Yanhan+Ye%2C+Zheyan+Luo%2C+Zhangchi+Feng%2C+and+Yongqiang+Ma%2E+2024%2E+Llamafactory%3A+Unified+efficient+fine%2Dtuning+of+100%2B+language+models%2E+In+Proceedings+of+the+62nd) [our card](https://md.research.elamur.ai/free/papers/zheng-2024-llamafactory-unified-efficient-fine-tuning/)
39. Maike Züfle and Jan Niehues. 2025. Contrastive learning for task-independent SpeechLLM-pretraining. In Findings of the Association for Computational Linguistics: ACL 2025, pages 8469–8490, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Maike+Z%C3%BCfle+and+Jan+Niehues%2E+2025%2E+Contrastive+learning+for+task%2Dindependent+SpeechLLM%2Dpretraining%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2025%2C+pages+8469%E2%80%938490%2C+Vienna%2C+Aus)
40. Maike Züfle, Sara Papi, Beatrice Savoldi, Marco Gaido, Luisa Bentivogli, and Jan Niehues. 2025. NUT-SHELL: A dataset for abstract generation from scientific talks. In Proceedings of the 22nd International Conference on Spoken Language Translation (IWSLT 2025), pages 19–32, Vienna, Austria (in-person… [find](https://scholar.google.com/scholar?q=Maike+Z%C3%BCfle%2C+Sara+Papi%2C+Beatrice+Savoldi%2C+Marco+Gaido%2C+Luisa+Bentivogli%2C+and+Jan+Niehues%2E+2025%2E+NUT%2DSHELL%3A+A+dataset+for+abstract+generation+from+scientific+talks%2E+In+Proceedings+of+the+22nd+Internati)