--- title: "Call Support Copilot — paper by Rakhmatillokhon et al. (2026)" description: "The authors build Call Support Copilot, a Streamlit dashboard that chains public pre-trained… 6.6% WER on 50 LibriSpeech. 1 reference." authors: ["Khoshimov Rakhmatillokhon", "Dmitry Rudshin", "Yanyang Luo"] year: 2026 venue: "Proceedings of the 11th Edition of the Swiss Text Analytics Conference" source: "https://aclanthology.org/2026.swisstext-1.6/" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/rakhmatillokhon-2026-call-support-copilot-reproducible-multimodal/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rakhmatillokhon-2026-call-support-copilot-reproducible-multimodal/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/24/) / Call Support Copilot: A Reproducible Multimodal System for Speech Emotion Recognition, Intent Understanding, and Agent Assistance # Call Support Copilot: A Reproducible Multimodal System for Speech Emotion Recognition, Intent Understanding, and Agent Assistance **Authors:** Khoshimov Rakhmatillokhon, Dmitry Rudshin and Yanyang Luo · **Year:** 2026 · **Venue:** [Proceedings of the 11th Edition of the Swiss Text Analytics Conference](https://md.research.elamur.ai/free/en/venues/swisstext/) ([Edition of the Swiss Text Analytics Conference 2026](https://md.research.elamur.ai/free/en/venues/swisstext/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.swisstext-1.6/) · [PDF](https://aclanthology.org/2026.swisstext-1.6.pdf) ## Summary Customer support agents must follow what a caller says, gauge their mood, and look up account data all at once. Prior work mostly evaluates the relevant technologies in isolation, and such systems are often hard to reproduce. The authors build Call Support Copilot, a Streamlit dashboard that chains public pre-trained checkpoints. faster-whisper ASR produces transcripts, which go to MarianMT for translation into English and to a DistilBERT intent classifier fine-tuned on BANKING77, with regex-based slot filling. In parallel, MERaLiON-SER-v1 tracks caller emotion as categorical labels plus valence-arousal-dominance scores over sliding windows. These signals are combined with records from an SQLite client database to suggest agent actions, such as follow-up questions or de-escalation tips. No ASR, MT or SER model is trained from scratch. Each module is benchmarked separately on standard data, and pipeline speed is measured. The intent classifier slightly outperforms a published BANKING77 baseline, and processing runs faster than real time. The authors note caveats: the emotion score rests on only six utterances, ASR error propagation is not measured, and there is no user study. The main contribution is a fully documented, auditable recipe. ## Key points - Component results: - 6.6% WER on 50 LibriSpeech test-clean samples (Systran/faster-whisper-base, int8). - 42.98 BLEU on 50 WMT16 German–English pairs (Helsinki-NLP opus-mt models). - 87.0% accuracy and 0.863 macro-F1 on BANKING77, versus 85.19% for USE+ConveRT (Casanueva et al., 2020). - The SER figure of 91.7% macro-F1 comes from the SUPERB ER session1 split (IEMOCAP-derived, four classes, n=6). The authors treat it only as a sanity check. - Mean real-time factor is 0.71 on LibriSpeech (n=10) and 0.67 on SUPERB ER (n=6). Sliding-window SER (4 s windows, 2 s overlap) accounts for most of the latency. - Translation covers six European source languages into English: German, Dutch, French, Spanish, Italian and Portuguese. The system handles uploaded recordings only, not live streams. - Model IDs, quantization settings, dataset slices, scripts and CSV/JSON outputs are released at github.com/rakhmatillokhon-khoshimov/call-support-copilot. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works 🧭 Topics (3): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 972 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 657 works · [🎙️ Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) · 480 works ## Cite Cite as: Khoshimov Rakhmatillokhon, Dmitry Rudshin and Yanyang Luo. 2026. Call Support Copilot: A Reproducible Multimodal System for Speech Emotion Recognition, Intent Understanding, and Agent Assistance. Proceedings of the 11th Edition of the Swiss Text Analytics Conference. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{rakhmatillokhon2026call, title = {{Call Support Copilot: A Reproducible Multimodal System for Speech Emotion Recognition, Intent Understanding, and Agent Assistance}}, author = {Khoshimov Rakhmatillokhon, Dmitry Rudshin and Yanyang Luo}, booktitle = {Proceedings of the 11th Edition of the Swiss Text Analytics Conference}, year = {2026}, url = {https://aclanthology.org/2026.swisstext-1.6/} } ``` - [← previous Calibration-Aware Policy Optimization for Reasoning LLMs](https://md.research.elamur.ai/free/en/papers/wang-2026-calibration-aware-policy-optimization-reasoning/) - [next → CAMMAR: Culture-Aware Matryoshka for Metaphorical Arabic Representations](https://md.research.elamur.ai/free/en/papers/awinat-2026-cammar-culture-aware-matryoshka-metaphorical/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/24/) ## References 1 references, 1 of them with a card of ours. ↗ leads to the work. 1. Rosana Ardila, Megan Branson, Kelly Davis, Michael Kohler, Josh Meyer, Michael Henretty, Reuben Morais, Lindsay Saunders, Francis Tyers, and Gregor Weber. 2020. [Common voice: A massively-multilingual speech corpus](https://md.research.elamur.ai/free/en/papers/ardila-2019-common-voice-massively-multilingual-speech/). In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 421… *Updated 1 November 2026*