--- title: "EmoAra: Emotion-Preserving English Speech — Hassan 2026" description: "The paper describes EmoAra, a cascaded pipeline for English-speaking banking customers being served in Arabic." authors: ["Besher Hassan", "Ibrahim Alsarraj", "Musaab Hasan", "Yousef Melhim", "Shahem Fadi", "Shahem Sultan"] year: 2026 venue: "arXiv:2602.01170" source: "https://arxiv.org/abs/2602.01170" doi: "10.48550/arxiv.2602.01170" arxiv: "2602.01170" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/hassan-2026-emoara-emotion-preserving-english-speech/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hassan-2026-emoara-emotion-preserving-english-speech/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/) / EmoAra: Emotion-Preserving English Speech Transcription and Cross-Lingual Translation with Arabic Text-to-Speech # EmoAra: Emotion-Preserving English Speech Transcription and Cross-Lingual Translation with Arabic Text-to-Speech **Authors:** [Besher Hassan](https://md.research.elamur.ai/free/en/authors/besher-hassan/), Ibrahim Alsarraj, Musaab Hasan, Yousef Melhim, Shahem Fadi, Shahem Sultan · **Year:** 2026 · **Venue:** arXiv:2602.01170 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2602.01170) · [PDF](https://arxiv.org/pdf/2602.01170) · [DOI 10.48550/arxiv.2602.01170](https://doi.org/10.48550/arxiv.2602.01170) · [arXiv 2602.01170](https://arxiv.org/abs/2602.01170) · [OpenAlex](https://openalex.org/W7127394943) **Citations:** 0 citations (openalex-snapshot · 2026-10-14) ## Summary The paper describes EmoAra, a cascaded pipeline for English-speaking banking customers being served in Arabic. It aims to carry the customer's emotional state through the language switch. The pipeline has four stages: - **Emotion recognition:** a 1D CNN classifies the speaker's emotion from hand-crafted acoustic features (ZCR, RMSE, MFCCs), computed on 1,440 labelled .wav clips expanded through augmentation. - **Transcription:** Whisper Base converts the English speech to text. - **Translation:** a MarianMT model, fine-tuned on a mix of banking-domain and general parallel data, translates the text into Arabic. - **Speech synthesis:** MMS-TTS-Ara voices the Arabic result. On eight emotion classes, the CNN reaches an average F1 of 0.94, far above a plain baseline CNN at 0.53. The fine-tuned translator reaches BLEU 56 and BERTScore F1 88.7%. On 100 banking test sentences, human raters gave it 81%. The work matters mainly as an engineering demonstration. Off-the-shelf pretrained components plus modest domain fine-tuning yield a usable service-oriented speech translation chain. The authors acknowledge a small dataset and weak handling of long sentences. ## Key points - **Emotion classifier:** a three-layer 1D CNN with batch normalization, max pooling, dropout and a 256-unit dense layer, trained with Adam and categorical cross-entropy. Its inputs are standardized ZCR, RMSE and MFCC features. - **Augmentation:** the training audio was varied with Gaussian noise, pitch shifting, time stretching and time shifting. - **Per-class emotion results:** angry scores highest (F1 0.97), and calm, fearful and sad score lowest (0.92). The same weak classes appear for LSTM (average 0.87) and ResNet50 (average 0.89). - **Translation ladder (BLEU / BERTScore F1):** | Model | BLEU | 95% CI | BERTScore F1 | |---|---|---|---| | Transformer trained from scratch | ~23.8 | — | 67% | | Pretrained MarianMT, no fine-tuning | 25.48 | — | 73% | | Fine-tuned MarianMT | 56 | 54.5–57.5 | 88.7% | The text gives the from-scratch BLEU as 23.00, while the table gives 23.78. - **Translation preprocessing:** lowercasing, removal of digits and punctuation, script-based filtering (Arabic script only on the Arabic side, Latin letters only on the English side), and truncation to 128 tokens. No training sentence exceeds 20 words, which the authors link to poorer translation of long inputs. ## Interesting ideas - An ablation baseline without feature extraction or augmentation shows how much those steps contribute: average F1 rises from 0.53 to 0.94, which isolates the value of the preparation work rather than the architecture. — For Mova (learn): Each new training trick for the parser or tagger could be reported against a stripped-down baseline, so the gain is attributed to the right component. - Fine-tuning on a blend of domain data and general data lets the model pick up specialist vocabulary without losing broad coverage. — For Mova (learn): When adapting gparse or the tagger to school texts or fables, mixing domain silver trees with general UD treebanks could limit forgetting of common constructions. - Translation quality is reported as BLEU with confidence intervals rather than a single point. This makes it easier to judge whether gaps between systems are real. — For Mova (learn): LAS and QA accuracy comparisons between Mova versions could carry bootstrap intervals, so that small differences are not mistaken for progress. - Human judges scored outputs on three separate axes: accuracy, fluency and domain terminology. This separates "says the right thing" from "reads well" and "uses the right words". — For Mova (back): Text regenerated from the meaning graph could be judged on the same three axes, showing whether losses come from the graph or from surface realization. - The authors trace a failure mode (long sentences) to a gap in the training-length distribution rather than to the model itself. — For Mova (tree): Parser errors could be bucketed by sentence length and checked against treebank length statistics, to see when long-sentence errors are really a data-coverage problem. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{hassan2026emoara, title = {{EmoAra: Emotion-Preserving English Speech Transcription and Cross-Lingual Translation with Arabic Text-to-Speech}}, author = {Hassan, Besher and Alsarraj, Ibrahim and Hasan, Musaab and Melhim, Yousef and Fadi, Shahem and Sultan, Shahem}, year = {2026}, doi = {10.48550/arxiv.2602.01170}, eprint = {2602.01170}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2602.01170} } ``` - [← previous Embedding Similarity Is Not Quality Estimation: Lessons from Replacing a Dedicated QE Model](https://md.research.elamur.ai/free/en/papers/zaikis-2026-embedding-similarity-is-not-quality/) - [next → Emotion Transcription in Conversation: A Benchmark for Capturing Subtle and Complex Emotional States through Natural Language](https://md.research.elamur.ai/free/en/papers/tanaka-2026-emotion-transcription-conversation-benchmark-capturing/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/4/) ## References 14 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. A. Baevski, H. Zhou, A. Mohamed, and M. Auli. wav2vec 2.0: A framework for self-supervised learning of speech representations. In Proceedings of the 34th Conference on Neural Information Processing Systems (NeurIPS), 2020. [find](https://scholar.google.com/scholar?q=A%2E+Baevski%2C+H%2E+Zhou%2C+A%2E+Mohamed%2C+and+M%2E+Auli%2E+wav2vec+2%2E0%3A+A+framework+for+self%2Dsupervised+learning+of+speech+representations%2E+In+Proceedings+of+the+34th+Conference+on+Neural+Information+Processing+Sy) [our card](https://md.research.elamur.ai/free/en/papers/baevski-2020-wav2vec-2-0-framework-self/) 2. Google Cloud. Cloud text-to-speech documentation. [https://cloud.google.com/text-to-speech](https://cloud.google.com/text-to-speech), 2021. [↗](https://cloud.google.com/text-to-speech) 3. Hugging Face Datasets. Banking 77 dataset. [https://huggingface.co/datasets/banking77](https://huggingface.co/datasets/banking77), 2020. Accessed via Hugging Face Datasets Library. [↗](https://huggingface.co/datasets/banking77) 4. Hugging Face. Marian: Multilingual machine translation model. [https://huggingface.co/docs/transformers/en/model_doc/marian](https://huggingface.co/docs/transformers/en/model_doc/marian), 2021. Accessed: 2024-11-14. [↗](https://huggingface.co/docs/transformers/en/model_doc/marian) 5. A. Hannun, A. Maas, D. Jurafsky, and A. Ng. Deepspeech: Scaling up end-to-end speech recognition. arXiv preprint arXiv:1412.5567, 2014. [↗](https://arxiv.org/abs/1412.5567) [our card](https://md.research.elamur.ai/free/en/papers/hannun-2014-deep-speech-scaling-up-end/) 6. Helsinki-NLP. Marianmt: Open neural machine translation models, 2020. Accessed: 2024-11-14. [find](https://scholar.google.com/scholar?q=Helsinki%2DNLP%2E+Marianmt%3A+Open+neural+machine+translation+models%2C+2020%2E+Accessed%3A+2024%2D11%2D14%2E) 7. Y. Huang, X. Yu, J. Ou, and W. Xu. Speech emotion recognition using transfer learning. In Proceedings of IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP), 2021. [find](https://scholar.google.com/scholar?q=Y%2E+Huang%2C+X%2E+Yu%2C+J%2E+Ou%2C+and+W%2E+Xu%2E+Speech+emotion+recognition+using+transfer+learning%2E+In+Proceedings+of+IEEE+International+Conference+on+Acoustics%2C+Speech%2C+and+Signal+Processing+%28ICASSP%29%2C+2021%2E) 8. M. Junczys-Dowmunt, R. Grundkiewicz, K. Heafield, and A. Birch. Marian: Fast neural machine translation in c++. In Proceedings of the ACL 2018 System Demonstrations, 2018. [find](https://scholar.google.com/scholar?q=M%2E+Junczys%2DDowmunt%2C+R%2E+Grundkiewicz%2C+K%2E+Heafield%2C+and+A%2E+Birch%2E+Marian%3A+Fast+neural+machine+translation+in+c%2B%2B%2E+In+Proceedings+of+the+ACL+2018+System+Demonstrations%2C+2018%2E) [our card](https://md.research.elamur.ai/free/en/papers/junczys-2018-dowmunt-marian-fast-neural-machine-translation/) 9. UWRF Kaggle. Ravdess emotional speech audio. ravdess-emotional-speech-audio, 2023. Accessed: 2024-09-16. [https://www.kaggle.com/datasets/uwrfkaggler/](https://www.kaggle.com/datasets/uwrfkaggler/) [↗](https://www.kaggle.com/datasets/uwrfkaggler/) 10. S. R. Livingstone and F. A. Russo. The ryerson audio-visual database of emotional speech and song (ravdess). PLOS ONE, 13(5):e0196391, 2018. [find](https://scholar.google.com/scholar?q=S%2E+R%2E+Livingstone+and+F%2E+A%2E+Russo%2E+The+ryerson+audio%2Dvisual+database+of+emotional+speech+and+song+%28ravdess%29%2E+PLOS+ONE%2C+13%285%29%3Ae0196391%2C+2018%2E) 11. MMS-TTS-Ara. facebook/mms-tts-ara - arabic tts model. huggingface.co/facebook/mms-tts-ara. [https://huggingface.co/facebook/mms-tts-ara](https://huggingface.co/facebook/mms-tts-ara), 2024. [↗](https://huggingface.co/facebook/mms-tts-ara) 12. Samir Moustafa. Arabic to english translation sentences dataset, 2024. Kaggle. [find](https://scholar.google.com/scholar?q=Samir+Moustafa%2E+Arabic+to+english+translation+sentences+dataset%2C+2024%2E+Kaggle%2E) 13. OpenAI. Whisper: Robust speech recognition model. [https://openai.com/index/whisper/](https://openai.com/index/whisper/), 2022. Accessed: 2024-11-14. [↗](https://openai.com/index/whisper/) 14. OpenAI. Whisper: Robust speech recognition model. [https://github.com/openai/whisper](https://github.com/openai/whisper), 2022. Accessed: 2024-11-14. https:// [↗](https://github.com/openai/whisper)