---
title: "A unified front-end framework for English text-to-speech synthesis"
description: "The work matters because it presents a complete, systematic English front-end rather than… 3.09% WER. 21 references, cited by 1 paper here."
authors: ["Zelin Ying", "Chen Li", "Yu Dong", "Qiuqiang Kong", "Qiao Tian", "Yuanyuan Huo", "Yuxuan Wang"]
year: 2023
venue: "arXiv:2305.10666"
source: "https://arxiv.org/abs/2305.10666"
doi: "10.1109/icassp48485.2024.10447144"
arxiv: "2305.10666"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ying-2023-unified-front-end-framework-english/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ying-2023-unified-front-end-framework-english/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/13/) / A unified front-end framework for English text-to-speech synthesis

# A unified front-end framework for English text-to-speech synthesis

**Authors:** Zelin Ying, [Chen Li](https://md.research.elamur.ai/free/en/authors/chen-li/), Yu Dong, Qiuqiang Kong, Qiao Tian, Yuanyuan Huo and [Yuxuan Wang](https://md.research.elamur.ai/free/en/authors/yuxuan-wang/) · **Year:** 2023 · **Venue:** [arXiv:2305.10666](https://md.research.elamur.ai/free/en/venues/ieee-international-conference-on-acoustics-speech-and-signal-processing/) ([IEEE International Conference on Acoustics Speech and Signal Processing 2023](https://md.research.elamur.ai/free/en/venues/ieee-international-conference-on-acoustics-speech-and-signal-processing/2023/)) · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://doi.org/10.1109/icassp48485.2024.10447144) · [PDF](https://arxiv.org/pdf/2305.10666) · [DOI 10.1109/icassp48485.2024.10447144](https://doi.org/10.1109/icassp48485.2024.10447144) · [arXiv 2305.10666](https://arxiv.org/abs/2305.10666) · [OpenAlex](https://openalex.org/W4392903542)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/ying-2023-unified-front-end-framework-english/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
English TTS front-ends usually consist of three separately built parts. Text normalization (TN) turns numbers, symbols and abbreviations into spoken words. The prosody word prosody phrase (PWPP) module predicts pause boundaries. Grapheme-to-phoneme (G2P) produces phonemes. The authors argue that building these in isolation ignores how they depend on each other.

Their proposal is a unified front-end built on one multilingual BERT, which is fine-tuned jointly across five tasks covering text normalization, pause prediction, out-of-vocabulary G2P, part-of-speech tagging and polyphone disambiguation (TN, PWPP, G2POOV, POS, Polyphone). Expert-written rules are wrapped around the model.

- **TN** combines model tagging with rule-based correction and verbalization.
- **PWPP** treats each pause level as its own binary tagging problem.
- **G2P** looks words up in a lexicon first, generates phonemes for unknown words with a character-level seq2seq model, and then fixes homographs using POS and a per-word pronunciation classifier.

The paper reports state-of-the-art results in every module, including a 1.19% sentence error rate for TN and a 3.09% WER for the full G2P pipeline. The work matters because it presents a complete, systematic English front-end rather than isolated components, and the authors claim it is the first unified one for English.

## Key points
- **One shared encoder.** A 12-layer pre-trained multilingual BERT serves five tasks. Taggers use a CBG encoder (1-D convolution bank plus bidirectional GRU, 256 hidden units) with a masked CRF. G2POOV uses a Transformer encoder-decoder (dimension 256, 4 heads). Training uses AdamW at a learning rate of 5×10⁻⁵.
- **Label sets.**
  - TN: 19 categories × BIES tags + "O" = 77 labels.
  - POS: 24 categories, giving 97 labels.
  - G2POOV: maps 61 input characters to 73 phonemes.
- **Data.**
  - TN is tested on the Google English text normalization dataset (755,441 sentences), where it reaches 1.19% SER against baselines including Seq2Edits.
  - PWPP uses an internal set of 100,026 sentences.
  - POS uses 102,164 sentences and Polyphone 28,097.
  - The full G2P module is evaluated on a real-world test set of 2,560 sentences.
- **PWPP results.** F1-scores are 90.83% for level #1, 57.65% for #2 and 83.36% for #3.
- **G2P results.** G2POOV alone has a WER of 19.42%. Adding the components one at a time lowers the full-module WER step by step: 3.42% with G2POOV, 3.17% with POS added, and 3.09% with Polyphone added.

## Interesting ideas
- The neural tagger proposes a category for each non-standard token, a rule pre-pass overrides categories it knows are wrong, and deterministic rules produce the final spoken form, which keeps cases like reading "911" digit by digit controllable
- Instead of one flat tag per position, each nested boundary level gets its own binary tagger and the strongest positive level wins, which respects the ordering between levels
- Lexicon lookup comes first, a character-level generator covers out-of-lexicon words, and context-based models only revise known ambiguous entries
- For words whose readings differ even within one part of speech (noun "lead"), the paper builds a separate classifier whose labels are that word's own ordered pronunciation list
- Components are added cumulatively and the error is measured after each addition on a real-world test set, which shows exactly how much each stage contributes

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): The tokenizer could classify numbers, dates and abbreviations with a model but let explicit rules veto and verbalize them, so every normalization step has a traceable reason.
- **I2** (tree): Clause, phrase and sentence boundary detection before parsing could be predicted level by level, making errors attributable to a specific level rather than to a merged label.
- **I3** (tokens): The lemmatizer and morphology could follow the same layering (dictionary, then character-level fallback for unseen words, then contextual override), logging which layer decided each word.
- **I4** (world): Sense choice against WordNet could likewise use per-lemma label inventories, triggered only when POS alone leaves the lemma ambiguous.
- **I5** (learn): Pipeline evaluations could report the score after enabling each stage in turn, exposing stages that add complexity without measurable gain.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [ByteDance](https://md.research.elamur.ai/free/en/companies/bytedance/) — Company, Beijing, China · 41 works · 4,180 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 870 works · [🔊 Signal Processing](https://md.research.elamur.ai/free/en/shelves/computer-science/signal-processing/) · 452 works

🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works · [🎙️ Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) · 480 works

## Cite

Cite as: Zelin Ying, Chen Li, Yu Dong, Qiuqiang Kong, Qiao Tian, Yuanyuan Huo and Yuxuan Wang. 2023. A unified front-end framework for English text-to-speech synthesis. arXiv:2305.10666.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{ying2023unified,
  title = {{A unified front-end framework for English text-to-speech synthesis}},
  author = {Zelin Ying, Chen Li, Yu Dong, Qiuqiang Kong, Qiao Tian, Yuanyuan Huo and Yuxuan Wang},
  year = {2023},
  doi = {10.1109/icassp48485.2024.10447144},
  eprint = {2305.10666},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1109/icassp48485.2024.10447144}
}
```

- [← previous A new learner language data set for the study of English for Specific Purposes at university](https://md.research.elamur.ai/free/en/papers/mallart-2023-new-learner-language-data-set/)
- [next → AAST-NLP at ArAIEval Shared Task: Tackling Persuasion technique and Disinformation Detection using Pre-Trained Language Models On Imbalanced Datasets](https://md.research.elamur.ai/free/en/papers/sayed-2023-aast-nlp-araieval-shared-task/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/13/)

## References

21 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alexandre Bérard, Laurent Besacier, Ali Can Kocabiyikoglu, and Olivier Pietquin, “End-to-end automatic speech translation of audiobooks,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2018, pp. 6224–6228. [find](https://scholar.google.com/scholar?q=Alexandre+Be%CC%81rard%2C+Laurent+Besacier%2C+Ali+Can+Kocabiyikoglu%2C+and+Olivier+Pietquin%2C+%E2%80%9CEnd%2Dto%2Dend+automatic+speech+translation+of+audiobooks%2C%E2%80%9D+in+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Sig)
2. Peter Ebden and Richard Sproat, “The Kestrel TTS text normalization system,” Natural Language Engineering, vol. 21, no. 3, pp. 333–353, 2015. [↗](https://doi.org/10.1017/s1351324914000175)
3. Richard Sproat and Navdeep Jaitly, “An RNN model of text normalization,” Proc. Interspeech 2017, pp. 754– 758, 2017. [find](https://scholar.google.com/scholar?q=Richard+Sproat+and+Navdeep+Jaitly%2C+%E2%80%9CAn+RNN+model+of+text+normalization%2C%E2%80%9D+Proc%2E+Interspeech+2017%2C+pp%2E+754%E2%80%93+758%2C+2017%2E)
4. Junhui Zhang, Junjie Pan, Xiang Yin, Chen Li, Shichao Liu, Yang Zhang, Yuxuan Wang, and Zejun Ma, “A hybrid text normalization system using multi-head self-attention for mandarin,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2020, pp. 6694–6698. [↗](https://doi.org/10.48550/arxiv.1911.04128)
5. Yibin Zheng, Jianhua Tao, Zhengqi Wen, and Ya Li, “BLSTM-CRF based end-to-end prosodic boundary prediction with context sensitive embeddings in a text-to-speech front-end.,” in Interspeech, 2018, pp. 47–51. [find](https://scholar.google.com/scholar?q=Yibin+Zheng%2C+Jianhua+Tao%2C+Zhengqi+Wen%2C+and+Ya+Li%2C+%E2%80%9CBLSTM%2DCRF+based+end%2Dto%2Dend+prosodic+boundary+prediction+with+context+sensitive+embeddings+in+a+text%2Dto%2Dspeech+front%2Dend%2E%2C%E2%80%9D+in+Interspeech%2C+2018%2C+pp%2E+)
6. Kanishka Rao, Fuchun Peng, Haşim Sak, and Françoise Beaufays, “Grapheme-to-phoneme conversion using long short-term memory recurrent neural networks,” in IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), 2015, pp. 4225–4229. [find](https://scholar.google.com/scholar?q=Kanishka+Rao%2C+Fuchun+Peng%2C+Has%CC%A7im+Sak%2C+and+Franc%CC%A7oise+Beaufays%2C+%E2%80%9CGrapheme%2Dto%2Dphoneme+conversion+using+long+short%2Dterm+memory+recurrent+neural+networks%2C%E2%80%9D+in+IEEE+International+Conference+on+Acoustics%2C+)
7. Junjie Pan, Xiang Yin, Zhiling Zhang, Shichao Liu, Yang Zhang, Zejun Ma, and Yuxuan Wang, “A unified sequence-to-sequence front-end model for mandarin text-to-speech synthesis,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 6689–6693. [↗](https://doi.org/10.48550/arxiv.1911.04111)
8. Yang Zhang, Liqun Deng, and Yasheng Wang, “Unified mandarin tts front-end based on distilled bert model,” arXiv preprint arXiv:2012.15404, 2020. [↗](https://arxiv.org/abs/2012.15404)
9. Siqi Sun, Korin Richmond, and Hao Tang, “Improving seq2seq tts frontends with transcribed speech audio,” IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2023. [↗](https://doi.org/10.1109/taslp.2023.3273414)
10. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova, “[BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/),” in Proceedings of NAACL-HLT, 2019, pp. 4171–4186.
11. Yuxuan Wang, RJ Skerry-Ryan, Daisy Stanton, Yonghui Wu, Ron J Weiss, Navdeep Jaitly, Zongheng Yang, Ying Xiao, Zhifeng Chen, Samy Bengio, et al., “Tacotron: Towards end-to-end speech synthesis,” speech 2017, pp. 4006–4010, 2017. Proc. Inter- [↗](https://doi.org/10.48550/arxiv.1703.10135)
12. Tianwen Wei, Jianwei Qi, Shenghuan He, and Songtao Sun, “Masked conditional random fields for sequence labeling,” in North American Chapter of the Association for Computational Linguistics: Human Language Technologies, 2021, pp. 2024–2035. [find](https://scholar.google.com/scholar?q=Tianwen+Wei%2C+Jianwei+Qi%2C+Shenghuan+He%2C+and+Songtao+Sun%2C+%E2%80%9CMasked+conditional+random+fields+for+sequence+labeling%2C%E2%80%9D+in+North+American+Chapter+of+the+Association+for+Computational+Linguistics%3A+Human+Lang)
13. Richard Sproat and Navdeep Jaitly, “RNN approaches to text normalization: A challenge,” arXiv preprint arXiv:1611.00068, 2016. [↗](https://arxiv.org/abs/1611.00068)
14. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin, “[Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/),” Advances in Neural Information Processing Systems, vol. 30, 2017.
15. Felix Stahlberg and Shankar Kumar, “Seq2Edits: Sequence transduction using span-level edit operations,” in Proceedings of the Conference on Empirical Methods in Natural Language Processing (EMNLP), 2020, pp. 5147–5159. [find](https://scholar.google.com/scholar?q=Felix+Stahlberg+and+Shankar+Kumar%2C+%E2%80%9CSeq2Edits%3A+Sequence+transduction+using+span%2Dlevel+edit+operations%2C%E2%80%9D+in+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2C+20)
16. Jae Hun Ro, Felix Stahlberg, Ke Wu, and Shankar Kumar, “Transformer-based models of text normalization for speech applications,” arXiv preprint arXiv:2202.00153, 2022. [↗](https://arxiv.org/abs/2202.00153)
17. Hao Zhang, Richard Sproat, Axel H Ng, Felix Stahlberg, Xiaochang Peng, Kyle Gorman, and Brian Roark, “[Neural models of text normalization for speech applications](https://md.research.elamur.ai/free/en/papers/zhang-2019-neural-models-text-normalization-speech/),” Computational Linguistics, vol. 45, no. 2, pp. 293–337, 2019. [↗](https://doi.org/10.1162/coli_a_00349)
18. Chendong Zhao, Jianzong Wang, Xiaoyang Qu, Haoqian Wang, and Jing Xiao, “r-g2p: Evaluating and enhancing robustness of grapheme to phoneme conversion by controlled noise introducing and contextual information incorporation,” in ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2022, pp. 6197–6201. [↗](https://doi.org/10.48550/arxiv.2202.11194)
19. Sevinj Yolchuyeva, Géza Németh, and Bálint Gyires-Tóth, “Transformer based grapheme-to-phoneme conversion,” Proc. Interspeech 2019, pp. 2095–2099, 2019. [find](https://scholar.google.com/scholar?q=Sevinj+Yolchuyeva%2C+Ge%CC%81za+Ne%CC%81meth%2C+and+Ba%CC%81lint+Gyires%2DTo%CC%81th%2C+%E2%80%9CTransformer+based+grapheme%2Dto%2Dphoneme+conversion%2C%E2%80%9D+Proc%2E+Interspeech+2019%2C+pp%2E+2095%E2%80%932099%2C+2019%2E)
20. Shubham Toshniwal and Karen Livescu, “Jointly learning to align and convert graphemes to phonemes with neural attention models,” in 2016 IEEE Spoken Language Technology Workshop (SLT). IEEE, 2016, pp. 76– 82. [find](https://scholar.google.com/scholar?q=Shubham+Toshniwal+and+Karen+Livescu%2C+%E2%80%9CJointly+learning+to+align+and+convert+graphemes+to+phonemes+with+neural+attention+models%2C%E2%80%9D+in+2016+IEEE+Spoken+Language+Technology+Workshop+%28SLT%29%2E+IEEE%2C+2016%2C+pp%2E)
21. I Loshchilov and F Hutter, “Fixing weight decay regularization in adam, iclr 2018 conference,” in International Conference on Learning Representations (ICLR), 2018. [find](https://scholar.google.com/scholar?q=I+Loshchilov+and+F+Hutter%2C+%E2%80%9CFixing+weight+decay+regularization+in+adam%2C+iclr+2018+conference%2C%E2%80%9D+in+International+Conference+on+Learning+Representations+%28ICLR%29%2C+2018%2E)

*Updated 1 November 2026*