---
title: "Neural Models of Text Normalization for Speech Applications"
description: "They release a large public dataset and test several neural architectures on English and Russian… 51 references, cited by 30 papers here."
authors: ["Hao Zhang", "Richard Sproat", "Axel H. Ng", "Felix Stahlberg", "Xiaochang Peng", "Kyle Gorman", "Brian Roark"]
year: 2019
venue: "Computational Linguistics"
source: "https://aclanthology.org/J19-2004/"
doi: "10.1162/coli_a_00349"
licence: "CC BY-NC-ND 4.0 (Computational Linguistics, MIT Press)"
licence_url: "https://creativecommons.org/licenses/by-nc-nd/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2019-neural-models-text-normalization-speech/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2019-neural-models-text-normalization-speech/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/) / Neural Models of Text Normalization for Speech Applications

# Neural Models of Text Normalization for Speech Applications

**Authors:** [Hao Zhang](https://md.research.elamur.ai/free/en/authors/hao-zhang/), [Richard Sproat](https://md.research.elamur.ai/free/en/authors/richard-sproat/), Axel H. Ng, [Felix Stahlberg](https://md.research.elamur.ai/free/en/authors/felix-stahlberg/), [Xiaochang Peng](https://md.research.elamur.ai/free/en/authors/xiaochang-peng/), [Kyle Gorman](https://md.research.elamur.ai/free/en/authors/kyle-gorman/) and [Brian Roark](https://md.research.elamur.ai/free/en/authors/brian-roark/) · **Year:** 2019 · **Venue:** [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/) ([Computational Linguistics 2019](https://md.research.elamur.ai/free/en/venues/cl/2019/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY-NC-ND 4.0 (Computational Linguistics, MIT Press)](https://creativecommons.org/licenses/by-nc-nd/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/J19-2004/) · [PDF](https://aclanthology.org/J19-2004.pdf) · [DOI 10.1162/coli\_a\_00349](https://doi.org/10.1162/coli_a_00349) · [OpenAlex](https://openalex.org/W2924677654)

**Citations:** 85 citations (OpenAlex, 2026-10-14) · [30](https://md.research.elamur.ai/free/en/papers/zhang-2019-neural-models-text-normalization-speech/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Text normalization for speech converts written forms such as digits, dates, times and measure expressions into the words a text-to-speech (TTS) engine should say. The correct reading depends on context: a number in a page count is read differently from one in a street address. Industrial systems such as Google's Kestrel still rely mostly on hand-built, language-specific grammars. The authors frame the task as sequence-to-sequence prediction, mapping each token, seen within its sentence, to its spoken form. They release a large public dataset and test several neural architectures on English and Russian against a sliding-window model and a Transformer baseline. The best design, in both accuracy and speed, encodes the sentence context once and reuses it while decoding each token. This design also lets tagging and segmentation be built into the model. Neural models sometimes make rare but meaning-changing mistakes, so the authors add finite-state covering grammars, mostly learned from data, to constrain training and/or decoding. The work shows that linguistic knowledge stays important in deep learning when output errors are costly.

## Key points
- The paper introduces a large-scale, publicly available dataset for TTS text normalization, with experiments on English and Russian.
- The proposed contextual model computes sentence context once and combines it with per-token decoding. It outperforms the sliding-window models of Sproat and Jaitly (2016, 2017) and a Transformer baseline (6 encoder/decoder layers, 8 heads, model dimension 512).
- The authors call errors that do not preserve meaning "unrecoverable," for example reading "3 cm" as kilometers or misreading digits in an address. Such errors are rare but unacceptable for TTS.
- Finite-state covering grammars, written in Thrax (e.g., a grammar for English years), are induced largely from data. They extend Gorman and Sproat (2016) from numbers to other semiotic classes such as dates and measure expressions.
- The covering grammars can guide the neural model during both training and decoding, or during decoding only.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Google](https://md.research.elamur.ai/free/en/companies/google/) — Company, Mountain View, United States · 430 works · 424,363 citations
- [Meta](https://md.research.elamur.ai/free/en/companies/meta/) — Company, Menlo Park, United States · 183 works · 228,558 citations
- [University of Cambridge](https://md.research.elamur.ai/free/en/universities/university-of-cambridge/) — University, Cambridge, United Kingdom · 243 works · 121,396 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

## Cite

Cite as: Hao Zhang, Richard Sproat, Axel H. Ng, Felix Stahlberg, Xiaochang Peng, Kyle Gorman and Brian Roark. 2019. Neural Models of Text Normalization for Speech Applications. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{zhang2019neural,
  title = {{Neural Models of Text Normalization for Speech Applications}},
  author = {Hao Zhang, Richard Sproat, Axel H. Ng, Felix Stahlberg, Xiaochang Peng, Kyle Gorman and Brian Roark},
  journal = {Computational Linguistics},
  year = {2019},
  doi = {10.1162/coli\_a\_00349},
  url = {https://doi.org/10.1162/coli_a_00349}
}
```

- [← previous ChID: A Large-scale Chinese IDiom Dataset for Cloze Test](https://md.research.elamur.ai/free/en/papers/zheng-2019-chid-large-scale-chinese/)
- [next → Argument Generation with Retrieval, Planning, and Realization](https://md.research.elamur.ai/free/en/papers/hua-2019-argument-generation-retrieval-planning-realization/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/)

## References

51 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Allauzen, Cyril and Michael Riley. 2012. A pushdown transducer extension for the OpenFst library. In CIAA, pages 66–77, Porto. [find](https://scholar.google.com/scholar?q=Allauzen%2C+Cyril+and+Michael+Riley%2E+2012%2E+A+pushdown+transducer+extension+for+the+OpenFst+library%2E+In+CIAA%2C+pages+66%E2%80%9377%2C+Porto%2E)
2. Allen, Jonathan, Sharon M. Hunnicutt, and Dennis Klatt. 1987. From Text to Speech: The MITalk System, Cambridge University Press, Cambridge. [find](https://scholar.google.com/scholar?q=Allen%2C+Jonathan%2C+Sharon+M%2E+Hunnicutt%2C+and+Dennis+Klatt%2E+1987%2E+From+Text+to+Speech%3A+The+MITalk+System%2C+Cambridge+University+Press%2C+Cambridge%2E)
3. Arthur, Philip, Graham Neubig, and Satoshi Nakamura. 2016. [Incorporating discrete translation lexicons into neural machine translation](https://md.research.elamur.ai/free/en/papers/arthur-2016-incorporating-discrete-translation-lexicons-into/). In EMNLP, pages 1557–1567, Austin, TX.
4. Arik, Sercan, Mike Chrzanowski, Adam Coates, Gregory Diamos, Andrew Gibiansky, Yongguo Kang, Xian Li, John Miller, Andrew Ng, Jonathan Raiman, Shubho Sengupta, and Mohammad Shoeybi. 2017. Deep voice: Real-time neural text-to-speech. ArXiv:1702.07825. [find](https://scholar.google.com/scholar?q=Arik%2C+Sercan%2C+Mike+Chrzanowski%2C+Adam+Coates%2C+Gregory+Diamos%2C+Andrew+Gibiansky%2C+Yongguo+Kang%2C+Xian+Li%2C+John+Miller%2C+Andrew+Ng%2C+Jonathan+Raiman%2C+Shubho+Sengupta%2C+and+Mohammad+Shoeybi%2E+2017%2E+Deep+voice%3A+)
5. Aw, Ai Ti and Lian Hau Lee. 2012. Personalized normalization for a multilingual chat system. In ACL, pages 31–36, Jeju Island. [find](https://scholar.google.com/scholar?q=Aw%2C+Ai+Ti+and+Lian+Hau+Lee%2E+2012%2E+Personalized+normalization+for+a+multilingual+chat+system%2E+In+ACL%2C+pages+31%E2%80%9336%2C+Jeju+Island%2E)
6. Bahdanau, Dzmitry, Kyunghyun Cho, and Yoshua Bengio. 2015. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). In ICLR, San Diego, CA. Beaufort, Richard, Sophie Roekhaut, Louise-Amélie Cougnon, and Cédrick Fairon. 2010. A hybrid rule/model-based finite-state framework for normal…
7. Chan, William, Navdeep Jaitly, Quoc V. Le, and Oriol Vinyals. 2016. Listen, attend and spell: A neural network for large vocabulary conversational speech recognition. In ICASSP, pages 4960–4964, Shanghai. [find](https://scholar.google.com/scholar?q=Chan%2C+William%2C+Navdeep+Jaitly%2C+Quoc+V%2E+Le%2C+and+Oriol+Vinyals%2E+2016%2E+Listen%2C+attend+and+spell%3A+A+neural+network+for+large+vocabulary+conversational+speech+recognition%2E+In+ICASSP%2C+pages+4960%E2%80%934964%2C+Shang)
8. Chen, Mia Xu, Orhan Firat, Ankur Bapna, Melvin Johnson, Wolfgang Macherey, George Foster, Llion Jones, Niki Parmar, Mike Schuster, Zhifeng Chen, Yonghui Wu, and Macduff Hughes. 2018. [The best of both worlds: Combining recent advances in neural machine translation](https://md.research.elamur.ai/free/en/papers/chen-2018-best-both-worlds-combining-recent/). CoRR, abs/1804.09849.
9. Chiu, Chung-Cheng, Tara N. Sainath, Yonghui Wu, Rohit Prabhavalkar, Patrick Nguyen, Zhifeng Chen, Anjuli Kannan, Ron J. Weiss, Kanishka Rao, Ekaterina Gonina, Navdeep Jaitly, Bo Li, Jan Chorowski, and Michiel Bacchiani. 2017. State-of-the-art speech recognition with sequence-to-sequence models. ArXi… [find](https://scholar.google.com/scholar?q=Chiu%2C+Chung%2DCheng%2C+Tara+N%2E+Sainath%2C+Yonghui+Wu%2C+Rohit+Prabhavalkar%2C+Patrick+Nguyen%2C+Zhifeng+Chen%2C+Anjuli+Kannan%2C+Ron+J%2E+Weiss%2C+Kanishka+Rao%2C+Ekaterina+Gonina%2C+Navdeep+Jaitly%2C+Bo+Li%2C+Jan+Chorowski%2C+and)
10. Choudhury, Monojit, Rahul Saraf, Vijit Jain, Sudesha Sarkar, and Anupam Basu. 2007. Investigation and modeling of the structure of texting language. International Journal of Document Analysis and Recognition, 10:157–174. [find](https://scholar.google.com/scholar?q=Choudhury%2C+Monojit%2C+Rahul+Saraf%2C+Vijit+Jain%2C+Sudesha+Sarkar%2C+and+Anupam+Basu%2E+2007%2E+Investigation+and+modeling+of+the+structure+of+texting+language%2E+International+Journal+of+Document+Analysis+and+Reco)
11. Chrupala, Grzegorz. 2014. Normalizing tweets with edit scripts and recurrent neural embeddings. In ACL, pages 680–686, Baltimore, MD. [find](https://scholar.google.com/scholar?q=Chrupala%2C+Grzegorz%2E+2014%2E+Normalizing+tweets+with+edit+scripts+and+recurrent+neural+embeddings%2E+In+ACL%2C+pages+680%E2%80%93686%2C+Baltimore%2C+MD%2E)
12. Ebden, Peter and Richard Sproat. 2014. The Kestrel TTS text normalization system. Natural Language Engineering, 21(3):1–21. van Esch, Daniel and Richard Sproat. 2017. An expanded taxonomy of semiotic classes for text normalization. In INTERSPEECH, pages 4016–4020, Stockholm. [find](https://scholar.google.com/scholar?q=Ebden%2C+Peter+and+Richard+Sproat%2E+2014%2E+The+Kestrel+TTS+text+normalization+system%2E+Natural+Language+Engineering%2C+21%283%29%3A1%E2%80%9321%2E+van+Esch%2C+Daniel+and+Richard+Sproat%2E+2017%2E+An+expanded+taxonomy+of+semiotic+)
13. Gillick, Larry and Stephen J. Cox. 1989. Some statistical issues in the comparison of speech recognition algorithms. In ICASSP, pages 1520–6149, Glasgow. [find](https://scholar.google.com/scholar?q=Gillick%2C+Larry+and+Stephen+J%2E+Cox%2E+1989%2E+Some+statistical+issues+in+the+comparison+of+speech+recognition+algorithms%2E+In+ICASSP%2C+pages+1520%E2%80%936149%2C+Glasgow%2E)
14. NIPS 2016 tutorial: Generative adversarial networks. ArXiv:1701.00160. — [arXiv:1701.00160](https://arxiv.org/abs/1701.00160) [find](https://scholar.google.com/scholar?q=NIPS+2016+tutorial%3A+Generative+adversarial+networks%2E+ArXiv%3A1701%2E00160%2E+%E2%80%94+%5BarXiv%3A1701%2E00160%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1701%2E00160%29)
15. Goodfellow, Ian, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial networks. In NIPS, pages 2672–2680, Montreal. [find](https://scholar.google.com/scholar?q=Goodfellow%2C+Ian%2C+Jean+Pouget%2DAbadie%2C+Mehdi+Mirza%2C+Bing+Xu%2C+David+Warde%2DFarley%2C+Sherjil+Ozair%2C+Aaron+Courville%2C+and+Yoshua+Bengio%2E+2014%2E+Generative+adversarial+networks%2E+In+NIPS%2C+pages+2672%E2%80%932680%2C+Montr)
16. Gorman, Kyle and Richard Sproat. 2016. Minimally supervised models for number normalization. Transactions of the Association for Computational Linguistics, 4:507–519. Graves, Alex, Greg Wayne, Malcolm Reynolds, Tim Harley, Ivo Danihelka, Agnieszka Grabska-Barwińska, Sergio Gómez Colmenarejo, Edwar… [find](https://scholar.google.com/scholar?q=Gorman%2C+Kyle+and+Richard+Sproat%2E+2016%2E+Minimally+supervised+models+for+number+normalization%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+4%3A507%E2%80%93519%2E+Graves%2C+Alex%2C+Greg+Wayne%2C+Malcolm)
17. Hassan, Hany and Arul Menezes. 2013. Social text normalization using contextual graph random walks. In ACL, pages 1577–1586. Hochreiter, Sepp and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735–1780. [find](https://scholar.google.com/scholar?q=Hassan%2C+Hany+and+Arul+Menezes%2E+2013%2E+Social+text+normalization+using+contextual+graph+random+walks%2E+In+ACL%2C+pages+1577%E2%80%931586%2E+Hochreiter%2C+Sepp+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neu)
18. Hurford, James. 1975. The Linguistic Theory of Numerals, Cambridge University Press, Cambridge. Kobus, Catherine, François Yvon, and Géraldine Damnati. 2008. Normalizing SMS: Are two metaphors better than one? In COLING, pages 441–448, Manchester. [find](https://scholar.google.com/scholar?q=Hurford%2C+James%2E+1975%2E+The+Linguistic+Theory+of+Numerals%2C+Cambridge+University+Press%2C+Cambridge%2E+Kobus%2C+Catherine%2C+Franc%CC%A7ois+Yvon%2C+and+Ge%CC%81raldine+Damnati%2E+2008%2E+Normalizing+SMS%3A+Are+two+metaphors+bette)
19. LeCun, Yann and Yoshua Bengio. 1995. Convolutional networks for images, speech, and time-series, In Michael A. Arbib, editor, The Handbook of Brain Theory and Neural Networks. MIT Press, Cambridge, 255–258. [find](https://scholar.google.com/scholar?q=LeCun%2C+Yann+and+Yoshua+Bengio%2E+1995%2E+Convolutional+networks+for+images%2C+speech%2C+and+time%2Dseries%2C+In+Michael+A%2E+Arbib%2C+editor%2C+The+Handbook+of+Brain+Theory+and+Neural+Networks%2E+MIT+Press%2C+Cambridge%2C+25)
20. Liu, Fei, Fuliang Weng, and Xiao Jiang. 2012. A broad-coverage normalization system for social media language. In ACL, pages 1035–1044, Jeju Island. [find](https://scholar.google.com/scholar?q=Liu%2C+Fei%2C+Fuliang+Weng%2C+and+Xiao+Jiang%2E+2012%2E+A+broad%2Dcoverage+normalization+system+for+social+media+language%2E+In+ACL%2C+pages+1035%E2%80%931044%2C+Jeju+Island%2E)
21. Liu, Fei, Fuliang Weng, Bingqing Wang, and Yang Liu. 2011. Insertion, deletion, or substitution? Normalizing text messages without pre-categorization nor supervision. In ACL, pages 71–76, Portland, OR. [find](https://scholar.google.com/scholar?q=Liu%2C+Fei%2C+Fuliang+Weng%2C+Bingqing+Wang%2C+and+Yang+Liu%2E+2011%2E+Insertion%2C+deletion%2C+or+substitution%3F+Normalizing+text+messages+without+pre%2Dcategorization+nor+supervision%2E+In+ACL%2C+pages+71%E2%80%9376%2C+Portland%2C+OR)
22. Liu, Xiaohua, Ming Zhou, Xiangyang Zhou, Zhongyang Fu, and Furu Wei. 2012. Joint inference of named entity recognition and normalization for tweets. In ACL, pages 526–535, Jeju Island. [find](https://scholar.google.com/scholar?q=Liu%2C+Xiaohua%2C+Ming+Zhou%2C+Xiangyang+Zhou%2C+Zhongyang+Fu%2C+and+Furu+Wei%2E+2012%2E+Joint+inference+of+named+entity+recognition+and+normalization+for+tweets%2E+In+ACL%2C+pages+526%E2%80%93535%2C+Jeju+Island%2E)
23. Mi, Haitao, Baskaran Sankaran, Zhiguo Wang, and Abe Ittycheriah. 2016. Coverage embedding models for neural machine translation. In EMNLP, pages 955–960, Austin, TX. [find](https://scholar.google.com/scholar?q=Mi%2C+Haitao%2C+Baskaran+Sankaran%2C+Zhiguo+Wang%2C+and+Abe+Ittycheriah%2E+2016%2E+Coverage+embedding+models+for+neural+machine+translation%2E+In+EMNLP%2C+pages+955%E2%80%93960%2C+Austin%2C+TX%2E)
24. Min, Wookhee and Bradford Mott. 2015. NCSU SAS WOOKHEE: A deep contextual long-short term memory model for text normalization. In WNUT, pages 111–119, Beijing. [find](https://scholar.google.com/scholar?q=Min%2C+Wookhee+and+Bradford+Mott%2E+2015%2E+NCSU+SAS+WOOKHEE%3A+A+deep+contextual+long%2Dshort+term+memory+model+for+text+normalization%2E+In+WNUT%2C+pages+111%E2%80%93119%2C+Beijing%2E)
25. Munteanu, Cosmin, Ronald Baecker, Gerald Penn, Elaine Toms, and David James. 2006. The effect of speech recognition accuracy rates on the usefulness and usability of webcast archives. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 493–502. [find](https://scholar.google.com/scholar?q=Munteanu%2C+Cosmin%2C+Ronald+Baecker%2C+Gerald+Penn%2C+Elaine+Toms%2C+and+David+James%2E+2006%2E+The+effect+of+speech+recognition+accuracy+rates+on+the+usefulness+and+usability+of+webcast+archives%2E+In+Proceedings+o)
26. Ng, Axel H., Kyle Gorman, and Richard Sproat. 2017. Minimally supervised written-to-spoken text normalization. In ASRU, pages 665–670, Okinawa. [find](https://scholar.google.com/scholar?q=Ng%2C+Axel+H%2E%2C+Kyle+Gorman%2C+and+Richard+Sproat%2E+2017%2E+Minimally+supervised+written%2Dto%2Dspoken+text+normalization%2E+In+ASRU%2C+pages+665%E2%80%93670%2C+Okinawa%2E)
27. Pennell, Deana and Yang Liu. 2011. A character-level machine translation approach for normalization of SMS abbreviations. In IJCNLP, pages 974–982, Chiang Mai. [find](https://scholar.google.com/scholar?q=Pennell%2C+Deana+and+Yang+Liu%2E+2011%2E+A+character%2Dlevel+machine+translation+approach+for+normalization+of+SMS+abbreviations%2E+In+IJCNLP%2C+pages+974%E2%80%93982%2C+Chiang+Mai%2E)
28. Pramanik, Subhojeet and Aman Hussain. 2018. Text normalization using memory augmented neural networks. ArXiv:1806.00044. — [arXiv:1806.00044](https://arxiv.org/abs/1806.00044) [find](https://scholar.google.com/scholar?q=Pramanik%2C+Subhojeet+and+Aman+Hussain%2E+2018%2E+Text+normalization+using+memory+augmented+neural+networks%2E+ArXiv%3A1806%2E00044%2E+%E2%80%94+%5BarXiv%3A1806%2E00044%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1806%2E00044%29)
29. Pusateri, Ernest, Bharat Ram Ambati, Elizabeth Brooks, Ondrej Platek, Donald McAllaster, and Venki Nagesha. 2017. A mostly data-driven approach to inverse text normalization. In INTERSPEECH, pages 2784–2788, Stockholm. [find](https://scholar.google.com/scholar?q=Pusateri%2C+Ernest%2C+Bharat+Ram+Ambati%2C+Elizabeth+Brooks%2C+Ondrej+Platek%2C+Donald+McAllaster%2C+and+Venki+Nagesha%2E+2017%2E+A+mostly+data%2Ddriven+approach+to+inverse+text+normalization%2E+In+INTERSPEECH%2C+pages+278)
30. Roark, Brian and Richard Sproat. 2014. Hippocratic abbreviation expansion. In ACL, pages 364–369, Baltimore, MD. [find](https://scholar.google.com/scholar?q=Roark%2C+Brian+and+Richard+Sproat%2E+2014%2E+Hippocratic+abbreviation+expansion%2E+In+ACL%2C+pages+364%E2%80%93369%2C+Baltimore%2C+MD%2E)
31. Roark, Brian, Richard Sproat, Cyril Allauzen, Michael Riley, Jeffrey Sorensen, and Terry Tai. 2012. The OpenGrm open-source finite-state grammar software libraries. In ACL, 61–66, Jeju Island. Sak, Haşim, Françoise Beaufays, Kaisuke Nakajima, and Cyril Allauzen. 2013. Language model verbalization… [find](https://scholar.google.com/scholar?q=Roark%2C+Brian%2C+Richard+Sproat%2C+Cyril+Allauzen%2C+Michael+Riley%2C+Jeffrey+Sorensen%2C+and+Terry+Tai%2E+2012%2E+The+OpenGrm+open%2Dsource+finite%2Dstate+grammar+software+libraries%2E+In+ACL%2C+61%E2%80%9366%2C+Jeju+Island%2E+Sak%2C+Ha)
32. Schuster, Michael and Kaisuke Nakajima. 2012. Japanese and Korean voice search. In ICASSP, pages 5149–5152, Kyoto. [find](https://scholar.google.com/scholar?q=Schuster%2C+Michael+and+Kaisuke+Nakajima%2E+2012%2E+Japanese+and+Korean+voice+search%2E+In+ICASSP%2C+pages+5149%E2%80%935152%2C+Kyoto%2E)
33. Sennrich, Rico, Barry Haddow, and Alexandra Birch. 2016. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In ACL, pages 1715–1725, Berlin.
34. Shugrina, Masha. 2010. Formatting time-aligned ASR transcripts for readability. In NAACL, pages 198–206, Los Angeles, CA. Sotelo, Jose, Soroush Mehri, Kundan Kumar, João Felipe Santos, Kyle Kastner, Aaron Courville, and Yoshua Bengio. 2017. Char2wav: End-to-end speech synthesis. In ICLR, Toulon. [find](https://scholar.google.com/scholar?q=Shugrina%2C+Masha%2E+2010%2E+Formatting+time%2Daligned+ASR+transcripts+for+readability%2E+In+NAACL%2C+pages+198%E2%80%93206%2C+Los+Angeles%2C+CA%2E+Sotelo%2C+Jose%2C+Soroush+Mehri%2C+Kundan+Kumar%2C+Joa%CC%83o+Felipe+Santos%2C+Kyle+Kastner%2C+)
35. Sproat, Richard. 1996. Multilingual text analysis for text-to-speech synthesis. Natural Language Engineering, 2(4):369–380. Sproat, Richard, editor . 1997. [find](https://scholar.google.com/scholar?q=Sproat%2C+Richard%2E+1996%2E+Multilingual+text+analysis+for+text%2Dto%2Dspeech+synthesis%2E+Natural+Language+Engineering%2C+2%284%29%3A369%E2%80%93380%2E+Sproat%2C+Richard%2C+editor+%2E+1997%2E)
36. Multilingual Text-to-Speech Synthesis: The Bell Labs Approach, Kluwer Academic Publishers, Boston. Sproat, Richard, Alan Black, Stanley Chen, Shankar Kumar, Mari Ostendorf, and Christopher Richards. 2001. Normalization of non-standard words. Computer Speech and Language, 15(3):287–333. [find](https://scholar.google.com/scholar?q=Multilingual+Text%2Dto%2DSpeech+Synthesis%3A+The+Bell+Labs+Approach%2C+Kluwer+Academic+Publishers%2C+Boston%2E+Sproat%2C+Richard%2C+Alan+Black%2C+Stanley+Chen%2C+Shankar+Kumar%2C+Mari+Ostendorf%2C+and+Christopher+Richards%2E+2)
37. Sproat, Richard and Kyle Gorman. 2018. A brief summary of the Kaggle text normalization challenge. [http://blog.kaggle.com/2018/02/07/](http://blog.kaggle.com/2018/02/07/) a-brief-summary-of-the-kaggle-text-normalization-challenge/. [↗](http://blog.kaggle.com/2018/02/07/)
38. Sproat, Richard and Keith Hall. 2014. Applications of maximum entropy rankers to problems in spoken language processing. In INTERSPEECH, pages 761–764, Singapore. [find](https://scholar.google.com/scholar?q=Sproat%2C+Richard+and+Keith+Hall%2E+2014%2E+Applications+of+maximum+entropy+rankers+to+problems+in+spoken+language+processing%2E+In+INTERSPEECH%2C+pages+761%E2%80%93764%2C+Singapore%2E)
39. Sproat, Richard and Navdeep Jaitly. 2016. RNN approaches to text normalization: A challenge. ArXiv:1611.00068. — [arXiv:1611.00068](https://arxiv.org/abs/1611.00068) [find](https://scholar.google.com/scholar?q=Sproat%2C+Richard+and+Navdeep+Jaitly%2E+2016%2E+RNN+approaches+to+text+normalization%3A+A+challenge%2E+ArXiv%3A1611%2E00068%2E+%E2%80%94+%5BarXiv%3A1611%2E00068%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1611%2E00068%29)
40. Sproat, Richard and Navdeep Jaitly. 2017. An RNN model of text normalization. In INTERSPEECH, pages 754–758, Stockholm. [find](https://scholar.google.com/scholar?q=Sproat%2C+Richard+and+Navdeep+Jaitly%2E+2017%2E+An+RNN+model+of+text+normalization%2E+In+INTERSPEECH%2C+pages+754%E2%80%93758%2C+Stockholm%2E)
41. Taylor, Paul. 2009. Text-to-Speech Synthesis, Cambridge University Press, Cambridge. [find](https://scholar.google.com/scholar?q=Taylor%2C+Paul%2E+2009%2E+Text%2Dto%2DSpeech+Synthesis%2C+Cambridge+University+Press%2C+Cambridge%2E)
42. Tu, Zhaopeng, Yang Liu, Lifeng Shang, Xiaohua Liu, and Hang Li. 2017. Neural machine translation with reconstruction. In AAAI, pages 3097–3103. [find](https://scholar.google.com/scholar?q=Tu%2C+Zhaopeng%2C+Yang+Liu%2C+Lifeng+Shang%2C+Xiaohua+Liu%2C+and+Hang+Li%2E+2017%2E+Neural+machine+translation+with+reconstruction%2E+In+AAAI%2C+pages+3097%E2%80%933103%2E)
43. Tu, Zhaopeng, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. 2016. [Modeling coverage for neural machine translation](https://md.research.elamur.ai/free/en/papers/tu-2016-modeling-coverage-neural-machine-translation/). In ACL, pages 76–85, San Francisco, CA.
44. Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). CoRR, abs/1706.03762.
45. Wang, Chunqi and Bo Xu. 2017. Convolutional neural network with word embeddings for Chinese word segmentation. In IJCNLP, pages 163–172, Taipei. [find](https://scholar.google.com/scholar?q=Wang%2C+Chunqi+and+Bo+Xu%2E+2017%2E+Convolutional+neural+network+with+word+embeddings+for+Chinese+word+segmentation%2E+In+IJCNLP%2C+pages+163%E2%80%93172%2C+Taipei%2E)
46. Wu, Lijun, Yingce Xia, Zhao Li, Fei Tian, Tao Qin, Jianhuang Lai, and Liu Tie-Yan. 2017. Adversarial neural machine translation. ArXiv:1704.06933. — [arXiv:1704.06933](https://arxiv.org/abs/1704.06933) [find](https://scholar.google.com/scholar?q=Wu%2C+Lijun%2C+Yingce+Xia%2C+Zhao+Li%2C+Fei+Tian%2C+Tao+Qin%2C+Jianhuang+Lai%2C+and+Liu+Tie%2DYan%2E+2017%2E+Adversarial+neural+machine+translation%2E+ArXiv%3A1704%2E06933%2E+%E2%80%94+%5BarXiv%3A1704%2E06933%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1704%2E06933)
47. Wu, Yonghui, Mike Schuster, Zhifeng Chen, Quoc V. Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, Jeff Klingner, Apurva Shah, Melvin Johnson, Xiaobing Liu, Lukasz Kaiser, Stephan Gouws, Yoshikiyo Kato, Taku Kudo, Hideto Kazawa, Keith Stevens, George Kurian,… [find](https://scholar.google.com/scholar?q=Wu%2C+Yonghui%2C+Mike+Schuster%2C+Zhifeng+Chen%2C+Quoc+V%2E+Le%2C+Mohammad+Norouzi%2C+Wolfgang+Macherey%2C+Maxim+Krikun%2C+Yuan+Cao%2C+Qin+Gao%2C+Klaus+Macherey%2C+Jeff+Klingner%2C+Apurva+Shah%2C+Melvin+Johnson%2C+Xiaobing+Liu%2C+Lu)
48. Xia, Yunqing, Kam-Fai Wong, and Wenjie Li. 2006. A phonetic-based approach to Chinese chat text normalization. In ACL, pages 993–1000, Sydney. [find](https://scholar.google.com/scholar?q=Xia%2C+Yunqing%2C+Kam%2DFai+Wong%2C+and+Wenjie+Li%2E+2006%2E+A+phonetic%2Dbased+approach+to+Chinese+chat+text+normalization%2E+In+ACL%2C+pages+993%E2%80%931000%2C+Sydney%2E)
49. Xie, Ziang. 2017. Neural text generation: A practical guide. ArXiv:1711.09534. — [arXiv:1711.09534](https://arxiv.org/abs/1711.09534) [find](https://scholar.google.com/scholar?q=Xie%2C+Ziang%2E+2017%2E+Neural+text+generation%3A+A+practical+guide%2E+ArXiv%3A1711%2E09534%2E+%E2%80%94+%5BarXiv%3A1711%2E09534%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1711%2E09534%29)
50. Yang, Yi and Jacob Eisenstein. 2013. A log-linear model for unsupervised text normalization. In EMNLP, 61–72, Waikoloa Beach, HI. [find](https://scholar.google.com/scholar?q=Yang%2C+Yi+and+Jacob+Eisenstein%2E+2013%2E+A+log%2Dlinear+model+for+unsupervised+text+normalization%2E+In+EMNLP%2C+61%E2%80%9372%2C+Waikoloa+Beach%2C+HI%2E)
51. Yang, Zhen, Wei Chen, Feng Wang, and Bo Xu. 2018. Improving neural machine translation with conditional sequence generative adversarial nets. ArXiv:1703.04887. Yolchuyeva, Sevinj, Géza Németh, and Bálint Gyires-Tóth. 2018. Text normalization with convolutional neural networks. International Jour… [find](https://scholar.google.com/scholar?q=Yang%2C+Zhen%2C+Wei+Chen%2C+Feng+Wang%2C+and+Bo+Xu%2E+2018%2E+Improving+neural+machine+translation+with+conditional+sequence+generative+adversarial+nets%2E+ArXiv%3A1703%2E04887%2E+Yolchuyeva%2C+Sevinj%2C+Ge%CC%81za+Ne%CC%81meth%2C+and+B)

*Updated 29 October 2026*