---
title: "Deterministic Non-Autoregressive Neural Sequence — Lee 2018"
description: "Autoregressive sequence models such as the Transformer generate output one token at a time."
authors: ["Jason Lee", "Elman Mansimov", "Kyunghyun Cho"]
year: 2018
venue: "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D18-1149/"
doi: "10.18653/v1/d18-1149"
arxiv: "1802.06901"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/lee-2018-deterministic-non-autoregressive-neural-sequence/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lee-2018-deterministic-non-autoregressive-neural-sequence/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/) / Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement

# Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement

**Authors:** [Jason Lee](https://md.research.elamur.ai/free/en/authors/jason-lee/), Elman Mansimov, [Kyunghyun Cho](https://md.research.elamur.ai/free/en/authors/kyunghyun-cho/) · **Year:** 2018 · **Venue:** Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D18-1149/) · [PDF](https://aclanthology.org/D18-1149.pdf) · [DOI 10.18653/v1/d18-1149](https://doi.org/10.18653/v1/d18-1149) · [arXiv 1802.06901](https://arxiv.org/abs/1802.06901) · [OpenAlex](https://openalex.org/W2963434219)

**Citations:** 364 citations (openalex-snapshot · 2026-10-14) · [183](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Autoregressive sequence models such as the Transformer generate output one token at a time. Decoding is therefore slow and hard to parallelize, and beam search adds further cost. Lee, Mansimov and Cho propose a deterministic non-autoregressive model that predicts the whole target sequence at once and then improves it over several passes. They frame the model in two ways: as a latent variable model whose intermediate variables are draft outputs, and as a conditional denoising autoencoder. Training combines both views by randomly switching between lower-bound terms and reconstruction from corrupted references. It also uses knowledge distillation from an autoregressive teacher and a separate target-length predictor. At inference time, a single shared refinement network runs repeatedly until the output stops changing, so the number of steps adapts to each input. On machine translation, the model decodes about 2–3× faster than autoregressive beam search and retains 90–95% of its quality. On MS COCO captioning, it keeps about 85% of the quality. This shows that parallel decoding can offer a practical speed–quality trade-off for any sequence generation task.

## Key points
- Evaluated on IWSLT'16 En↔De, WMT'16 En↔Ro and WMT'14 En↔De for translation, and on MS COCO for image captioning. The baseline is a Transformer.
- Translation decoding is roughly 2–3× faster than beam search on both GPU and CPU while preserving 90–95% of translation quality. Captioning is about 3× faster on GPU and 5× faster on CPU, with roughly 85% of caption quality.
- The training objective mixes the latent-variable loss and the denoising loss per refinement step using a Bernoulli variable with probability p_DAE. The corruption process, adapted from Hill et al. (2016), repeats tokens, substitutes random tokens or swaps adjacent tokens.
- Refinement stops once the Jaccard distance between consecutive outputs falls below a threshold. Qualitatively, each pass adds missing words and removes spurious ones.
- The code, preprocessed data and trained models are released at github.com/nyu-dl/dl4mt-nonauto.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

183 papers of our library cite it (of the 140884 whose bibliographies we read); 9 of them have a page here, 174 not yet.

- [Iterative Translation Refinement with Large Language Models](https://md.research.elamur.ai/free/en/papers/chen-2024-iterative-translation-refinement-large-language/)
- [Seamless: Multilingual Expressive and Streaming Speech Translation](https://md.research.elamur.ai/free/en/papers/communication-2023-seamless-multilingual-expressive-streaming-speech/)
- [Mask-Predict: Parallel Decoding of Conditional Masked Language Models](https://md.research.elamur.ai/free/en/papers/ghazvininejad-2019-mask-predict-parallel-decoding-conditional/)
- [Deep Encoder, Shallow Decoder: Reevaluating Non-autoregressive Machine Translation](https://md.research.elamur.ai/free/en/papers/kasai-2020-deep-encoder-shallow-decoder-reevaluating/)
- [Contextualized Perturbation for Textual Adversarial Attack](https://md.research.elamur.ai/free/en/papers/li-2021-contextualized-perturbation-textual-adversarial-attack/)
- [End-to-End Non-Autoregressive Neural Machine Translation with Connectionist Temporal Classification](https://md.research.elamur.ai/free/en/papers/libovicky-2018-end-end-non-autoregressive-neural/)
- [Glancing Transformer for Non-Autoregressive Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/qian-2021-glancing-transformer-non-autoregressive-neural/)
- [Recursive Non-Autoregressive Graph-to-Graph Transformer for Dependency Parsing with Iterative Refinement](https://md.research.elamur.ai/free/en/papers/ud-2021-mohammadshahi-recursive-non-autoregressive-graph-graph/)
- [Pay Less Attention with Lightweight and Dynamic Convolutions](https://md.research.elamur.ai/free/en/papers/wu-2019-pay-less-attention-lightweight-dynamic/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{lee2018deterministic,
  title = {{Deterministic Non-Autoregressive Neural Sequence Modeling by Iterative Refinement}},
  author = {Lee, Jason and Mansimov, Elman and Cho, Kyunghyun},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1149},
  eprint = {1802.06901},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/d18-1149}
}
```

- [← previous Derivational Relations in Arabic WordNet](https://md.research.elamur.ai/free/en/papers/aclbulk-2018-batita-derivational-relations-arabic-wordnet/)
- [next → Developing the Bangla RST Discourse Treebank](https://md.research.elamur.ai/free/en/papers/das-2018-developing-bangla-rst-discourse-treebank/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/)

## References

38 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Guillaume Alain and Yoshua Bengio. 2014. What regularized auto-encoders learn from the data-generating distribution. The Journal of Machine Learning Research, 15(1). [find](https://scholar.google.com/scholar?q=Guillaume+Alain+and+Yoshua+Bengio%2E+2014%2E+What+regularized+auto%2Dencoders+learn+from+the+data%2Dgenerating+distribution%2E+The+Journal+of+Machine+Learning+Research%2C+15%281%29%2E)
2. Mikel Artetxe, Gorka Labaka, Eneko Agirre, and Kyunghyun Cho. 2017. Unsupervised neural machine translation. arXiv preprint arXiv:1710.11041. [find](https://scholar.google.com/scholar?q=Mikel+Artetxe%2C+Gorka+Labaka%2C+Eneko+Agirre%2C+and+Kyunghyun+Cho%2E+2017%2E+Unsupervised+neural+machine+translation%2E+arXiv+preprint+arXiv%3A1710%2E11041%2E) [our card](https://md.research.elamur.ai/free/en/papers/artetxe-2017-unsupervised-neural-machine-translation/)
3. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2014%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+arXiv+preprint+arXiv%3A1409%2E0473%2E) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
4. Florian Bordes, Sina Honari, and Pascal Vincent. 2017. Learning to generate samples from noise through infusion training. In ICLR. [find](https://scholar.google.com/scholar?q=Florian+Bordes%2C+Sina+Honari%2C+and+Pascal+Vincent%2E+2017%2E+Learning+to+generate+samples+from+noise+through+infusion+training%2E+In+ICLR%2E)
5. Chung-Cheng Chiu, Tara N Sainath, Yonghui Wu, Rohit Prabhavalkar, Patrick Nguyen, Zhifeng Chen, Anjuli Kannan, Ron J Weiss, Kanishka Rao, Katya Gonina, et al. 2017. State-of-the-art speech recognition with sequence-to-sequence models. arXiv preprint arXiv:1712.01769. — [arXiv:1712.01769](https://arxiv.org/abs/1712.01769) [find](https://scholar.google.com/scholar?q=Chung%2DCheng+Chiu%2C+Tara+N+Sainath%2C+Yonghui+Wu%2C+Rohit+Prabhavalkar%2C+Patrick+Nguyen%2C+Zhifeng+Chen%2C+Anjuli+Kannan%2C+Ron+J+Weiss%2C+Kanishka+Rao%2C+Katya+Gonina%2C+et+al%2E+2017%2E+State%2Dof%2Dthe%2Dart+speech+recognition)
6. Kyunghyun Cho. 2016. Noisy parallel approximate decoding for conditional recurrent language model. arXiv preprint arXiv:1605.03835. — [arXiv:1605.03835](https://arxiv.org/abs/1605.03835) [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2E+2016%2E+Noisy+parallel+approximate+decoding+for+conditional+recurrent+language+model%2E+arXiv+preprint+arXiv%3A1605%2E03835%2E+%E2%80%94+%5BarXiv%3A1605%2E03835%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1605%2E03835%29)
7. Kyunghyun Cho, Aaron Courville, and Yoshua Bengio. 2015. Describing multimedia content using attention-based encoder-decoder networks. IEEE Transactions on Multimedia, 17(11). [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2C+Aaron+Courville%2C+and+Yoshua+Bengio%2E+2015%2E+Describing+multimedia+content+using+attention%2Dbased+encoder%2Ddecoder+networks%2E+IEEE+Transactions+on+Multimedia%2C+17%2811%29%2E)
8. Jan K Chorowski, Dzmitry Bahdanau, Dmitriy Serdyuk, Kyunghyun Cho, and Yoshua Bengio. 2015. Attention-based models for speech recognition. In NIPS. [find](https://scholar.google.com/scholar?q=Jan+K+Chorowski%2C+Dzmitry+Bahdanau%2C+Dmitriy+Serdyuk%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2015%2E+Attention%2Dbased+models+for+speech+recognition%2E+In+NIPS%2E) [our card](https://md.research.elamur.ai/free/en/papers/chorowski-2015-attention-based-models-speech-recognition/)
9. Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In CVPR. [find](https://scholar.google.com/scholar?q=Jia+Deng%2C+Wei+Dong%2C+Richard+Socher%2C+Li%2DJia+Li%2C+Kai+Li%2C+and+Li+Fei%2DFei%2E+2009%2E+Imagenet%3A+A+large%2Dscale+hierarchical+image+database%2E+In+CVPR%2E)
10. Chris Dyer, Victor Chahuneau, and Noah A Smith. 2013. A simple, fast, and effective reparameterization of ibm model 2. In ACL. [find](https://scholar.google.com/scholar?q=Chris+Dyer%2C+Victor+Chahuneau%2C+and+Noah+A+Smith%2E+2013%2E+A+simple%2C+fast%2C+and+effective+reparameterization+of+ibm+model+2%2E+In+ACL%2E)
11. David Grangier and Michael Auli. 2017. Quickedit: Editing text & translations via simple delete actions. arXiv preprint arXiv:1711.04805. — [arXiv:1711.04805](https://arxiv.org/abs/1711.04805) [find](https://scholar.google.com/scholar?q=David+Grangier+and+Michael+Auli%2E+2017%2E+Quickedit%3A+Editing+text+%26+translations+via+simple+delete+actions%2E+arXiv+preprint+arXiv%3A1711%2E04805%2E+%E2%80%94+%5BarXiv%3A1711%2E04805%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1711%2E04805%29)
12. Jiatao Gu, James Bradbury, Caiming Xiong, Victor OK Li, and Richard Socher. 2017. Non-autoregressive neural machine translation. arXiv preprint arXiv:1711.02281. [find](https://scholar.google.com/scholar?q=Jiatao+Gu%2C+James+Bradbury%2C+Caiming+Xiong%2C+Victor+OK+Li%2C+and+Richard+Socher%2E+2017%2E+Non%2Dautoregressive+neural+machine+translation%2E+arXiv+preprint+arXiv%3A1711%2E02281%2E) [our card](https://md.research.elamur.ai/free/en/papers/gu-2017-non-autoregressive-neural-machine-translation/)
13. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In CVPR. [find](https://scholar.google.com/scholar?q=Kaiming+He%2C+Xiangyu+Zhang%2C+Shaoqing+Ren%2C+and+Jian+Sun%2E+2016%2E+Deep+residual+learning+for+image+recognition%2E+In+CVPR%2E)
14. Felix Hill, Kyunghyun Cho, and Anna Korhonen. 2016. Learning distributed representations of sentences from unlabelled data. In NAACL. [find](https://scholar.google.com/scholar?q=Felix+Hill%2C+Kyunghyun+Cho%2C+and+Anna+Korhonen%2E+2016%2E+Learning+distributed+representations+of+sentences+from+unlabelled+data%2E+In+NAACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/hill-2016-learning-distributed-representations-sentences-unlabelled/)
15. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531. [find](https://scholar.google.com/scholar?q=Geoffrey+Hinton%2C+Oriol+Vinyals%2C+and+Jeff+Dean%2E+2015%2E+Distilling+the+knowledge+in+a+neural+network%2E+arXiv+preprint+arXiv%3A1503%2E02531%2E) [our card](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/)
16. Cong Duy Vu Hoang, Gholamreza Haffari, and Trevor Cohn. 2017. Decoding as continuous optimization in neural machine translation. arXiv preprint arXiv:1701.02854. — [arXiv:1701.02854](https://arxiv.org/abs/1701.02854) [find](https://scholar.google.com/scholar?q=Cong+Duy+Vu+Hoang%2C+Gholamreza+Haffari%2C+and+Trevor+Cohn%2E+2017%2E+Decoding+as+continuous+optimization+in+neural+machine+translation%2E+arXiv+preprint+arXiv%3A1701%2E02854%2E+%E2%80%94+%5BarXiv%3A1701%2E02854%5D%28https%3A%2F%2Farxiv%2Eorg)
17. Norman P Jouppi, Cliff Young, Nishant Patil, David Patterson, Gaurav Agrawal, Raminder Bajwa, Sarah Bates, Suresh Bhatia, Nan Boden, Al Borchers, et al. 2017. In-datacenter performance analysis of a tensor processing unit. In Proceedings of the 44th Annual International Symposium on Computer Archite… [find](https://scholar.google.com/scholar?q=Norman+P+Jouppi%2C+Cliff+Young%2C+Nishant+Patil%2C+David+Patterson%2C+Gaurav+Agrawal%2C+Raminder+Bajwa%2C+Sarah+Bates%2C+Suresh+Bhatia%2C+Nan+Boden%2C+Al+Borchers%2C+et+al%2E+2017%2E+In%2Ddatacenter+performance+analysis+of+a+t)
18. Łukasz Kaiser and Samy Bengio. 2016. Can active memory replace attention? In NIPS. [find](https://scholar.google.com/scholar?q=%C5%81ukasz+Kaiser+and+Samy+Bengio%2E+2016%2E+Can+active+memory+replace+attention%3F+In+NIPS%2E)
19. Łukasz Kaiser and Ilya Sutskever. 2015. Neural GPUs learn algorithms. arXiv preprint arXiv:1511.08228. [find](https://scholar.google.com/scholar?q=%C5%81ukasz+Kaiser+and+Ilya+Sutskever%2E+2015%2E+Neural+GPUs+learn+algorithms%2E+arXiv+preprint+arXiv%3A1511%2E08228%2E)
20. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent continuous translation models. In EMNLP. [find](https://scholar.google.com/scholar?q=Nal+Kalchbrenner+and+Phil+Blunsom%2E+2013%2E+Recurrent+continuous+translation+models%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2013-recurrent-continuous-translation-models/)
21. Andrej Karpathy and Fei-Fei Li. 2015. Deep visual-semantic alignments for generating image descriptions. In CVPR. [find](https://scholar.google.com/scholar?q=Andrej+Karpathy+and+Fei%2DFei+Li%2E+2015%2E+Deep+visual%2Dsemantic+alignments+for+generating+image+descriptions%2E+In+CVPR%2E) [our card](https://md.research.elamur.ai/free/en/papers/karpathy-2014-deep-visual-semantic-alignments-generating/)
22. Yoon Kim and Alexander M Rush. 2016. Sequence-level knowledge distillation. arXiv preprint arXiv:1606.07947. [find](https://scholar.google.com/scholar?q=Yoon+Kim+and+Alexander+M+Rush%2E+2016%2E+Sequence%2Dlevel+knowledge+distillation%2E+arXiv+preprint+arXiv%3A1606%2E07947%2E) [our card](https://md.research.elamur.ai/free/en/papers/kim-2016-sequence-level-knowledge-distillation/)
23. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. [find](https://scholar.google.com/scholar?q=Diederik+P+Kingma+and+Jimmy+Ba%2E+2014%2E+Adam%3A+A+method+for+stochastic+optimization%2E+arXiv+preprint+arXiv%3A1412%2E6980%2E)
24. Diederik P Kingma, Tim Salimans, Rafal Jozefowicz, Xi Chen, Ilya Sutskever, and Max Welling. 2016. Improved variational inference with inverse autoregressive flow. In NIPS. [find](https://scholar.google.com/scholar?q=Diederik+P+Kingma%2C+Tim+Salimans%2C+Rafal+Jozefowicz%2C+Xi+Chen%2C+Ilya+Sutskever%2C+and+Max+Welling%2E+2016%2E+Improved+variational+inference+with+inverse+autoregressive+flow%2E+In+NIPS%2E)
25. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, et al. 2007. Moses: Open source toolkit for statistical machine translation. In ACL. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+et+al%2E+2007%2E+Moses%3A+Open+source+toolkit+fo)
26. Guillaume Lample, Ludovic Denoyer, and Marc’Aurelio Ranzato. 2017. Unsupervised machine translation using monolingual corpora only. arXiv preprint arXiv:1711.00043. [find](https://scholar.google.com/scholar?q=Guillaume+Lample%2C+Ludovic+Denoyer%2C+and+Marc%E2%80%99Aurelio+Ranzato%2E+2017%2E+Unsupervised+machine+translation+using+monolingual+corpora+only%2E+arXiv+preprint+arXiv%3A1711%2E00043%2E) [our card](https://md.research.elamur.ai/free/en/papers/lample-2017-unsupervised-machine-translation-using-monolingual/)
27. T.Y. Lin, M. Maire, S. Belongie, J. Hays, P. Perona, D. Ramanan, P. Dollár, and C. L. Zitnick. 2014. Microsoft COCO: Common objects in context. In ECCV. [find](https://scholar.google.com/scholar?q=T%2EY%2E+Lin%2C+M%2E+Maire%2C+S%2E+Belongie%2C+J%2E+Hays%2C+P%2E+Perona%2C+D%2E+Ramanan%2C+P%2E+Dolla%CC%81r%2C+and+C%2E+L%2E+Zitnick%2E+2014%2E+Microsoft+COCO%3A+Common+objects+in+context%2E+In+ECCV%2E) [our card](https://md.research.elamur.ai/free/en/papers/lin-2014-microsoft-coco-common-objects-context/)
28. Tomáš Mikolov, Martin Karafiát, Lukáš Burget, Jan Černockỳ, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In Eleventh Annual Conference of the International Speech Communication Association. [find](https://scholar.google.com/scholar?q=Toma%CC%81s%CC%8C+Mikolov%2C+Martin+Karafia%CC%81t%2C+Luka%CC%81s%CC%8C+Burget%2C+Jan+C%CC%8Cernocky%CC%80%2C+and+Sanjeev+Khudanpur%2E+2010%2E+Recurrent+neural+network+based+language+model%2E+In+Eleventh+Annual+Conference+of+the+International+Speech)
29. Roman Novak, Michael Auli, and David Grangier. 2016. Iterative refinement for machine translation. arXiv preprint arXiv:1610.06602. — [arXiv:1610.06602](https://arxiv.org/abs/1610.06602) [find](https://scholar.google.com/scholar?q=Roman+Novak%2C+Michael+Auli%2C+and+David+Grangier%2E+2016%2E+Iterative+refinement+for+machine+translation%2E+arXiv+preprint+arXiv%3A1610%2E06602%2E+%E2%80%94+%5BarXiv%3A1610%2E06602%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1610%2E06602%29)
30. Aaron van den Oord, Sander Dieleman, Heiga Zen, Karen Simonyan, Oriol Vinyals, Alex Graves, Nal Kalchbrenner, Andrew Senior, and Koray Kavukcuoglu. 2016. Wavenet: A generative model for raw audio. arXiv preprint arXiv:1609.03499. [find](https://scholar.google.com/scholar?q=Aaron+van+den+Oord%2C+Sander+Dieleman%2C+Heiga+Zen%2C+Karen+Simonyan%2C+Oriol+Vinyals%2C+Alex+Graves%2C+Nal+Kalchbrenner%2C+Andrew+Senior%2C+and+Koray+Kavukcuoglu%2E+2016%2E+Wavenet%3A+A+generative+model+for+raw+audio%2E+arX) [our card](https://md.research.elamur.ai/free/en/papers/oord-2016-wavenet-generative-model-raw-audio/)
31. Aaron van den Oord, Yazhe Li, Igor Babuschkin, Karen Simonyan, Oriol Vinyals, Koray Kavukcuoglu, George van den Driessche, Edward Lockhart, Luis C Cobo, Florian Stimberg, et al. 2017. Parallel wavenet: Fast high-fidelity speech synthesis. arXiv preprint arXiv:1711.10433. — [arXiv:1711.10433](https://arxiv.org/abs/1711.10433) [find](https://scholar.google.com/scholar?q=Aaron+van+den+Oord%2C+Yazhe+Li%2C+Igor+Babuschkin%2C+Karen+Simonyan%2C+Oriol+Vinyals%2C+Koray+Kavukcuoglu%2C+George+van+den+Driessche%2C+Edward+Lockhart%2C+Luis+C+Cobo%2C+Florian+Stimberg%2C+et+al%2E+2017%2E+Parallel+wavenet)
32. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In ACL. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
33. Holger Schwenk. 2012. Continuous space translation models for phrase-based statistical machine translation. Proceedings of COLING 2012: Posters. [find](https://scholar.google.com/scholar?q=Holger+Schwenk%2E+2012%2E+Continuous+space+translation+models+for+phrase%2Dbased+statistical+machine+translation%2E+Proceedings+of+COLING+2012%3A+Posters%2E)
34. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. Neural machine translation of rare words with subword units. In ACL. [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016%2E+Neural+machine+translation+of+rare+words+with+subword+units%2E+In+ACL%2E)
35. Rupesh Kumar Srivastava, Klaus Greff, and Jürgen Schmidhuber. 2015. Highway networks. arXiv preprint arXiv:1505.00387. [find](https://scholar.google.com/scholar?q=Rupesh+Kumar+Srivastava%2C+Klaus+Greff%2C+and+Ju%CC%88rgen+Schmidhuber%2E+2015%2E+Highway+networks%2E+arXiv+preprint+arXiv%3A1505%2E00387%2E) [our card](https://md.research.elamur.ai/free/en/papers/srivastava-2015-highway-networks/)
36. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In NIPS. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+NIPS%2E)
37. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+NIPS%2E)
38. Yingce Xia, Fei Tian, Lijun Wu, Jianxin Lin, Tao Qin, Nenghai Yu, and Tie-Yan Liu. 2017. Deliberation networks: Sequence generation beyond one-pass decoding. In NIPS. [find](https://scholar.google.com/scholar?q=Yingce+Xia%2C+Fei+Tian%2C+Lijun+Wu%2C+Jianxin+Lin%2C+Tao+Qin%2C+Nenghai+Yu%2C+and+Tie%2DYan+Liu%2E+2017%2E+Deliberation+networks%3A+Sequence+generation+beyond+one%2Dpass+decoding%2E+In+NIPS%2E)