---
title: "Lexically Constrained Neural Machine Translation with"
description: "The authors build on the Levenshtein Transformer (LevT), a non-autoregressive model that refines its… 17 references, cited by 54 papers here."
authors: ["Raymond Hendy Susanto", "Shamil Chollampatt", "Liling Tan"]
year: 2020
venue: "Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/2020.acl-main.325/"
doi: "10.18653/v1/2020.acl-main.325"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/susanto-2020-lexically-constrained-neural-machine-translation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/susanto-2020-lexically-constrained-neural-machine-translation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/) / Lexically Constrained Neural Machine Translation with Levenshtein Transformer

# Lexically Constrained Neural Machine Translation with Levenshtein Transformer

**Authors:** Raymond Hendy Susanto, [Shamil Chollampatt](https://md.research.elamur.ai/free/en/authors/shamil-chollampatt/) and [Liling Tan](https://md.research.elamur.ai/free/en/authors/liling-tan/) · **Year:** 2020 · **Venue:** [Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2020](https://md.research.elamur.ai/free/en/venues/acl/2020/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.acl-main.325/) · [PDF](https://aclanthology.org/2020.acl-main.325.pdf) · [DOI 10.18653/v1/2020.acl-main.325](https://doi.org/10.18653/v1/2020.acl-main.325) · [OpenAlex](https://openalex.org/W3035540807)

**Citations:** 56 citations (OpenAlex, 2026-10-14) · [54](https://md.research.elamur.ai/free/en/papers/susanto-2020-lexically-constrained-neural-machine-translation/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary

The paper addresses how to force a neural machine translation system to use required terminology, such as brand names or entries from domain dictionaries. Earlier solutions either retrain the model with constraint-aware data, which does not guarantee that the terms appear, or alter beam search, which makes decoding considerably slower. The authors build on the Levenshtein Transformer (LevT), a non-autoregressive model that refines its output through repeated deletion and insertion steps. Before decoding starts, they place the target constraint terms into the initial sequence. Two optional masks then stop the model from deleting these terms or inserting placeholders inside multi-token terms. Training is left unchanged, so custom dictionaries can be applied at inference time.

Experiments cover WMT'14 English-German, using constraints from Wiktionary, and the WMT'17 subsets with Wiktionary and IATE terms released by Dinu et al. The method achieves 100% term usage, slightly improves BLEU and keeps decoding speed essentially unchanged. This makes terminology control practical when translation must be fast.

## Key points

- **Implementation:** The method extends the FAIRSEQ LevT implementation using a base Transformer configuration (6 layers, 8 heads, d_model 512). The model is trained with knowledge distillation from a Transformer teacher.
- **Term usage on WMT'14 En-De:** Term% rises from 80.23 (baseline) to 94.43 with constraint insertion alone. It reaches 99.62 when deletion of constraints is blocked and 100.00 when insertion inside constraints is also blocked.
- **BLEU and speed on WMT'14 En-De:** On the 454-sentence subset that contains constraints, BLEU increases from 29.86 to 30.49, a statistically significant gain of about 0.6. On the full 3,003-sentence test set the gain is only about 0.1. Speed stays around 255–263 sentences per second.
- **Comparison with prior work on WMT'17 (Wiktionary / IATE):** LevT gains +0.96 and +1.16 BLEU respectively (to 31.20 and 30.13) with 100% Term%. Both prior systems, POST18 (constrained beam search) and DINU19 (data augmentation), obtain lower BLEU scores, and neither reaches 100% term usage on both sets.
- **Analysis:** Inserting the terms at random positions in the baseline output lowers BLEU from 29.9 to 29.3, which suggests LevT places the constraints in sensible positions. A remaining limitation is that constraints cannot be reordered, although 97–99% of En-De constraints already follow the source order.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

Also on the byline: [Rakuten (Japan)](https://ror.org/0098kke80).

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 958 works

## Cite

Cite as: Raymond Hendy Susanto, Shamil Chollampatt and Liling Tan. 2020. Lexically Constrained Neural Machine Translation with Levenshtein Transformer. Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{susanto2020lexically,
  title = {{Lexically Constrained Neural Machine Translation with Levenshtein Transformer}},
  author = {Raymond Hendy Susanto, Shamil Chollampatt and Liling Tan},
  booktitle = {Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics},
  year = {2020},
  doi = {10.18653/v1/2020.acl-main.325},
  url = {https://doi.org/10.18653/v1/2020.acl-main.325}
}
```

- [← previous XL-WiC: A Multilingual Benchmark for Evaluating Semantic Contextualization](https://md.research.elamur.ai/free/en/papers/raganato-2020-xl-wic-multilingual-benchmark-evaluating/)
- [next → On the Ability and Limitations of Transformers to Recognize Formal Languages](https://md.research.elamur.ai/free/en/papers/bhattamishra-2020-ability-limitations-transformers-recognize-formal/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/)

## References

17 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ondřej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, Radu Soricut, Lucia Specia, and Aleš Tamchyna. 2014. [Findings of the 2014 workshop on statistical machine translation](https://md.research.elamur.ai/free/en/papers/bojar-2014-findings-2014-workshop-statistical-machine/). In Proceedings of the…
2. Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Shujian Huang, Matthias Huck, Philipp Koehn, Qun Liu, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Raphael Rubino, Lucia Specia, and Marco Turchi. 2017. [Findings of the 2017 conference on machine transla…](https://md.research.elamur.ai/free/en/papers/bojar-2017-findings-2017-conference-machine-translation/)
3. Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Aurélie Névéol, Mariana Neves, Martin Popel, Matt Post, Raphael Rubino, Carolina Scarton, Lucia Specia, Marco Turc… [find](https://scholar.google.com/scholar?q=Ondr%CC%8Cej+Bojar%2C+Rajen+Chatterjee%2C+Christian+Federmann%2C+Yvette+Graham%2C+Barry+Haddow%2C+Matthias+Huck%2C+Antonio+Jimeno+Yepes%2C+Philipp+Koehn%2C+Varvara+Logacheva%2C+Christof+Monz%2C+Matteo+Negri%2C+Aure%CC%81lie+Ne%CC%81ve%CC%81ol)
4. Zampieri. 2016. [Findings of the 2016 conference on machine translation](https://md.research.elamur.ai/free/en/papers/bojar-2016-findings-2016-conference-machine-translation/). In Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers.
5. Josep Maria Crego, Jungi Kim, Guillaume Klein, Anabel Rebollo, Kathy Yang, Jean Senellart, Egor Akhanov, Patrice Brunelle, Aurelien Coquard, Yongchao Deng, Satoshi Enoue, Chiyo Geiss, Joshua Johanson, Ardas Khalsa, Raoum Khiari, Byeongil Ko, Catherine Kobus, Jean Lorieux, Leidiana Martins, Dang-Chua… — [arXiv:1610.05540](https://arxiv.org/abs/1610.05540) [find](https://scholar.google.com/scholar?q=Josep+Maria+Crego%2C+Jungi+Kim%2C+Guillaume+Klein%2C+Anabel+Rebollo%2C+Kathy+Yang%2C+Jean+Senellart%2C+Egor+Akhanov%2C+Patrice+Brunelle%2C+Aurelien+Coquard%2C+Yongchao+Deng%2C+Satoshi+Enoue%2C+Chiyo+Geiss%2C+Joshua+Johanson%2C)
6. Georgiana Dinu, Prashant Mathur, Marcello Federico, and Yaser Al-Onaizan. 2019. [Training neural machine translation to apply terminology constraints](https://md.research.elamur.ai/free/en/papers/dinu-2019-training-neural-machine-translation-apply/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.
7. Marjan Ghazvininejad, Omer Levy, Yinhan Liu, and Luke Zettlemoyer. 2019. [Mask-predict: Parallel decoding of conditional masked language models](https://md.research.elamur.ai/free/en/papers/ghazvininejad-2019-mask-predict-parallel-decoding-conditional/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Pro…
8. Jiatao Gu, James Bradbury, Caiming Xiong, Victor O. K. Li, and Richard Socher. 2018. [Non-autoregressive neural machine translation](https://md.research.elamur.ai/free/en/papers/gu-2017-non-autoregressive-neural-machine-translation/). In 6th International Conference on Learning Representations, Conference Track Proceedings.
9. Jiatao Gu, Changhan Wang, and Junbo Zhao. 2019. Levenshtein transformer. In Advances in Neural Information Processing Systems 32. [find](https://scholar.google.com/scholar?q=Jiatao+Gu%2C+Changhan+Wang%2C+and+Junbo+Zhao%2E+2019%2E+Levenshtein+transformer%2E+In+Advances+in+Neural+Information+Processing+Systems+32%2E)
10. Eva Hasler, Adrià de Gispert, Gonzalo Iglesias, and Bill Byrne. 2018. [Neural machine translation decoding with terminology constraints](https://md.research.elamur.ai/free/en/papers/hasler-2018-neural-machine-translation-decoding-terminology/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Short…
11. Chris Hokamp and Qun Liu. 2017. [Lexically constrained decoding for sequence generation using grid beam search](https://md.research.elamur.ai/free/en/papers/hokamp-2017-lexically-constrained-decoding-sequence-generation/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).
12. J. Edward Hu, Huda Khayrallah, Ryan Culkin, Patrick Xia, Tongfei Chen, Matt Post, and Benjamin Van Durme. 2019. [Improved lexically constrained decoding for translation and monolingual rewriting](https://md.research.elamur.ai/free/en/papers/hu-2019-improved-lexically-constrained-decoding-translation/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational…
13. Yoon Kim and Alexander M. Rush. 2016. [Sequence-level knowledge distillation](https://md.research.elamur.ai/free/en/papers/kim-2016-sequence-level-knowledge-distillation/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing.
14. Philipp Koehn. 2004. [Statistical significance tests for machine translation evaluation](https://md.research.elamur.ai/free/en/papers/koehn-2004-statistical-significance-tests-machine-translation/). In Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing.
15. Jason Lee, Elman Mansimov, and Kyunghyun Cho. 2018. [Deterministic non-autoregressive neural sequence modeling by iterative refinement](https://md.research.elamur.ai/free/en/papers/lee-2018-deterministic-non-autoregressive-neural-sequence/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.
16. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019. [fairseq: A fast, extensible toolkit for sequence modeling](https://md.research.elamur.ai/free/en/papers/ott-2019-fairseq-fast-extensible-toolkit-sequence/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonstr…
17. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [BLEU: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics.

*Updated 24 October 2026*