---
title: "Guiding Neural Machine Translation with Retrieved Translation"
description: "Zhang et al. propose a lightweight test-time method that leaves the trained NMT model and its… 27 references, cited by 71 papers here."
authors: ["Jingyi Zhang", "Masao Utiyama", "Eiichro Sumita", "Graham Neubig", "Satoshi Nakamura"]
year: 2018
venue: "Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)"
source: "https://aclanthology.org/N18-1120/"
doi: "10.18653/v1/n18-1120"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2018-guiding-neural-machine-translation-retrieved/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2018-guiding-neural-machine-translation-retrieved/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/) / Guiding Neural Machine Translation with Retrieved Translation Pieces

# Guiding Neural Machine Translation with Retrieved Translation Pieces

**Authors:** Jingyi Zhang, [Masao Utiyama](https://md.research.elamur.ai/free/en/authors/masao-utiyama/), Eiichro Sumita, [Graham Neubig](https://md.research.elamur.ai/free/en/authors/graham-neubig/) and [Satoshi Nakamura](https://md.research.elamur.ai/free/en/authors/satoshi-nakamura/) · **Year:** 2018 · **Venue:** [Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)](https://md.research.elamur.ai/free/en/venues/naacl/) ([NAACL 2018](https://md.research.elamur.ai/free/en/venues/naacl/2018/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/N18-1120/) · [PDF](https://aclanthology.org/N18-1120.pdf) · [DOI 10.18653/v1/n18-1120](https://doi.org/10.18653/v1/n18-1120) · [OpenAlex](https://openalex.org/W2963829526)

**Citations:** 115 citations (OpenAlex, 2026-10-14) · [71](https://md.research.elamur.ai/free/en/papers/zhang-2018-guiding-neural-machine-translation-retrieved/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Neural machine translation often fails to recall and correctly render rare words and phrases, especially in narrow domains where similar sentences recur. Zhang et al. propose a lightweight test-time method that leaves the trained NMT model and its parameters unchanged. For each input sentence, the search engine Lucene retrieves similar source sentences from the word-aligned training corpus. Edit distance identifies which source words match the input. Target-side n-grams (up to 4-grams) aligned only to those matching words become "translation pieces." Each piece is weighted by the highest similarity among the retrieved sentences that produced it. During beam search, the decoder adds a tunable bonus (λ) to output words that complete such pieces.

On three narrow-domain tasks (en-de, en-fr, en-es), this improves BLEU by up to 6 points with only a small increase in decoding time. It matches or beats the more complex SGNMT approach. Its value lies in giving NMT memory-like recall of domain phrasing without retraining or architectural changes.

## Key points
- **Retrieval and weighting:** Retrieval uses Lucene. Sentence similarity is computed as 1 − edit distance / max sentence length, following Gu et al. (2017). About 10 retrieved sentences per input offers a good speed–quality balance.
- **Effect on rare n-grams:** Gains are largest for n-grams that appear in the training data but occur fewer than 5 times. N-grams absent from training see little benefit, and the improvement shrinks as frequency increases.
- **Timing (en-de, seconds per sentence):** Baseline decoding takes 0.215 s and guided decoding 0.306 s. Retrieval is negligible at about 0.016 s. Translation-piece collection takes about 0.52 s in an unoptimized Python implementation.
- **Comparison with SGNMT on its datasets (test BLEU):**
  - en-de: 50.36 vs. 48.80
  - en-fr: 65.69 vs. 64.60
  - en-es: 57.11 vs. 57.27 (slightly lower)
- **Training cost:** Training is unaffected. By contrast, SGNMT changes the model structure and can roughly double translation time.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 365 works · 158,048 citations
- [Nara Institute of Science and Technology](https://md.research.elamur.ai/free/en/universities/nara-institute-of-science-and-technology/) — University, Ikoma, Japan · 34 works · 4,419 citations
- [National Institute of Information and Communications Technology](https://md.research.elamur.ai/free/en/labs/national-institute-of-information-and-communications-technology/) — Research institute, Tokyo, Japan · 13 works · 1,970 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 958 works

## Cite

Cite as: Jingyi Zhang, Masao Utiyama, Eiichro Sumita, Graham Neubig and Satoshi Nakamura. 2018. Guiding Neural Machine Translation with Retrieved Translation Pieces. Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2018guiding,
  title = {{Guiding Neural Machine Translation with Retrieved Translation Pieces}},
  author = {Jingyi Zhang, Masao Utiyama, Eiichro Sumita, Graham Neubig and Satoshi Nakamura},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-1120},
  url = {https://doi.org/10.18653/v1/n18-1120}
}
```

- [← previous An End-to-end Approach for Handling Unknown Slot Values in Dialogue State Tracking](https://md.research.elamur.ai/free/en/papers/xu-2018-end-end-approach-handling-unknown/)
- [next → Integrating Stance Detection and Fact Checking in a Unified Corpus](https://md.research.elamur.ai/free/en/papers/baly-2018-integrating-stance-detection-fact-checking/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/)

## References

27 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Philip Arthur, Graham Neubig, and Satoshi Nakamura. 2016. [Incorporating discrete translation lexicons into neural machine translation](https://md.research.elamur.ai/free/en/papers/arthur-2016-incorporating-discrete-translation-lexicons-into/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. pages 1557– 1567. [https://aclweb.org/anthology/](https://aclweb.org/anthology/) D16-1162. [↗](https://aclweb.org/anthology/)
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. [Neural machine translation by jointly learning to align and translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/). arXiv preprint arXiv:1409.0473 [https://arxiv.org/pdf/](https://arxiv.org/pdf/) 1409.0473.pdf. [↗](https://arxiv.org/pdf/)
3. Rajen Chatterjee, Matteo Negri, Marco Turchi, Marcello Federico, Lucia Specia, and Frédéric Blain. 2017. Guiding neural machine translation decoding with external knowledge. In Proceedings of the Second Conference on Machine Translation. pages 157–168. [http://www.aclweb.org/](http://www.aclweb.org/) anthology/W17-4716. [↗](http://www.aclweb.org/)
4. Leonard Dahlmann, Evgeny Matusov, Pavel Petrushkov, and Shahram Khadivi. 2017. Neural machine translation leveraging phrase-based models in a hybrid search. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. pages 1411–1420. [https://www.aclweb.org/anthology/](https://www.aclweb.org/anthology/) D… [↗](https://www.aclweb.org/anthology/)
5. M. Amin Farajian, Marco Turchi, Matteo Negri, and Marcello Federico. 2017. [Multi-domain neural machine translation through unsupervised adaptation](https://md.research.elamur.ai/free/en/papers/farajian-2017-multi-domain-neural-machine-translation/). In Proceedings of the Second Conference on Machine Translation. pages 127–137. [http://www.aclweb.org/anthology/W/](http://www.aclweb.org/anthology/W/) W17/W17-4713.pdf. [↗](http://www.aclweb.org/anthology/W/)
6. Ulrich Germann. 2015. Sampling phrase tables for the moses statistical machine translation system. The Prague Bulletin of Mathematical Linguistics 104(1):39–50. [https://ufal.mff.cuni.cz/pbml/104/art-germann.pdf](https://ufal.mff.cuni.cz/pbml/104/art-germann.pdf). [↗](https://ufal.mff.cuni.cz/pbml/104/art-germann.pdf)
7. Gregory Grefenstette. 1999. The world wide web as a resource for example-based machine translation tasks. In Proceedings of the ASLIB Conference on Translating and the Computer. volume 21. [http://www.mt-archive.info/](http://www.mt-archive.info/) Aslib-1999-Grefenstette.pdf. [↗](http://www.mt-archive.info/)
8. Jiatao Gu, Yong Wang, Kyunghyun Cho, and Victor OK Li. 2017. Search engine guided non-parametric neural machine translation. arXiv preprint arXiv:1705.07267 [https://arxiv.org/pdf/1705.07267.pdf](https://arxiv.org/pdf/1705.07267.pdf). — [arXiv:1705.07267](https://arxiv.org/abs/1705.07267) [↗](https://arxiv.org/pdf/1705.07267.pdf)
9. Wei He, Zhongjun He, Hua Wu, and Haifeng Wang. 2016. Improved neural machine translation with smt features. In AAAI. pages 151–157. [https://www.aaai.org/ocs/index.php/AAAI/](https://www.aaai.org/ocs/index.php/AAAI/) AAAI16/paper/view/12189/11577. [↗](https://www.aaai.org/ocs/index.php/AAAI/)
10. Łukasz Kaiser, Ofir Nachum, Aurko Roy, and Samy Bengio. 2017. Learning to remember rare events. arXiv preprint arXiv:1703.03129 [https://arxiv.org/pdf/1703.03129.pdf](https://arxiv.org/pdf/1703.03129.pdf). [↗](https://arxiv.org/pdf/1703.03129.pdf)
11. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proc… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+Chris+Dyer%2C+Ondrej+Bojar%2C+Alexandra+Consta)
12. Philipp Koehn and Rebecca Knowles. 2017. Six challenges for neural machine translation. In Proceedings of the First Workshop on Neural Machine Translation. pages 28–39. [http://www.aclweb.org/anthology/W17-3204](http://www.aclweb.org/anthology/W17-3204). [↗](http://www.aclweb.org/anthology/W17-3204)
13. Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of the 2003 Conference of the North American Chapter of the Association for Computational Linguistics on Human Language Technology-Volume 1. pages 48–54. [http://www.aclweb](http://www.aclweb). org/anthology/N/N0… [↗](http://www.aclweb)
14. Xiaoqing Li, Jiajun Zhang, and Chengqing Zong. 2016. One sentence one model for neural machine translation. arXiv preprint arXiv:1609.06490 [https://arxiv.org/pdf/1609.06490.pdf](https://arxiv.org/pdf/1609.06490.pdf). [↗](https://arxiv.org/pdf/1609.06490.pdf)
15. Adam Lopez. 2007. Hierarchical phrase-based translation with suffix arrays. In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EMNLP-CoNLL). pages 976–985. [http://www.aclweb.org/](http://www.aclweb.org/) anthology/D/D07/D07-1104. [↗](http://www.aclweb.org/)
16. Makoto Nagao. 1984. A framework of a mechanical translation between japanese and english by analogy principle. Artificial and human intelligence pages 351–354. [http://www.mt-archive.info/](http://www.mt-archive.info/) Nagao-1984.pdf. [↗](http://www.mt-archive.info/)
17. Toan Q Nguyen and David Chiang. 2017. Improving lexical choice in neural machine translation. arXiv preprint arXiv:1710.01329 [https://arxiv.org/pdf/1710.01329.pdf](https://arxiv.org/pdf/1710.01329.pdf). [↗](https://arxiv.org/pdf/1710.01329.pdf)
18. Franz Josef Och and Hermann Ney. 2003. A systematic comparison of various statistical alignment models. Computational linguistics 29(1):19– 51. [http://www.aclweb.org/anthology/](http://www.aclweb.org/anthology/) J/J03/J03-1002.pdf. [↗](http://www.aclweb.org/anthology/)
19. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016a. [Edinburgh neural machine translation systems for WMT 16](https://md.research.elamur.ai/free/en/papers/sennrich-2016-edinburgh-neural-machine-translation-systems/). In Proceedings of the First Conference on Machine Translation. pages 371–376. [http://www.aclweb.org/](http://www.aclweb.org/) anthology/W/W16/W16-2323.pdf. [↗](http://www.aclweb.org/)
20. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016b. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pages 1715–1725. [http://www.aclweb.org/](http://www.aclweb.org/) anthology/P/P16/P16-1162.p… [↗](http://www.aclweb.org/)
21. Felix Stahlberg, Adrià de Gispert, Eva Hasler, and Bill Byrne. 2017. Neural machine translation by minimising the Bayes-risk with respect to syntactic translation lattices. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Shor… [find](https://scholar.google.com/scholar?q=Felix+Stahlberg%2C+Adria%CC%80+de+Gispert%2C+Eva+Hasler%2C+and+Bill+Byrne%2E+2017%2E+Neural+machine+translation+by+minimising+the+Bayes%2Drisk+with+respect+to+syntactic+translation+lattices%2E+In+Proceedings+of+the+15th)
22. Yaohua Tang, Fandong Meng, Zhengdong Lu, Hang Li, and Philip LH Yu. 2016. Neural machine translation with external phrase memory. arXiv preprint arXiv:1606.01792 [https://arxiv.org/pdf/](https://arxiv.org/pdf/) 1606.01792.pdf. — [arXiv:1606.01792](https://arxiv.org/abs/1606.01792) [↗](https://arxiv.org/pdf/)
23. Masao Utiyama, Graham Neubig, Takashi Onishi, and Eiichiro Sumita. 2011. Searching translation memories for paraphrases. In Machine Translation Summit. volume 13, pages 325–331. [http://mt-archive.info/](http://mt-archive.info/) MTS-2011-Utiyama.pdf. [↗](http://mt-archive.info/)
24. Rui Wang, Andrew Finch, Masao Utiyama, and Eiichiro Sumita. 2017a. Sentence embedding for neural machine translation domain adaptation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). pages 560–566. [http://aclweb.org/anthology/P17…](http://aclweb.org/anthology/P17…) [↗](http://aclweb.org/anthology/P17…)
25. Yuguang Wang, Shanbo Cheng, Liyang Jiang, Jiajun Yang, Wei Chen, Muze Li, Lin Shi, Yanfeng Wang, and Hongtao Yang. 2017b. Sogou neural machine translation systems for wmt17. In Proceedings of the Second Conference on Machine Translation. pages 410–415. [http://www.aclweb.org/anthology/W17-4742](http://www.aclweb.org/anthology/W17-4742). [↗](http://www.aclweb.org/anthology/W17-4742)
26. Jiacheng Zhang, Yang Liu, Huanbo Luan, Jingfang Xu, and Maosong Sun. 2017. Prior knowledge integration for neural machine translation using posterior regularization. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). pages 1514–1523. h… [find](https://scholar.google.com/scholar?q=Jiacheng+Zhang%2C+Yang+Liu%2C+Huanbo+Luan%2C+Jingfang+Xu%2C+and+Maosong+Sun%2E+2017%2E+Prior+knowledge+integration+for+neural+machine+translation+using+posterior+regularization%2E+In+Proceedings+of+the+55th+Annual+)
27. Jian Zhang, Liangyou Li, Andy Way, and Qun Liu. 2016. Topic-informed neural machine translation. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers. pages 1807–1817. [http://aclweb.org/anthology/C16-1170](http://aclweb.org/anthology/C16-1170). [↗](http://aclweb.org/anthology/C16-1170)

*Updated 24 October 2026*