---
title: "Log-linear Combinations of — Junczys-Dowmunt 2016"
description: "The paper tackles automatic post-editing (APE): repairing the output of an unseen machine translation system (English→German, IT domain) in the WMT 2016…"
authors: ["Marcin Junczys-Dowmunt", "Roman Grundkiewicz"]
year: 2016
venue: "Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers"
source: "https://aclanthology.org/W16-2378/"
doi: "10.18653/v1/w16-2378"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/dowmunt-2016-log-linear-combinations-monolingual-bilingual/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/dowmunt-2016-log-linear-combinations-monolingual-bilingual/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2016](https://md.research.elamur.ai/free/papers/2016/page/2/) / Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editing

# Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editing

**Authors:** [Marcin Junczys-Dowmunt](https://md.research.elamur.ai/free/authors/marcin-junczys-dowmunt/), [Roman Grundkiewicz](https://md.research.elamur.ai/free/authors/roman-grundkiewicz/) · **Year:** 2016 · **Venue:** Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/W16-2378/) · [PDF](https://aclanthology.org/W16-2378.pdf) · [DOI 10.18653/v1/w16-2378](https://doi.org/10.18653/v1/w16-2378) · [OpenAlex](https://openalex.org/W2405897494)

**Citations:** 16 citations (openalex-snapshot · 2026-10-14) · [70](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
The paper tackles automatic post-editing (APE): repairing the output of an unseen machine translation system (English→German, IT domain) in the WMT 2016 shared task. The AMU team applied attention-based neural encoder-decoder models to this problem. The official post-editing data was far too small, so they created millions of synthetic triplets. They filtered German Common Crawl text by cross-entropy and passed it through German→English→German round-trip translation with phrase-based systems. They then kept the triplets whose TER statistics most closely resembled the real APE data. They trained two kinds of NMT models: monolingual ones (MT output → post-edit) and bilingual ones (English source → post-edit). Their ensembles were merged in a tuned log-linear model with a simple Post-Editing Penalty that discourages words absent from the input. This setup captures both the MT output and the source sentence and keeps corrections conservative. The final system ranked first in the shared task by a wide margin. It showed early on that neural models, given enough synthetic data, can beat phrase-based APE approaches.

## Key points
- Data: 12,000 official training triplets and 1,000 development triplets, plus about 10 million artificial triplets from round-trip translation. These were filtered by TER similarity into round-trip.n10 (4,335,715 triplets) and round-trip.n1 (531,839 triplets).
- Preprocessing used the Moses tokenizer and truecaser and BPE subword units with 40,000 units per language. German MT output and post-edits shared one vocabulary.
- Models: four mt→pe and four src→pe checkpoints were combined log-linearly in AmuNMT. The Post-Editing Penalty adds a −1 cost for each output word not seen in the input.
- On the development set, the full system improved over the raw MT baseline by −3.7% TER and +6.0% BLEU. It also gained −1.8% TER and +2.1% BLEU over the monolingual-only ensemble.
- On the test set, the system scored 21.52 TER and 67.65 BLEU, against 24.76 TER and 62.11 BLEU for uncorrected MT (−3.2% TER, +5.5% BLEU). The best other team, FBK, reached 23.92 TER.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/shelves/machine-translation/)

Topics: [Machine translation](https://md.research.elamur.ai/free/topics/machine-translation/)

## Cited by

Cited by

70 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 65 not yet.

- [Findings of the 2021 Conference on Machine Translation (WMT21)](https://md.research.elamur.ai/free/papers/akhbardeh-2021-findings-2021-conference-machine-translation/)
- [Findings of the 2016 Conference on Machine Translation](https://md.research.elamur.ai/free/papers/bojar-2016-findings-2016-conference-machine-translation/)
- [Findings of the 2017 Conference on Machine Translation (WMT17)](https://md.research.elamur.ai/free/papers/bojar-2017-findings-2017-conference-machine-translation/)
- [Findings of the WMT 2019 Shared Task on Automatic Post-Editing](https://md.research.elamur.ai/free/papers/chatterjee-2019-findings-wmt-2019-shared-task/)
- [APE at Scale and Its Implications on MT Evaluation Biases](https://md.research.elamur.ai/free/papers/freitag-2019-ape-scale-its-implications-mt/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{junczysdowmunt2016log,
  title = {{Log-linear Combinations of Monolingual and Bilingual Neural Machine Translation Models for Automatic Post-Editing}},
  author = {Junczys-Dowmunt, Marcin and Grundkiewicz, Roman},
  booktitle = {Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers},
  year = {2016},
  doi = {10.18653/v1/w16-2378},
  url = {https://doi.org/10.18653/v1/w16-2378}
}
```

- [← previous Linguistic Input Features Improve Neural Machine Translation](https://md.research.elamur.ai/free/papers/sennrich-2016-linguistic-input-features-improve-neural/)
- [next → Long Short-Term Memory-Networks for Machine Reading](https://md.research.elamur.ai/free/papers/cheng-2016-long-short-term-memory-networks/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/papers/2016/page/2/)

## References

24 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 644–648, Atlanta, Georgia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=2013+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%3A+Human+Language+Technologies%2C+pages+644%E2%80%93648%2C+Atlanta%2C+Georgia%2E+Association+for+Computational+Linguistics%2E)
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In Proceedings of the International Conference on Learning Representations, San Diego, CA. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2015%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+In+Proceedings+of+the+International+Conference+on+Learning+Representati) [our card](https://md.research.elamur.ai/free/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
3. Philip Gage. 1994. A new algorithm for data compression. The C Users Journal, (2):23–38. [find](https://scholar.google.com/scholar?q=Philip+Gage%2E+1994%2E+A+new+algorithm+for+data+compression%2E+The+C+Users+Journal%2C+%282%29%3A23%E2%80%9338%2E)
4. Hanna Béchara, Yanjun Ma, and Josef van Genabith. 2011. Statistical post-editing for a statistical MT system. In Proceedings of the 13th Machine Translation Summit, pages 308–315, Xiamen, China. [find](https://scholar.google.com/scholar?q=Hanna+Be%CC%81chara%2C+Yanjun+Ma%2C+and+Josef+van+Genabith%2E+2011%2E+Statistical+post%2Dediting+for+a+statistical+MT+system%2E+In+Proceedings+of+the+13th+Machine+Translation+Summit%2C+pages+308%E2%80%93315%2C+Xiamen%2C+China%2E)
5. Ulrich Germann. 2015. Sampling phrase tables for the Moses statistical machine translation system. Prague Bulletin of Mathematical Linguistics, (1):39–50. [find](https://scholar.google.com/scholar?q=Ulrich+Germann%2E+2015%2E+Sampling+phrase+tables+for+the+Moses+statistical+machine+translation+system%2E+Prague+Bulletin+of+Mathematical+Linguistics%2C+%281%29%3A39%E2%80%9350%2E)
6. Hanna Béchara, Raphaël Rubino, Yifan He, Yanjun Ma, and Josef van Genabith. 2012. An evaluation of statistical post-editing systems applied to RBMT and SMT systems. In Proceedings of COL-ING 2012, pages 215–230, Mumbai, India. [find](https://scholar.google.com/scholar?q=Hanna+Be%CC%81chara%2C+Raphae%CC%88l+Rubino%2C+Yifan+He%2C+Yanjun+Ma%2C+and+Josef+van+Genabith%2E+2012%2E+An+evaluation+of+statistical+post%2Dediting+systems+applied+to+RBMT+and+SMT+systems%2E+In+Proceedings+of+COL%2DING+2012%2C+p)
7. Kenneth Heafield, Ivan Pouzyrevsky, Jonathan H. Clark, and Philipp Koehn. 2013. Scalable modified Kneser-Ney language model estimation. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 690–696, Sofia, Bulgaria. [find](https://scholar.google.com/scholar?q=Kenneth+Heafield%2C+Ivan+Pouzyrevsky%2C+Jonathan+H%2E+Clark%2C+and+Philipp+Koehn%2E+2013%2E+Scalable+modified+Kneser%2DNey+language+model+estimation%2E+In+Proceedings+of+the+51st+Annual+Meeting+of+the+Association+for) [our card](https://md.research.elamur.ai/free/papers/heafield-2013-scalable-modified-kneser-ney-language/)
8. Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Barry Haddow, Matthias Huck, Chris Hokamp, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Carolina Scarton, Lucia Specia, and Marco Turchi. 2015. Findings of the 2015 Workshop on Statistical Machine Translation. In Proc… [find](https://scholar.google.com/scholar?q=Ondr%CC%8Cej+Bojar%2C+Rajen+Chatterjee%2C+Christian+Federmann%2C+Barry+Haddow%2C+Matthias+Huck%2C+Chris+Hokamp%2C+Philipp+Koehn%2C+Varvara+Logacheva%2C+Christof+Monz%2C+Matteo+Negri%2C+Matt+Post%2C+Carolina+Scarton%2C+Lucia+Speci) [our card](https://md.research.elamur.ai/free/papers/bojar-2015-findings-2015-workshop-statistical-machine/)
9. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, et al. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the Association for… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+et+al%2E+2007%2E+Moses%3A+Open+source+toolkit+fo)
10. Rajen Chatterjee, Marco Turchi, and Matteo Negri. 2015a. The FBK participation in the WMT15 automatic post-editing shared task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 210–215, Lisbon, Portugal. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Rajen+Chatterjee%2C+Marco+Turchi%2C+and+Matteo+Negri%2E+2015a%2E+The+FBK+participation+in+the+WMT15+automatic+post%2Dediting+shared+task%2E+In+Proceedings+of+the+Tenth+Workshop+on+Statistical+Machine+Translation%2C)
11. Robert C. Moore and William Lewis. 2010. Intelligent selection of language model training data. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 220– 224, Stroudsburg, PA, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Robert+C%2E+Moore+and+William+Lewis%2E+2010%2E+Intelligent+selection+of+language+model+training+data%2E+In+Proceedings+of+the+48th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+220%E2%80%93+2) [our card](https://md.research.elamur.ai/free/papers/moore-2010-intelligent-selection-language-model-training/)
12. Rajen Chatterjee, Marion Weller, Matteo Negri, and Marco Turchi. 2015b. Exploring the planet of the APEs: a comparative study of state-of-the-art methods for MT automatic post-editing. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th Internationa… [find](https://scholar.google.com/scholar?q=Rajen+Chatterjee%2C+Marion+Weller%2C+Matteo+Negri%2C+and+Marco+Turchi%2E+2015b%2E+Exploring+the+planet+of+the+APEs%3A+a+comparative+study+of+state%2Dof%2Dthe%2Dart+methods+for+MT+automatic+post%2Dediting%2E+In+Proceedings+)
13. Santanu Pal, Mihaela Vela, Sudip Kumar Naskar, and Josef van Genabith. 2015. USAAR-SAPE: An English–Spanish statistical automatic post-editing system. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 216–221, Lisbon, Portugal. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Santanu+Pal%2C+Mihaela+Vela%2C+Sudip+Kumar+Naskar%2C+and+Josef+van+Genabith%2E+2015%2E+USAAR%2DSAPE%3A+An+English%E2%80%93Spanish+statistical+automatic+post%2Dediting+system%2E+In+Proceedings+of+the+Tenth+Workshop+on+Statistic)
14. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. BLEU: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, pages 311–318, Stroudsburg, PA, USA. Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+BLEU%3A+A+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+Annual+Meeting+on+Association+for+Computat) [our card](https://md.research.elamur.ai/free/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
15. Colin Cherry and George Foster. 2012. Batch tuning strategies for statistical machine translation. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 428–436, Stroudsburg, PA, USA. Association for C… [find](https://scholar.google.com/scholar?q=Colin+Cherry+and+George+Foster%2E+2012%2E+Batch+tuning+strategies+for+statistical+machine+translation%2E+In+Proceedings+of+the+2012+Conference+of+the+North+American+Chapter+of+the+Association+for+Computatio)
16. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015a. Improving neural machine translation models with monolingual data. arXiv preprint arXiv:1511.06709. [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2015a%2E+Improving+neural+machine+translation+models+with+monolingual+data%2E+arXiv+preprint+arXiv%3A1511%2E06709%2E)
17. Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Langua… [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2C+Bart+van+Merrienboer%2C+Caglar+Gulcehre%2C+Dzmitry+Bahdanau%2C+Fethi+Bougares%2C+Holger+Schwenk%2C+and+Yoshua+Bengio%2E+2014%2E+Learning+phrase+representations+using+RNN+encoder%E2%80%93decoder+for+statistic)
18. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2015b. Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909. [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2015b%2E+Neural+machine+translation+of+rare+words+with+subword+units%2E+arXiv+preprint+arXiv%3A1508%2E07909%2E)
19. Michel Simard, Cyril Goutte, and Pierre Isabelle. 2007. Statistical phrase-based post-editing. In Proceedings of the Conference of the North American Chris Dyer, Victor Chahuneau, and Noah A. Smith. 2013. A simple, fast, and effective reparameterization of IBM model 2. In Proceedings of the Chapter… [find](https://scholar.google.com/scholar?q=Michel+Simard%2C+Cyril+Goutte%2C+and+Pierre+Isabelle%2E+2007%2E+Statistical+phrase%2Dbased+post%2Dediting%2E+In+Proceedings+of+the+Conference+of+the+North+American+Chris+Dyer%2C+Victor+Chahuneau%2C+and+Noah+A%2E+Smith%2E+2)
20. Matthew Snover, Bonnie Dorr, Richard Schwartz, Linnea Micciulla, and John Makhoul. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of Association for Machine Translation in the Americas, pages 223–231, Cambridge, Massachusetts. [find](https://scholar.google.com/scholar?q=Matthew+Snover%2C+Bonnie+Dorr%2C+Richard+Schwartz%2C+Linnea+Micciulla%2C+and+John+Makhoul%2E+2006%2E+A+study+of+translation+edit+rate+with+targeted+human+annotation%2E+In+Proceedings+of+Association+for+Machine+Tran) [our card](https://md.research.elamur.ai/free/papers/snover-2006-study-translation-edit-rate-targeted/)
21. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in Neural Information Processing Systems 27: 28th Annual Conference on Neural Information Processing Systems 2014, pages 3104– 3112, Montreal, Canada. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Advances+in+Neural+Information+Processing+Systems+27%3A+28th+Annual+Conference+on+Neural+Inform)
22. Guillaume Wisniewski, Nicolas Pécheux, and François Yvon. 2015. Why predicting post-edition is so hard? failure analysis of LIMSI submission to the APE shared task. In Proceedings of the Tenth Workshop on Statistical Machine Translation, pages 222– 227, Lisbon, Portugal. Association for Computatio… [find](https://scholar.google.com/scholar?q=Guillaume+Wisniewski%2C+Nicolas+Pe%CC%81cheux%2C+and+Franc%CC%A7ois+Yvon%2E+2015%2E+Why+predicting+post%2Dedition+is+so+hard%3F+failure+analysis+of+LIMSI+submission+to+the+APE+shared+task%2E+In+Proceedings+of+the+Tenth+Works)
23. Matthew D. Zeiler. 2012. ADADELTA: an adaptive learning rate method. arXiv preprint arXiv:1212.5701. [find](https://scholar.google.com/scholar?q=Matthew+D%2E+Zeiler%2E+2012%2E+ADADELTA%3A+an+adaptive+learning+rate+method%2E+arXiv+preprint+arXiv%3A1212%2E5701%2E) [our card](https://md.research.elamur.ai/free/papers/zeiler-2012-adadelta-adaptive-learning-rate-method/)
24. Barret Zoph and Kevin Knight. source neural translation. arXiv:1601.00710. 2016. Multi-arXiv preprint — [arXiv:1601.00710](https://arxiv.org/abs/1601.00710) [find](https://scholar.google.com/scholar?q=Barret+Zoph+and+Kevin+Knight%2E+source+neural+translation%2E+arXiv%3A1601%2E00710%2E+2016%2E+Multi%2DarXiv+preprint+%E2%80%94+%5BarXiv%3A1601%2E00710%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1601%2E00710%29)