---
title: "CUNI Transformer Neural MT System for WMT18 — Popel 2018"
description: "This paper describes CUNI Transformer, Charles University's system for the English→Czech and Czech→English news translation task at WMT2018."
authors: ["Martin Popel"]
year: 2018
venue: "Proceedings of the Third Conference on Machine Translation: Shared Task Papers"
source: "https://aclanthology.org/W18-6424/"
doi: "10.18653/v1/w18-6424"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/popel-2018-cuni-transformer-neural-mt-system/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/popel-2018-cuni-transformer-neural-mt-system/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/) / CUNI Transformer Neural MT System for WMT18

# CUNI Transformer Neural MT System for WMT18

**Authors:** [Martin Popel](https://md.research.elamur.ai/free/en/authors/martin-popel/) · **Year:** 2018 · **Venue:** Proceedings of the Third Conference on Machine Translation: Shared Task Papers · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/W18-6424/) · [PDF](https://aclanthology.org/W18-6424.pdf) · [DOI 10.18653/v1/w18-6424](https://doi.org/10.18653/v1/w18-6424) · [OpenAlex](https://openalex.org/W2902767466)

**Citations:** 34 citations (openalex-snapshot · 2026-10-14) · [27](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
This paper describes CUNI Transformer, Charles University's system for the English→Czech and Czech→English news translation task at WMT2018. The author used only the constrained WMT data: about 58M authentic sentence pairs, mostly from CzEng 1.7, plus Czech and English NewsCrawl monolingual text of similar size. The model is a big Transformer in Tensor2Tensor, trained with the Adafactor optimizer and checkpoint averaging.

Five lightweight techniques improved it:
- cleaning the synthetic data,
- an iterated backtranslation scheme in which authentic and synthetic corpora are simply concatenated rather than shuffled,
- picking separate checkpoints for Czech-origin and foreign-origin documents,
- coreference-driven insertion of dropped Czech female pronouns,
- regex-based output fixes.

The system ranked first in both directions at WMT2018 and was significantly better than the runner-up, UEdin NMT, on both BLEU and human direct assessment. A small manual check suggested its output can match or exceed the human references. This makes single-reference automatic metrics questionable at this quality level. All five techniques are architecture-agnostic.

## Key points
- **Data filtering (~3% of synthetic sentences removed):**
  - Czech-side sentences without any accented character were dropped, which caught misplaced English text.
  - Sentences over 500 characters were removed.
  - Sentences with words repeated more than twice were filtered with a regex.
- **Iterated "concat" backtranslation:** Nematus2016 → English→Czech Transformer → Czech→English Transformer → final English→Czech model. Each training run took eight days on eight GTX 1080 Ti GPUs. Better synthetic data added +0.83 BLEU on newstest2017.
- **Origin-based checkpoint selection:** checkpoints were chosen on the Czech-origin and non-Czech-origin parts of newstest2013 (774k and 788k steps). This added +0.21 BLEU, reaching 26.81 cased BLEU on newstest2017, versus 22.80 for the WMT17 winner Nematus.
- **Coreference pre-processing:** Treex with its coreference resolver inserted "ona" (she) where a human antecedent was female. This touched about 1% of sentences and left BLEU unchanged, though the author judged most of those translations improved.
- **WMT2018 newstest2018 results:**

  | Direction | Cased BLEU | refDA | UEdin NMT (BLEU / refDA) |
  |---|---|---|---|
  | English→Czech | 26.01 | 67.2 | 23.42 / 60.6 |
  | Czech→English | 33.91 | 71.8 | 33.06 / 67.9 |

  The gap over UEdin NMT is significant at p \< 0.05 in both directions.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/en/shelves/machine-translation/)

Topics: [Coreference resolution](https://md.research.elamur.ai/free/en/topics/coreference-resolution/) · [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/)

## Cited by

Cited by

27 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 22 not yet.

- [Findings of the 2019 Conference on Machine Translation (WMT19)](https://md.research.elamur.ai/free/en/papers/barrault-2019-findings-2019-conference-machine-translation/)
- [Findings of the 2020 Conference on Machine Translation (WMT20)](https://md.research.elamur.ai/free/en/papers/barrault-2020-findings-2020-conference-machine-translation/)
- [Findings of the 2018 Conference on Machine Translation (WMT18)](https://md.research.elamur.ai/free/en/papers/bojar-2018-findings-2018-conference-machine-translation/)
- [APE at Scale and Its Implications on MT Evaluation Biases](https://md.research.elamur.ai/free/en/papers/freitag-2019-ape-scale-its-implications-mt/)
- [BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation](https://md.research.elamur.ai/free/en/papers/jiang-2022-blonde-automatic-evaluation-metric-document/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{popel2018cuni,
  title = {{CUNI Transformer Neural MT System for WMT18}},
  author = {Popel, Martin},
  booktitle = {Proceedings of the Third Conference on Machine Translation: Shared Task Papers},
  year = {2018},
  doi = {10.18653/v1/w18-6424},
  url = {https://doi.org/10.18653/v1/w18-6424}
}
```

- [← previous Crowdsourcing Question-Answer Meaning Representations](https://md.research.elamur.ai/free/en/papers/michael-2018-crowdsourcing-question-answer-meaning-representations/)
- [next → Data Augmentation via Dependency Tree Morphing for Low-Resource Languages](https://md.research.elamur.ai/free/en/papers/ahin-2018-data-augmentation-dependency-tree-morphing/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/)

## References

8 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Philipp Koehn. 2004. Statistical significance tests for machine translation evaluation. In Proceedings of EMNLP, volume 4, pages 388–395. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2E+2004%2E+Statistical+significance+tests+for+machine+translation+evaluation%2E+In+Proceedings+of+EMNLP%2C+volume+4%2C+pages+388%E2%80%93395%2E)
2. Shaohui Kuang, Deyi Xiong, Weihua Luo, and Guodong Zhou. 2017. Cache-based Document-level Neural Machine Translation. CoRR, arXiv/1711.11221. [find](https://scholar.google.com/scholar?q=Shaohui+Kuang%2C+Deyi+Xiong%2C+Weihua+Luo%2C+and+Guodong+Zhou%2E+2017%2E+Cache%2Dbased+Document%2Dlevel+Neural+Machine+Translation%2E+CoRR%2C+arXiv%2F1711%2E11221%2E)
3. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016b. Improving Neural Machine Translation Models with Monolingual Data. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 86–96, Berlin, Germany. Association for Computational… [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Barry+Haddow%2C+and+Alexandra+Birch%2E+2016b%2E+Improving+Neural+Machine+Translation+Models+with+Monolingual+Data%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computation)
4. Matouš Macháček and Ondřej Bojar. 2013. Results of the WMT13 Metrics Shared Task. In Proceedings of the Eighth Workshop on Statistical Machine Translation, pages 45–51, Sofia, Bulgaria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Matou%C5%A1+Mach%C3%A1c%CC%8Cek+and+Ondr%CC%8Cej+Bojar%2E+2013%2E+Results+of+the+WMT13+Metrics+Shared+Task%2E+In+Proceedings+of+the+Eighth+Workshop+on+Statistical+Machine+Translation%2C+pages+45%E2%80%9351%2C+Sofia%2C+Bulgaria%2E+Association+)
5. Noam Shazeer and Mitchell Stern. 2018. Adafactor: Adaptive Learning Rates with Sublinear Memory Cost. CoRR, arXiv/1804.04235. [find](https://scholar.google.com/scholar?q=Noam+Shazeer+and+Mitchell+Stern%2E+2018%2E+Adafactor%3A+Adaptive+Learning+Rates+with+Sublinear+Memory+Cost%2E+CoRR%2C+arXiv%2F1804%2E04235%2E)
6. Michal Novák. 2017. Coreference resolution system not only for czech. In Proceedings of the 17th conference ITAT 2017: Slovenskočeský NLP workshop (SloNLP 2017), pages 193–200, Praha, Czechia. CreateSpace Independent Publishing Platform. [find](https://scholar.google.com/scholar?q=Michal+Nov%C3%A1k%2E+2017%2E+Coreference+resolution+system+not+only+for+czech%2E+In+Proceedings+of+the+17th+conference+ITAT+2017%3A+Slovenskoc%CC%8Cesk%C3%BD+NLP+workshop+%28SloNLP+2017%29%2C+pages+193%E2%80%93200%2C+Praha%2C+Czechia%2E+Create)
7. Roman Sudarikov, Martin Popel, Ondřej Bojar, Aljoscha Burchardt, and Ondřej Klejch. 2016. Using MT-ComparEval. In Translation Evaluation: From Fragmented Tools and Data Sets to an Integrated Ecosystem, pages 76–82, Portorož, Slovenia. LREC. [find](https://scholar.google.com/scholar?q=Roman+Sudarikov%2C+Martin+Popel%2C+Ondr%CC%8Cej+Bojar%2C+Aljoscha+Burchardt%2C+and+Ondr%CC%8Cej+Klejch%2E+2016%2E+Using+MT%2DComparEval%2E+In+Translation+Evaluation%3A+From+Fragmented+Tools+and+Data+Sets+to+an+Integrated+Ecosyst)
8. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In Advances in Neural Information Processing Systems 30, pages 6000–6010. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+Advances+in+Neural+Information+Processi)