---
title: "Achieving Human Parity on Automatic Chinese to English News Translation — Hassan 2018"
description: "They then describe Microsoft's Chinese-to-English NMT system, which adds several techniques to attentional… 52 references, cited by 273 papers here."
authors: ["Hany Hassan", "Anthony Aue", "Chang Chen", "Vishal Chowdhary", "Jonathan Clark", "Christian Federmann", "Xuedong Huang", "Marcin Junczys-Dowmunt", "William Lewis", "Mu Li", "Shujie Liu", "Tie-Yan Liu", "Renqian Luo", "Arul Menezes", "Tao Qin", "Frank Seide", "Xu Tan", "Fei Tian", "Lijun Wu", "Shuangzhi Wu", "Yingce Xia", "Dongdong Zhang", "Zhirui Zhang", "Ming Zhou"]
year: 2018
venue: "arXiv:1803.05567"
source: "https://arxiv.org/abs/1803.05567"
doi: "10.48550/arxiv.1803.05567"
arxiv: "1803.05567"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hassan-2018-achieving-human-parity-automatic-chinese/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hassan-2018-achieving-human-parity-automatic-chinese/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/) / Achieving Human Parity on Automatic Chinese to English News Translation

# Achieving Human Parity on Automatic Chinese to English News Translation

**Authors:** [Hany Hassan](https://md.research.elamur.ai/free/en/authors/hany-hassan/), Anthony Aue, Chang Chen, Vishal Chowdhary, [Jonathan Clark](https://md.research.elamur.ai/free/en/authors/jonathan-h-clark/), [Christian Federmann](https://md.research.elamur.ai/free/en/authors/christian-federmann/), Xuedong Huang, [Marcin Junczys-Dowmunt](https://md.research.elamur.ai/free/en/authors/marcin-junczys-dowmunt/), [William Lewis](https://md.research.elamur.ai/free/en/authors/william-lewis/), [Mu Li](https://md.research.elamur.ai/free/en/authors/mu-li/), [Shujie Liu](https://md.research.elamur.ai/free/en/authors/shujie-liu/), [Tie-Yan Liu](https://md.research.elamur.ai/free/en/authors/tie-yan-liu/), Renqian Luo, [Arul Menezes](https://md.research.elamur.ai/free/en/authors/arul-menezes/), [Tao Qin](https://md.research.elamur.ai/free/en/authors/tao-qin/), Frank Seide, [Xu Tan](https://md.research.elamur.ai/free/en/authors/xu-tan/), [Fei Tian](https://md.research.elamur.ai/free/en/authors/fei-tian/), Lijun Wu, Shuangzhi Wu, Yingce Xia, [Dongdong Zhang](https://md.research.elamur.ai/free/en/authors/dongdong-zhang/), [Zhirui Zhang](https://md.research.elamur.ai/free/en/authors/zhirui-zhang/) and [Ming Zhou](https://md.research.elamur.ai/free/en/authors/ming-zhou/) · **Year:** 2018 · **Venue:** arXiv:1803.05567 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1803.05567) · [PDF](https://arxiv.org/pdf/1803.05567) · [DOI 10.48550/arxiv.1803.05567](https://doi.org/10.48550/arxiv.1803.05567) · [arXiv 1803.05567](https://arxiv.org/abs/1803.05567) · [OpenAlex](https://openalex.org/W2794365787)

**Citations:** 521 citations (OpenAlex, 2026-10-14) · [273](https://md.research.elamur.ai/free/en/papers/hassan-2018-achieving-human-parity-automatic-chinese/cited-by/) in our library, of the 140,884 papers whose bibliographies we read

## Summary
The paper asks whether machine translation can match professional human translators, and first makes that question testable. A system reaches human parity when bilingual judges score its output and the human translations of the same test set, and the difference between those scores is not statistically significant. Reference-based metrics such as BLEU penalize valid alternative wordings, and references can themselves be poor because they are often crowd-sourced or MT-assisted. For these reasons, the authors use source-based direct assessment with bilingual annotators. They then describe Microsoft's Chinese-to-English NMT system, which adds several techniques to attentional encoder-decoder models:
- dual learning that exploits monolingual data;
- joint training of forward and backward models;
- Deliberation networks for two-pass decoding;
- agreement regularization between left-to-right and right-to-left decoders;
- data filtering with a bilingual sentence encoder;
- combination of heterogeneous systems.

On the WMT 2017 news task (newstest2017, zh→en), the best combined systems were statistically indistinguishable from professional translations and outperformed crowd-sourced references. The work matters as a milestone claim and as a reproducible evaluation protocol, since all judgments and new references were released.

## Key points
- Evaluation: annotators compare the source with a candidate on a 0–100 slider. Scores are converted to z-scores, and systems are clustered by significance using a rank test at p ≤ 0.05. Quality control relies on deliberately degraded "bad references", and the setup is double-blind via the Appraise tool.
- Compared outputs include the research systems Combo-4, Combo-5 and Combo-6, plus Online-A-1710 and Sogou. These are judged against three references: Reference-WMT (original), Reference-HT (translated from scratch) and Reference-PE (post-edited).
- BLEU scores varied substantially depending on which reference was used. The authors take this as evidence that automatic metrics become unreliable near human quality.
- An error analysis of 500 sampled Combo-6 sentences found the following shares of sentences with each error type:
  - Incorrect Words: 7.64%
  - Ungrammatical: 6.33%
  - Missing Words: 5.46%
  - Named Entity: 4.38%
- The release (http://aka.ms/Translator-HumanParityData) contains the new references, the Combo outputs, the Online-A-1710 outputs and all anonymized human judgments. The authors do not claim the result generalizes to other language pairs or domains.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Machine translation](https://md.research.elamur.ai/free/en/shelves/machine-translation/)

Topics: [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/)

## Cite

Cite as: Hany Hassan, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, Xuedong Huang, Marcin Junczys-Dowmunt, William Lewis, Mu Li, Shujie Liu, Tie-Yan Liu, Renqian Luo, Arul Menezes, Tao Qin, Frank Seide, Xu Tan, Fei Tian, Lijun Wu, Shuangzhi Wu, Yingce Xia, Dongdong Zhang, Zhirui Zhang and Ming Zhou. 2018. Achieving Human Parity on Automatic Chinese to English News Translation. arXiv:1803.05567.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{hassan2018achieving,
  title = {{Achieving Human Parity on Automatic Chinese to English News Translation}},
  author = {Hany Hassan, Anthony Aue, Chang Chen, Vishal Chowdhary, Jonathan Clark, Christian Federmann, Xuedong Huang, Marcin Junczys-Dowmunt, William Lewis, Mu Li, Shujie Liu, Tie-Yan Liu, Renqian Luo, Arul Menezes, Tao Qin, Frank Seide, Xu Tan, Fei Tian, Lijun Wu, Shuangzhi Wu, Yingce Xia, Dongdong Zhang, Zhirui Zhang and Ming Zhou},
  year = {2018},
  doi = {10.48550/arxiv.1803.05567},
  eprint = {1803.05567},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1803.05567}
}
```

- [← previous Obtaining Reliable Human Ratings of Valence, Arousal, and Dominance for 20,000 English Words](https://md.research.elamur.ai/free/en/papers/mohammad-2018-obtaining-reliable-human-ratings-valence/)
- [next → Newsroom: A Dataset of 1.3 Million Summaries with Diverse Extractive Strategies](https://md.research.elamur.ai/free/en/papers/grusky-2018-newsroom-dataset-1-3-million/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/)

## References

52 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Artetxe, M., Labaka, G., Agirre, E., and Cho, K. Unsupervised neural machine translation. In International Conference on Learning Representations (2018). [our card](https://md.research.elamur.ai/free/en/papers/artetxe-2017-unsupervised-neural-machine-translation/)
2. Axelrod, A., He, X., and Gao, J. Domain adaptation via pseudo in-domain data selection. In Proceedings of the Conference on Empirical Methods in Natural Language Processing (Stroudsburg, PA, USA, 2011), EMNLP ’11, Association for Computational Linguistics, pp. 355–362. [our card](https://md.research.elamur.ai/free/en/papers/axelrod-2011-domain-adaptation-pseudo-domain-data/)
3. Bahdanau, D., Cho, K., and Bengio, Y. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014). [↗](https://arxiv.org/abs/1409.0473) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
4. Belinkov, Y., and Bisk, Y. Synthetic and natural noise both break neural machine translation. CoRR abs/1711.02173 (2017). [↗](https://arxiv.org/abs/1711.02173)
5. Bengio, S., Vinyals, O., Jaitly, N., and Shazeer, N. Scheduled sampling for sequence prediction with recurrent neural networks. In NIPS (2015), pp. 1171–1179. [find](https://scholar.google.com/scholar?q=Bengio%2C+S%2E%2C+Vinyals%2C+O%2E%2C+Jaitly%2C+N%2E%2C+and+Shazeer%2C+N%2E+Scheduled+sampling+for+sequence+prediction+with+recurrent+neural+networks%2E+In+NIPS+%282015%29%2C+pp%2E+1171%E2%80%931179%2E)
6. Bojar, O., Chatterjee, R., Federmann, C., Graham, Y., Haddow, B., Huang, S., Huck, M., Koehn, P., Liu, Q., Logacheva, V., Monz, C., Negri, M., Post, M., Rubino, R., Specia, L., and Turchi, M. Findings of the 2017 conference on machine translation (wmt17). In Proceedings of the Second Conference on Machine Translation, Volume 2: Shared Task Papers (Copenhagen, Denmark, September 2017), Association for Computational Linguistics, pp. 169–214. [our card](https://md.research.elamur.ai/free/en/papers/bojar-2017-findings-2017-conference-machine-translation/)
7. Cettolo, M., Federico, M., Bentivogli, L., Niehues, J., Stüker, S., Sudoh, K., Yoshino, K., and Federmann, C. Overview of the iwslt 2017 evaluation campaign. In Proceedings of the 14th International Workshop on Spoken Language Translation (IWSLT) (Tokyo, Japan, December 2017), IWSLT, pp. 2–12. [our card](https://md.research.elamur.ai/free/en/papers/cettolo-2017-overview-iwslt-2017-evaluation-campaign/)
8. Cherry, C., and Foster, G. Batch tuning strategies for statistical machine translation. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Stroudsburg, PA, USA, 2012), NAACL HLT ’12, Association for Computational Linguistics, pp. 427–436. [our card](https://md.research.elamur.ai/free/en/papers/cherry-2012-batch-tuning-strategies-statistical-machine/)
9. Clark, J. H., Dyer, C., Lavie, A., and Smith, N. A. Better hypothesis testing for statistical machine translation: Controlling for optimizer instability. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: short papers-Volume 2 (2011), Association for Computational Linguistics, pp. 176–181. [find](https://scholar.google.com/scholar?q=Clark%2C+J%2E+H%2E%2C+Dyer%2C+C%2E%2C+Lavie%2C+A%2E%2C+and+Smith%2C+N%2E+A%2E+Better+hypothesis+testing+for+statistical+machine+translation%3A+Controlling+for+optimizer+instability%2E+In+Proceedings+of+the+49th+Annual+Meeting+of+t)
10. Denkowski, M., and Lavie, A. Meteor 1.3: Automatic Metric for Reliable Optimization and Evaluation of Machine Translation Systems. In Proceedings of the EMNLP 2011 Workshop on Statistical Machine Translation (2011). [our card](https://md.research.elamur.ai/free/en/papers/denkowski-2011-meteor-1-3-automatic-metric/)
11. Devlin, J. Sharp models on dull hardware: Fast and accurate neural machine translation decoding on the cpu. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing (Copenhagen, Denmark, September 2017), Association for Computational Linguistics, pp. 2810–2815. [find](https://scholar.google.com/scholar?q=Devlin%2C+J%2E+Sharp+models+on+dull+hardware%3A+Fast+and+accurate+neural+machine+translation+decoding+on+the+cpu%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28C)
12. Dreyer, M., and Marcu, D. HyTER: Meaning-Equivalent Semantics for Translation Evaluation. In Proceedings of the 2012 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (2012), Association for Computational Linguistics, pp. 162–171. [find](https://scholar.google.com/scholar?q=Dreyer%2C+M%2E%2C+and+Marcu%2C+D%2E+HyTER%3A+Meaning%2DEquivalent+Semantics+for+Translation+Evaluation%2E+In+Proceedings+of+the+2012+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Lingu)
13. Edgington, E. S. Validity of Randomization Tests for One-subject Experiments. Journal of Educational Statistics 5, 3 (1980), 235–251. [find](https://scholar.google.com/scholar?q=Edgington%2C+E%2E+S%2E+Validity+of+Randomization+Tests+for+One%2Dsubject+Experiments%2E+Journal+of+Educational+Statistics+5%2C+3+%281980%29%2C+235%E2%80%93251%2E)
14. Federmann, C. Appraise: An open-source toolkit for manual evaluation of machine translation output. The Prague Bulletin of Mathematical Linguistics 98 (September 2012), 25–35. [find](https://scholar.google.com/scholar?q=Federmann%2C+C%2E+Appraise%3A+An+open%2Dsource+toolkit+for+manual+evaluation+of+machine+translation+output%2E+The+Prague+Bulletin+of+Mathematical+Linguistics+98+%28September+2012%29%2C+25%E2%80%9335%2E)
15. Feng, S., Liu, S., Yang, N., Li, M., Zhou, M., and Zhu, K. Q. Improving attention modeling with implicit distortion and fertility for machine translation. In COLING 2016, 26th International Conference on Computational Linguistics, Proceedings of the Conference: Technical Papers, December 11-16, 2016, Osaka, Japan (2016), pp. 3082–3092. [find](https://scholar.google.com/scholar?q=Feng%2C+S%2E%2C+Liu%2C+S%2E%2C+Yang%2C+N%2E%2C+Li%2C+M%2E%2C+Zhou%2C+M%2E%2C+and+Zhu%2C+K%2E+Q%2E+Improving+attention+modeling+with+implicit+distortion+and+fertility+for+machine+translation%2E+In+COLING+2016%2C+26th+International+Conference)
16. Gehring, J., Auli, M., Grangier, D., Yarats, D., and Dauphin, Y. N. Convolutional sequence to sequence learning. arXiv preprint arXiv:1705.03122 (2017). [↗](https://arxiv.org/abs/1705.03122) [our card](https://md.research.elamur.ai/free/en/papers/gehring-2017-convolutional-sequence-sequence-learning/)
17. Graham, Y., Baldwin, T., Moffat, A., and Zobel, J. Can machine translation systems be evaluated by the crowd alone? Natural Language Engineering, 23(1), 3-30. doi:10.1017/S1351324915000339 (2016). [↗](https://doi.org/10.1017/s1351324915000339)
18. He, D., Xia, Y., Qin, T., Wang, L., Yu, N., Liu, T., and Ma, W.-Y. Dual learning for machine translation. In Advances in Neural Information Processing Systems (2016), pp. 820– 828. [find](https://scholar.google.com/scholar?q=He%2C+D%2E%2C+Xia%2C+Y%2E%2C+Qin%2C+T%2E%2C+Wang%2C+L%2E%2C+Yu%2C+N%2E%2C+Liu%2C+T%2E%2C+and+Ma%2C+W%2E%2DY%2E+Dual+learning+for+machine+translation%2E+In+Advances+in+Neural+Information+Processing+Systems+%282016%29%2C+pp%2E+820%E2%80%93+828%2E)
19. He, K., Zhang, X., Ren, S., and Sun, J. Deep residual learning for image recognition. arXiv preprint arXiv:1512.03385 (2015). [↗](https://arxiv.org/abs/1512.03385)
20. Johnson, M., Schuster, M., Le, Q. V., Krikun, M., Wu, Y., Chen, Z., Thorat, N., Viégas, F., Wattenberg, M., Corrado, G., et al. Google’s multilingual neural machine translation system: enabling zero-shot translation. arXiv preprint arXiv:1611.04558 (2016). [↗](https://arxiv.org/abs/1611.04558)
21. Kingma, D. P., and Ba, J. abs/1412.6980 (2014). Adam: A method for stochastic optimization. [↗](https://arxiv.org/abs/1412.6980)
22. Lample, G., Conneau, A., Denoyer, L., and Ranzato, M. Unsupervised machine translation using monolingual corpora only. In International Conference on Learning Representations (2018). [our card](https://md.research.elamur.ai/free/en/papers/lample-2017-unsupervised-machine-translation-using-monolingual/)
23. Lin, J., Xia, Y., Qin, T., Chen, Z., and Liu, T.-Y. Conditional image-to-image translation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (July 2018). [find](https://scholar.google.com/scholar?q=Lin%2C+J%2E%2C+Xia%2C+Y%2E%2C+Qin%2C+T%2E%2C+Chen%2C+Z%2E%2C+and+Liu%2C+T%2E%2DY%2E+Conditional+image%2Dto%2Dimage+translation%2E+In+The+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVPR%29+%28July+2018%29%2E)
24. Luo, P., Wang, G., Lin, L., and Wang, X. Deep dual learning for semantic image segmentation. In IEEE International Conference on Computer Vision, ICCV 2017, Venice, Italy, October 22-29, 2017 (2017), pp. 2737–2745. [find](https://scholar.google.com/scholar?q=Luo%2C+P%2E%2C+Wang%2C+G%2E%2C+Lin%2C+L%2E%2C+and+Wang%2C+X%2E+Deep+dual+learning+for+semantic+image+segmentation%2E+In+IEEE+International+Conference+on+Computer+Vision%2C+ICCV+2017%2C+Venice%2C+Italy%2C+October+22%2D29%2C+2017+%282017%29%2C+)
25. Mann, H. B., and Whitney, D. R. On a test of whether one of two random variables is stochastically larger than the other. The annals of mathematical statistics (1947), 50–60. [find](https://scholar.google.com/scholar?q=Mann%2C+H%2E+B%2E%2C+and+Whitney%2C+D%2E+R%2E+On+a+test+of+whether+one+of+two+random+variables+is+stochastically+larger+than+the+other%2E+The+annals+of+mathematical+statistics+%281947%29%2C+50%E2%80%9360%2E)
26. Moore, R. C., and Lewis, W. Intelligent selection of language model training data. In Proceedings of the ACL 2010 Conference Short Papers (Stroudsburg, PA, USA, 2010), ACLShort ’10, Association for Computational Linguistics, pp. 220–224. [our card](https://md.research.elamur.ai/free/en/papers/moore-2010-intelligent-selection-language-model-training/)
27. Papineni, K., Roukos, S., Ward, T., and Zhu, W.-J. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting on association for computational linguistics (2002), Association for Computational Linguistics, pp. 311–318. [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
28. Sennrich, R., Haddow, B., and Birch, A. Improving neural machine translation models with monolingual data. arXiv preprint arXiv:1511.06709 (2015). [↗](https://arxiv.org/abs/1511.06709)
29. Sennrich, R., Haddow, B., and Birch, A. Neural machine translation of rare words with subword units. arXiv preprint arXiv:1508.07909 (2015). [↗](https://arxiv.org/abs/1508.07909) [our card](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/)
30. Shen, W., and Liu, R. Learning residual images for face attribute manipulation. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2017), IEEE, pp. 1225–1233. [find](https://scholar.google.com/scholar?q=Shen%2C+W%2E%2C+and+Liu%2C+R%2E+Learning+residual+images+for+face+attribute+manipulation%2E+In+2017+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVPR%29+%282017%29%2C+IEEE%2C+pp%2E+1225%E2%80%931233%2E)
31. Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., Driessche, G. V. D., Schrittwieser, J., Antonoglou, I., Panneershelvam, V., Lanctot, M., and et al. Mastering the game of go with deep neural networks and tree search. Nature, vol. 529, pp. 484489 (2016). CoRR [find](https://scholar.google.com/scholar?q=Silver%2C+D%2E%2C+Huang%2C+A%2E%2C+Maddison%2C+C%2E+J%2E%2C+Guez%2C+A%2E%2C+Sifre%2C+L%2E%2C+Driessche%2C+G%2E+V%2E+D%2E%2C+Schrittwieser%2C+J%2E%2C+Antonoglou%2C+I%2E%2C+Panneershelvam%2C+V%2E%2C+Lanctot%2C+M%2E%2C+and+et+al%2E+Mastering+the+game+of+go+with+deep+neur)
32. Snover, M., Dorr, B., Schwartz, R., Micciulla, L., and Makhoul, J. A study of translation edit rate with targeted human annotation. In Proceedings of association for machine translation in the Americas (2006), vol. 200. [our card](https://md.research.elamur.ai/free/en/papers/snover-2006-study-translation-edit-rate-targeted/)
33. Sutskever, I., Vinyals, O., and Le, Q. V. Sequence to sequence learning with neural networks. NIPS (2014). [find](https://scholar.google.com/scholar?q=Sutskever%2C+I%2E%2C+Vinyals%2C+O%2E%2C+and+Le%2C+Q%2E+V%2E+Sequence+to+sequence+learning+with+neural+networks%2E+NIPS+%282014%29%2E)
34. Tang, D., Duan, N., Qin, T., and Zhou, M. Question answering and question generation as dual tasks. arXiv preprint arXiv:1706.02027 (2017). [↗](https://arxiv.org/abs/1706.02027)
35. Tu, Z., Lu, Z., Liu, Y., Liu, X., and Li, H. Modeling coverage for neural machine translation. arXiv preprint arXiv:1601.04811 (2016). [↗](https://arxiv.org/abs/1601.04811) [our card](https://md.research.elamur.ai/free/en/papers/tu-2016-modeling-coverage-neural-machine-translation/)
36. van der Wees, M., Bisazza, A., and Monz, C. Dynamic data selection for neural machine translation. CoRR abs/1708.00712 (2017). [↗](https://arxiv.org/abs/1708.00712)
37. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is all you need. arXiv preprint arXiv:1706.03762 (2017). [↗](https://arxiv.org/abs/1706.03762) [our card](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/)
38. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L., and Polosukhin, I. Attention is all you need. In NIPS (2017). [our card](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/)
39. Vilar, D., Xu, J., D’Haro, L. F., and Ney, H. Error analysis of statistical machine translation output. In Proceedings of the Fifth International Conference on Language Resources and Evaluation, LREC 2006, Genoa, Italy, May 22-28, 2006. (2006), pp. 697–702. [our card](https://md.research.elamur.ai/free/en/papers/vilar-2006-error-analysis-statistical-machine-translation/)
40. Wang, Y., Cheng, S., Jiang, L., Yang, J., Chen, W., Li, M., Shi, L., Wang, Y., and Yang, H. Sogou neural machine translation systems for WMT17. In Proceedings of the Second Conference on Machine Translation, WMT 2017, Copenhagen, Denmark, September 7-8, 2017 (2017), pp. 410–415. [find](https://scholar.google.com/scholar?q=Wang%2C+Y%2E%2C+Cheng%2C+S%2E%2C+Jiang%2C+L%2E%2C+Yang%2C+J%2E%2C+Chen%2C+W%2E%2C+Li%2C+M%2E%2C+Shi%2C+L%2E%2C+Wang%2C+Y%2E%2C+and+Yang%2C+H%2E+Sogou+neural+machine+translation+systems+for+WMT17%2E+In+Proceedings+of+the+Second+Conference+on+Machine+Trans)
41. Wang, Y., Li, X., Cheng, S., Jiang, L., Yang, J., Chen, W., Shi, L., Wang, Y., and Yang, H. Sogou neural machine translation systems for wmt17. In Proceedings of the Second Conference on Machine Translation, Volume 2: Shared Task Papers (Copenhagen, Denmark, September 2017), Association for Computational Linguistics, pp. 410–415. [find](https://scholar.google.com/scholar?q=Wang%2C+Y%2E%2C+Li%2C+X%2E%2C+Cheng%2C+S%2E%2C+Jiang%2C+L%2E%2C+Yang%2C+J%2E%2C+Chen%2C+W%2E%2C+Shi%2C+L%2E%2C+Wang%2C+Y%2E%2C+and+Yang%2C+H%2E+Sogou+neural+machine+translation+systems+for+wmt17%2E+In+Proceedings+of+the+Second+Conference+on+Machine+Trans)
42. Wang, Y., Xia, Y., Zhao, L., Bian, J., Qin, T., Liu, G., and Liu, T. Dual transfer learning for neural machine translation with marginal distribution regularization. In AAAI (2018). [find](https://scholar.google.com/scholar?q=Wang%2C+Y%2E%2C+Xia%2C+Y%2E%2C+Zhao%2C+L%2E%2C+Bian%2C+J%2E%2C+Qin%2C+T%2E%2C+Liu%2C+G%2E%2C+and+Liu%2C+T%2E+Dual+transfer+learning+for+neural+machine+translation+with+marginal+distribution+regularization%2E+In+AAAI+%282018%29%2E)
43. Wilcoxon, F. Individual Comparisons by Ranking Methods. Biometrics Bulletin 1, 6 (1945), 80–83. [find](https://scholar.google.com/scholar?q=Wilcoxon%2C+F%2E+Individual+Comparisons+by+Ranking+Methods%2E+Biometrics+Bulletin+1%2C+6+%281945%29%2C+80%E2%80%9383%2E)
44. Wu, Y., Schuster, M., Chen, Z., Le, Q. V., Norouzi, M., Macherey, W., Krikun, M., Cao, Y., Gao, Q., Macherey, K., Klingner, J., Shah, A., Johnson, M., Liu, X., Kaiser, Ł., Gouws, S., Kato, Y., Kudo, T., Kazawa, H., Stevens, K., Kurian, G., Patil, N., Wang, W., Young, C., Smith, J., Riesa, J., Rudnick, A., Vinyals, O., Corrado, G., Hughes, M., and Dean, J. Google’s Neural Machine Translation System: Bridging the Gap between Human and Machine Translation. ArXiv e-prints (Sept. 2016). [find](https://scholar.google.com/scholar?q=Wu%2C+Y%2E%2C+Schuster%2C+M%2E%2C+Chen%2C+Z%2E%2C+Le%2C+Q%2E+V%2E%2C+Norouzi%2C+M%2E%2C+Macherey%2C+W%2E%2C+Krikun%2C+M%2E%2C+Cao%2C+Y%2E%2C+Gao%2C+Q%2E%2C+Macherey%2C+K%2E%2C+Klingner%2C+J%2E%2C+Shah%2C+A%2E%2C+Johnson%2C+M%2E%2C+Liu%2C+X%2E%2C+Kaiser%2C+%C5%81%2E%2C+Gouws%2C+S%2E%2C+Kato%2C+Y%2E%2C+Kudo%2C+T)
45. Xia, Y., Bian, J., Qin, T., Yu, N., and Liu, T.-Y. Dual inference for machine learning. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17 (2017), pp. 3112–3118. [find](https://scholar.google.com/scholar?q=Xia%2C+Y%2E%2C+Bian%2C+J%2E%2C+Qin%2C+T%2E%2C+Yu%2C+N%2E%2C+and+Liu%2C+T%2E%2DY%2E+Dual+inference+for+machine+learning%2E+In+Proceedings+of+the+Twenty%2DSixth+International+Joint+Conference+on+Artificial+Intelligence%2C+IJCAI%2D17+%282017%29%2C+p)
46. Xia, Y., Qin, T., Chen, W., Bian, J., Yu, N., and Liu, T. Dual supervised learning. In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017 (2017), pp. 3789–3798. [find](https://scholar.google.com/scholar?q=Xia%2C+Y%2E%2C+Qin%2C+T%2E%2C+Chen%2C+W%2E%2C+Bian%2C+J%2E%2C+Yu%2C+N%2E%2C+and+Liu%2C+T%2E+Dual+supervised+learning%2E+In+Proceedings+of+the+34th+International+Conference+on+Machine+Learning%2C+ICML+2017%2C+Sydney%2C+NSW%2C+Australia%2C+6%2D11+Aug)
47. Xia, Y., Tian, F., Wu, L., Lin, J., Qin, T., Yu, N., and Liu, T.-Y. Deliberation networks: Sequence generation beyond one-pass decoding. In Advances in Neural Information Processing Systems (2017), pp. 1782–1792. [find](https://scholar.google.com/scholar?q=Xia%2C+Y%2E%2C+Tian%2C+F%2E%2C+Wu%2C+L%2E%2C+Lin%2C+J%2E%2C+Qin%2C+T%2E%2C+Yu%2C+N%2E%2C+and+Liu%2C+T%2E%2DY%2E+Deliberation+networks%3A+Sequence+generation+beyond+one%2Dpass+decoding%2E+In+Advances+in+Neural+Information+Processing+Systems+%282017%29%2C+pp)
48. Xiong, W., Droppo, J., Huang, X., Seide, F., Seltzer, M. L., Stolcke, A., Yu, D., and Zweig, G. Toward human parity in conversational speech recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing 25, 12 (2017), 2410–2423. [find](https://scholar.google.com/scholar?q=Xiong%2C+W%2E%2C+Droppo%2C+J%2E%2C+Huang%2C+X%2E%2C+Seide%2C+F%2E%2C+Seltzer%2C+M%2E+L%2E%2C+Stolcke%2C+A%2E%2C+Yu%2C+D%2E%2C+and+Zweig%2C+G%2E+Toward+human+parity+in+conversational+speech+recognition%2E+IEEE%2FACM+Transactions+on+Audio%2C+Speech%2C+and+La)
49. Yi, Z., Zhang, H., Tan, P., and Gong, M. Dualgan: Unsupervised dual learning for image-to-image translation. ICCV (2017). [find](https://scholar.google.com/scholar?q=Yi%2C+Z%2E%2C+Zhang%2C+H%2E%2C+Tan%2C+P%2E%2C+and+Gong%2C+M%2E+Dualgan%3A+Unsupervised+dual+learning+for+image%2Dto%2Dimage+translation%2E+ICCV+%282017%29%2E)
50. Zhang, Z., Liu, S., Li, M., Zhou, M., and Chen, E. Joint training for neural machine translation models with monolingual data, 2018. [find](https://scholar.google.com/scholar?q=Zhang%2C+Z%2E%2C+Liu%2C+S%2E%2C+Li%2C+M%2E%2C+Zhou%2C+M%2E%2C+and+Chen%2C+E%2E+Joint+training+for+neural+machine+translation+models+with+monolingual+data%2C+2018%2E)
51. Zhu, J.-Y., Park, T., Isola, P., and Efros, A. A. Unpaired image-to-image translation using cycle-consistent adversarial networks. arXiv preprint arXiv:1703.10593 (2017). [↗](https://arxiv.org/abs/1703.10593)
52. Zoph, B., Yuret, D., May, J., and Knight, K. Transfer learning for low-resource neural machine translation. arXiv preprint arXiv:1604.02201 (2016). [↗](https://arxiv.org/abs/1604.02201)