---
title: "MTNT: A Testbed for Machine Translation of — Michel 2018"
description: "Modern MT systems, including neural ones, often break down on the informal, error-filled language of social media."
authors: ["Paul Michel", "Graham Neubig"]
year: 2018
venue: "Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D18-1050/"
doi: "10.18653/v1/d18-1050"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/michel-2018-mtnt-testbed-machine-translation-noisy/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/michel-2018-mtnt-testbed-machine-translation-noisy/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/) / MTNT: A Testbed for Machine Translation of Noisy Text

# MTNT: A Testbed for Machine Translation of Noisy Text

**Authors:** [Paul Michel](https://md.research.elamur.ai/free/en/authors/paul-michel/), [Graham Neubig](https://md.research.elamur.ai/free/en/authors/graham-neubig/) · **Year:** 2018 · **Venue:** Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D18-1050/) · [PDF](https://aclanthology.org/D18-1050.pdf) · [DOI 10.18653/v1/d18-1050](https://doi.org/10.18653/v1/d18-1050) · [OpenAlex](https://openalex.org/W2889746679)

**Citations:** 122 citations (OpenAlex, 2026-10-14) · [102](https://md.research.elamur.ai/free/en/papers/michel-2018-mtnt-testbed-machine-translation-noisy/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary
Modern MT systems, including neural ones, often break down on the informal, error-filled language of social media. Researchers had no public parallel corpus of naturally noisy text, so earlier robustness work relied on synthetic perturbations. Michel and Neubig fill this gap with MTNT, a benchmark built from Reddit comments in English, French and Japanese, which professional translators rendered to and from English.

Noisy comments were chosen automatically. The authors first discarded URLs, bot posts and text in other languages. They then kept comments containing words absent from clean contrast corpora (for English and French) and comments whose lines scored very low under a subword language model.

The resource covers four directions (en-fr, fr-en, en-ja, ja-en) and adds monolingual data for semi-supervised adaptation and noisy language modeling. NMT baselines trained on standard corpora perform poorly on it. Brief fine-tuning on the in-domain training split raises BLEU noticeably, but translations of heavily noisy sentences remain inadequate. MTNT therefore offers a realistic, open testbed for noise-robust translation research.

## Key points
- **Dataset size:** each language pair has about 1,000 test sentences and roughly 6k–36k training sentences.
- **Contrast corpora:**
  - English: the WMT 2017 de-en English side (≈5.85M sentences).
  - French: the WMT 2015 en-fr French side (≈40.86M sentences).
  - Japanese: KFTT, JESC and TED combined (≈4.19M sentences).
- **Noise detection:**
  - Text was segmented with sentencepiece BPE, using vocabularies of 1,000 for English and French and 4,000 for Japanese.
  - A 5-gram Kneser-Ney model built with kenLM scored each line.
  - A comment was flagged as noisy if any line fell below the 1st percentile of reference-corpus scores.
- **Fine-tuning:** one epoch of SGD on the MTNT training data improved BLEU by 3.17–7.96 points, yet very noisy inputs still failed.
- **Language-modeling baselines:**
  - Character-level baselines were a 6-gram KenLM model and AWD-LSTM.
  - AWD-LSTM was better, with test bpc of 1.810 (English), 1.705 (French) and 5.225 (Japanese).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{michel2018mtnt,
  title = {{MTNT: A Testbed for Machine Translation of Noisy Text}},
  author = {Michel, Paul and Neubig, Graham},
  booktitle = {Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing},
  year = {2018},
  doi = {10.18653/v1/d18-1050},
  url = {https://doi.org/10.18653/v1/d18-1050}
}
```

- [← previous Interpretation of Natural Language Rules in Conversational Machine Reading](https://md.research.elamur.ai/free/en/papers/saeidi-2018-interpretation-natural-language-rules-conversational/)
- [next → Few-Shot Charge Prediction with Discriminative Legal Attributes](https://md.research.elamur.ai/free/en/papers/hu-2018-few-shot-charge-prediction-discriminative/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/)

## References

39 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Amittai Axelrod, Xiaodong He, and Jianfeng Gao. Domain adaptation via pseudo in-domain data selection. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 355–362. [find](https://scholar.google.com/scholar?q=Amittai+Axelrod%2C+Xiaodong+He%2C+and+Jianfeng+Gao%2E+Domain+adaptation+via+pseudo+in%2Ddomain+data+selection%2E+In+Proceedings+of+the+2011+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+) [our card](https://md.research.elamur.ai/free/en/papers/axelrod-2011-domain-adaptation-pseudo-domain-data/)
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2014%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+In+International+Conference+on+Learning+Representations%2E) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
3. Timothy Baldwin, Paul Cook, Marco Lui, Andrew MacKinlay, and Li Wang. How noisy social media text, how diffrnt social media sources? In Proceedings of the Sixth International Joint Conference on Natural Language Processing, pages 356–364. [find](https://scholar.google.com/scholar?q=Timothy+Baldwin%2C+Paul+Cook%2C+Marco+Lui%2C+Andrew+MacKinlay%2C+and+Li+Wang%2E+How+noisy+social+media+text%2C+how+diffrnt+social+media+sources%3F+In+Proceedings+of+the+Sixth+International+Joint+Conference+on+Natur) [our card](https://md.research.elamur.ai/free/en/papers/baldwin-2013-how-noisy-social-media-text/)
4. Yonatan Belinkov and Yonatan Bisk. 2018. Synthetic and natural noise both break neural machine translation. International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Yonatan+Belinkov+and+Yonatan+Bisk%2E+2018%2E+Synthetic+and+natural+noise+both+break+neural+machine+translation%2E+International+Conference+on+Learning+Representations%2E)
5. Su Lin Blodgett, Johnny Wei, and Brendan O’Connor. 2017. A dataset and classifier for recognizing social Huda Khayrallah and Philipp Koehn. On the impact of various types of noise on neural machine translation. In Proceedings of the 2nd Workshop on Neural Machine Translation and Generation, pages 74… [find](https://scholar.google.com/scholar?q=Su+Lin+Blodgett%2C+Johnny+Wei%2C+and+Brendan+O%E2%80%99Connor%2E+2017%2E+A+dataset+and+classifier+for+recognizing+social+Huda+Khayrallah+and+Philipp+Koehn%2E+On+the+impact+of+various+types+of+noise+on+neural+machine+tr) [our card](https://md.research.elamur.ai/free/en/papers/khayrallah-2018-impact-various-types-noise-neural/)
6. Mauro Cettolo, Christian Girardi, and Marcello Federico. 2012. Wit 3 : Web inventory of transcribed and translated talks. In Proceedings of the 16 th Conference of the European Association for Machine Translation (EAMT), pages 261–268. [find](https://scholar.google.com/scholar?q=Mauro+Cettolo%2C+Christian+Girardi%2C+and+Marcello+Federico%2E+2012%2E+Wit+3+%3A+Web+inventory+of+transcribed+and+translated+talks%2E+In+Proceedings+of+the+16+th+Conference+of+the+European+Association+for+Machine)
7. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Diederik+P%2E+Kingma+and+Jimmy+Ba%2E+2014%2E+Adam%3A+A+method+for+stochastic+optimization%2E+In+International+Conference+on+Learning+Representations%2E)
8. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. Moses: Open source toolkit for statistical machine translation. In Proceedin… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+Chris+Dyer%2C+Ondr%CC%8Cej+Bojar%2C+Alexandra+Const)
9. Yong Cheng, Zhaopeng Tu, Fandong Meng, Junjie Zhai, and Yang Liu. 2018. Towards robust neural machine translation. [find](https://scholar.google.com/scholar?q=Yong+Cheng%2C+Zhaopeng+Tu%2C+Fandong+Meng%2C+Junjie+Zhai%2C+and+Yang+Liu%2E+2018%2E+Towards+robust+neural+machine+translation%2E)
10. Yong Cheng, Wei Xu, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. Semi-supervised learning for neural machine translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1965–1974. [find](https://scholar.google.com/scholar?q=Yong+Cheng%2C+Wei+Xu%2C+Zhongjun+He%2C+Wei+He%2C+Hua+Wu%2C+Maosong+Sun%2C+and+Yang+Liu%2E+Semi%2Dsupervised+learning+for+neural+machine+translation%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Co)
11. Lingpeng Kong, Nathan Schneider, Swabha Swayamdipta, Archna Bhatia, Chris Dyer, and Noah A. Smith. A dependency parser for tweets. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1001–1012. [find](https://scholar.google.com/scholar?q=Lingpeng+Kong%2C+Nathan+Schneider%2C+Swabha+Swayamdipta%2C+Archna+Bhatia%2C+Chris+Dyer%2C+and+Noah+A%2E+Smith%2E+A+dependency+parser+for+tweets%2E+In+Proceedings+of+the+2014+Conference+on+Empirical+Methods+in+Natural) [our card](https://md.research.elamur.ai/free/en/papers/aclbulk-2014-kong-dependency-parser-tweets/)
12. Chenhui Chu, Raj Dabre, and Sadao Kurohashi. An empirical comparison of domain adaptation methods for neural machine translation. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 385–391. Mu Li, Yinggong Zhao, Dongdong Zhang,… [find](https://scholar.google.com/scholar?q=Chenhui+Chu%2C+Raj+Dabre%2C+and+Sadao+Kurohashi%2E+An+empirical+comparison+of+domain+adaptation+methods+for+neural+machine+translation%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Compu)
13. David Crystal. 2001. Language and the Internet. Cambridge University Press. [find](https://scholar.google.com/scholar?q=David+Crystal%2E+2001%2E+Language+and+the+Internet%2E+Cambridge+University+Press%2E)
14. Brenda Danet and Susan Herring. 2007. The Multilingual Internet: Language, Culture, and Communication Online. Oxford University Press., New York. [find](https://scholar.google.com/scholar?q=Brenda+Danet+and+Susan+Herring%2E+2007%2E+The+Multilingual+Internet%3A+Language%2C+Culture%2C+and+Communication+Online%2E+Oxford+University+Press%2E%2C+New+York%2E)
15. Marco Lui and Timothy Baldwin. langid.py: An off-the-shelf language identification tool. In Proceedings of the Association for Computational Linguistics 2012 System Demonstrations, pages 25–30. [find](https://scholar.google.com/scholar?q=Marco+Lui+and+Timothy+Baldwin%2E+langid%2Epy%3A+An+off%2Dthe%2Dshelf+language+identification+tool%2E+In+Proceedings+of+the+Association+for+Computational+Linguistics+2012+System+Demonstrations%2C+pages+25%E2%80%9330%2E) [our card](https://md.research.elamur.ai/free/en/papers/lui-2012-langid-py-off-shelf-language/)
16. Jacob Eisenstein. What to do about bad language on the internet. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 359–369. [find](https://scholar.google.com/scholar?q=Jacob+Eisenstein%2E+What+to+do+about+bad+language+on+the+internet%2E+In+Proceedings+of+the+2013+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics%3A+Human+Language+Te) [our card](https://md.research.elamur.ai/free/en/papers/eisenstein-2013-what-do-about-bad-language/)
17. Minh-Thang Luong and Christopher D Manning. 2015. Stanford neural machine translation systems for spoken language domains. In Proceedings of the International Workshop on Spoken Language Translation. [find](https://scholar.google.com/scholar?q=Minh%2DThang+Luong+and+Christopher+D+Manning%2E+2015%2E+Stanford+neural+machine+translation+systems+for+spoken+language+domains%2E+In+Proceedings+of+the+International+Workshop+on+Spoken+Language+Translation%2E)
18. Lluís Formiga and José A. R. Fonollosa. Dealing with input noise in statistical machine translation. In Proceedings of the Conference on Computational Linguistics 2012: Posters, pages 319–328. [find](https://scholar.google.com/scholar?q=Llu%C3%ADs+Formiga+and+Jos%C3%A9+A%2E+R%2E+Fonollosa%2E+Dealing+with+input+noise+in+statistical+machine+translation%2E+In+Proceedings+of+the+Conference+on+Computational+Linguistics+2012%3A+Posters%2C+pages+319%E2%80%93328%2E)
19. Stephen Merity, Nitish Shirish Keskar, and Richard Socher. 2018. Regularizing and optimizing lstm language models. International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Stephen+Merity%2C+Nitish+Shirish+Keskar%2C+and+Richard+Socher%2E+2018%2E+Regularizing+and+optimizing+lstm+language+models%2E+International+Conference+on+Learning+Representations%2E) [our card](https://md.research.elamur.ai/free/en/papers/merity-2017-regularizing-optimizing-lstm-language-models/)
20. Kenneth Heafield, Ivan Pouzyrevsky, Jonathan H. Clark, and Philipp Koehn. Scalable modified Kneser-Ney language model estimation. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, pages 690–696. Antonio Valerio Miceli Barone, Barry Haddow, Ulrich Germann, an… [find](https://scholar.google.com/scholar?q=Kenneth+Heafield%2C+Ivan+Pouzyrevsky%2C+Jonathan+H%2E+Clark%2C+and+Philipp+Koehn%2E+Scalable+modified+Kneser%2DNey+language+model+estimation%2E+In+Proceedings+of+the+51st+Annual+Meeting+of+the+Association+for+Compu)
21. Susan Herring, editor. 2003. Media and Language Change. Special issue of Journal of Historical Pragmatics 4:1. [find](https://scholar.google.com/scholar?q=Susan+Herring%2C+editor%2E+2003%2E+Media+and+Language+Change%2E+Special+issue+of+Journal+of+Historical+Pragmatics+4%3A1%2E)
22. Pierre Isabelle, Colin Cherry, and George Foster. A challenge set approach to evaluating machine translation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2486–2496. [find](https://scholar.google.com/scholar?q=Pierre+Isabelle%2C+Colin+Cherry%2C+and+George+Foster%2E+A+challenge+set+approach+to+evaluating+machine+translation%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C) [our card](https://md.research.elamur.ai/free/en/papers/isabelle-2017-challenge-set-approach-evaluating-machine/)
23. Paul Michel and Graham Neubig. 2018. Extreme adaptation for personalized neural machine translation. [find](https://scholar.google.com/scholar?q=Paul+Michel+and+Graham+Neubig%2E+2018%2E+Extreme+adaptation+for+personalized+neural+machine+translation%2E)
24. Robert C. Moore and William Lewis. Intelligent selection of language model training data. In Proceedings of the Association for Computational Linguistics 2010 Conference Short Papers, pages 220–224. [find](https://scholar.google.com/scholar?q=Robert+C%2E+Moore+and+William+Lewis%2E+Intelligent+selection+of+language+model+training+data%2E+In+Proceedings+of+the+Association+for+Computational+Linguistics+2010+Conference+Short+Papers%2C+pages+220%E2%80%93224%2E) [our card](https://md.research.elamur.ai/free/en/papers/moore-2010-intelligent-selection-language-model-training/)
25. Nal Kalchbrenner and Phil Blunsom. Recurrent continuous translation models. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1700–1709. [find](https://scholar.google.com/scholar?q=Nal+Kalchbrenner+and+Phil+Blunsom%2E+Recurrent+continuous+translation+models%2E+In+Proceedings+of+the+2013+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+1700%E2%80%931709%2E) [our card](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2013-recurrent-continuous-translation-models/)
26. Graham Neubig. 2011. The Kyoto free translation task. [http://www.phontron.com/kftt](http://www.phontron.com/kftt). [↗](http://www.phontron.com/kftt)
27. Chenhao Tan, Vlad Niculae, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. Winning arguments: Interaction dynamics and persuasion strategies in good-faith online discussions. In Proceedings of the 25th international conference on world wide web, pages 613–624. International World Wide Web Confere… [find](https://scholar.google.com/scholar?q=Chenhao+Tan%2C+Vlad+Niculae%2C+Cristian+Danescu%2DNiculescu%2DMizil%2C+and+Lillian+Lee%2E+Winning+arguments%3A+Interaction+dynamics+and+persuasion+strategies+in+good%2Dfaith+online+discussions%2E+In+Proceedings+of+the+)
28. Graham Neubig, Chris Dyer, Yoav Goldberg, Austin Matthews, Waleed Ammar, Antonios Anastasopoulos, Miguel Ballesteros, David Chiang, Daniel Clothiaux, Trevor Cohn, et al. 2017. Dynet: The dynamic neural network toolkit. arXiv preprint arXiv:1701.03980. [find](https://scholar.google.com/scholar?q=Graham+Neubig%2C+Chris+Dyer%2C+Yoav+Goldberg%2C+Austin+Matthews%2C+Waleed+Ammar%2C+Antonios+Anastasopoulos%2C+Miguel+Ballesteros%2C+David+Chiang%2C+Daniel+Clothiaux%2C+Trevor+Cohn%2C+et+al%2E+2017%2E+Dynet%3A+The+dynamic+neura) [our card](https://md.research.elamur.ai/free/en/papers/neubig-2017-dynet-dynamic-neural-network-toolkit/)
29. Graham Neubig, Yosuke Nakata, and Shinsuke Mori. 2011. Pointwise prediction for robust, adaptable japanese morphological analysis. In The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies (ACL-HLT), pages 529–533. [find](https://scholar.google.com/scholar?q=Graham+Neubig%2C+Yosuke+Nakata%2C+and+Shinsuke+Mori%2E+2011%2E+Pointwise+prediction+for+robust%2C+adaptable+japanese+morphological+analysis%2E+In+The+49th+Annual+Meeting+of+the+Association+for+Computational+Lingu) [our card](https://md.research.elamur.ai/free/en/papers/neubig-2011-pointwise-prediction-robust-adaptable-japanese/)
30. Rui Wang, Masao Utiyama, Lemao Liu, Kehai Chen, and Eiichiro Sumita. Instance weighting for neural machine translation domain adaptation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1483–1489. [find](https://scholar.google.com/scholar?q=Rui+Wang%2C+Masao+Utiyama%2C+Lemao+Liu%2C+Kehai+Chen%2C+and+Eiichiro+Sumita%2E+Instance+weighting+for+neural+machine+translation+domain+adaptation%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+)
31. Olutobi Owoputi, Brendan O’Connor, Chris Dyer, Kevin Gimpel, Nathan Schneider, and Noah A Smith. Improved part-of-speech tagging for online conversational text with word clusters. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Olutobi+Owoputi%2C+Brendan+O%E2%80%99Connor%2C+Chris+Dyer%2C+Kevin+Gimpel%2C+Nathan+Schneider%2C+and+Noah+A+Smith%2E+Improved+part%2Dof%2Dspeech+tagging+for+online+conversational+text+with+word+clusters%2E+Association+for+Comp) [our card](https://md.research.elamur.ai/free/en/papers/owoputi-2013-improved-part-speech-tagging-online/)
32. Yonghui Wu, Mike Schuster, Zhifeng Chen, Quoc V Le, Mohammad Norouzi, Wolfgang Macherey, Maxim Krikun, Yuan Cao, Qin Gao, Klaus Macherey, et al. 2016. Google’s neural machine translation system: Bridging the gap between human and machine translation. arXiv preprint arXiv:1609.08144. [find](https://scholar.google.com/scholar?q=Yonghui+Wu%2C+Mike+Schuster%2C+Zhifeng+Chen%2C+Quoc+V+Le%2C+Mohammad+Norouzi%2C+Wolfgang+Macherey%2C+Maxim+Krikun%2C+Yuan+Cao%2C+Qin+Gao%2C+Klaus+Macherey%2C+et+al%2E+2016%2E+Google%E2%80%99s+neural+machine+translation+system%3A+Bridg)
33. Kay Peterson. 2011. Openmt12 evaluation. [find](https://scholar.google.com/scholar?q=Kay+Peterson%2E+2011%2E+Openmt12+evaluation%2E)
34. Jiajun Zhang and Chengqing Zong. Exploiting source-side monolingual data in neural machine translation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1535–1545. [find](https://scholar.google.com/scholar?q=Jiajun+Zhang+and+Chengqing+Zong%2E+Exploiting+source%2Dside+monolingual+data+in+neural+machine+translation%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2016-exploiting-source-side-monolingual-data/)
35. Matt Post. 2018. A call for clarity in reporting bleu scores. arXiv preprint arXiv:1804.08771. [find](https://scholar.google.com/scholar?q=Matt+Post%2E+2018%2E+A+call+for+clarity+in+reporting+bleu+scores%2E+arXiv+preprint+arXiv%3A1804%2E08771%2E) [our card](https://md.research.elamur.ai/free/en/papers/post-2018-call-clarity-reporting-bleu-scores/)
36. Ofir Press and Lior Wolf. Using the output embedding to improve language models. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 157–163. R. Pryzant, Y. Chung, D. Jurafsky, and D. Britz. Jesc: Japanese-engl… [find](https://scholar.google.com/scholar?q=Ofir+Press+and+Lior+Wolf%2E+Using+the+output+embedding+to+improve+language+models%2E+In+Proceedings+of+the+15th+Conference+of+the+European+Chapter+of+the+Association+for+Computational+Linguistics%3A+Volume+) [our card](https://md.research.elamur.ai/free/en/papers/press-2017-output-embedding-improve-language-models/)
37. Keisuke Sakaguchi, Kevin Duh, Matt Post, and Benjamin Van Durme. 2017. Robsut wrod reocginiton via semi-character recurrent neural network. In AAAI, pages 3281–3287. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Kevin+Duh%2C+Matt+Post%2C+and+Benjamin+Van+Durme%2E+2017%2E+Robsut+wrod+reocginiton+via+semi%2Dcharacter+recurrent+neural+network%2E+In+AAAI%2C+pages+3281%E2%80%933287%2E)
38. Rico Sennrich. How grammatical is character-level neural machine translation? assessing mt quality with contrastive translation pairs. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 376–382. Matthias Sperb… [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2E+How+grammatical+is+character%2Dlevel+neural+machine+translation%3F+assessing+mt+quality+with+contrastive+translation+pairs%2E+In+Proceedings+of+the+15th+Conference+of+the+European+Chapter+of+) [our card](https://md.research.elamur.ai/free/en/papers/sennrich-2017-how-grammatical-is-character-level/)
39. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Advances+in+neural+information+processing+systems%2C+pages+3104%E2%80%933112%2E)