---
title: "Probing the Need for Visual Context in — Caglayan 2019"
description: "Earlier multimodal machine translation (MMT) studies found that images add little or nothing."
authors: ["Ozan Caglayan", "Pranava Madhyastha", "Lucia Specia", "Loïc Barrault"]
year: 2019
venue: "Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)"
source: "https://aclanthology.org/N19-1422/"
doi: "10.18653/v1/n19-1422"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/caglayan-2019-probing-need-visual-context-multimodal/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/caglayan-2019-probing-need-visual-context-multimodal/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/) / Probing the Need for Visual Context in Multimodal Machine Translation

# Probing the Need for Visual Context in Multimodal Machine Translation

**Authors:** Ozan Caglayan, Pranava Madhyastha, [Lucia Specia](https://md.research.elamur.ai/free/en/authors/lucia-specia/), [Loïc Barrault](https://md.research.elamur.ai/free/en/authors/loic-barrault/) · **Year:** 2019 · **Venue:** Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/N19-1422/) · [PDF](https://aclanthology.org/N19-1422.pdf) · [DOI 10.18653/v1/n19-1422](https://doi.org/10.18653/v1/n19-1422) · [OpenAlex](https://openalex.org/W2924679246)

**Citations:** 139 citations (openalex-snapshot · 2026-10-14) · [75](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Earlier multimodal machine translation (MMT) studies found that images add little or nothing. Many attributed this to weak image features or poor fusion methods. The authors propose another cause: the Multi30K captions are short, simple and repetitive, so the English source already holds enough context and the image becomes redundant.

To test this, they deliberately remove information from the source text in both training and test data. They mask colour words, mask head nouns of visually grounded entities, and keep only the first k tokens. They then compare a text-only attentive GRU NMT baseline with three MMT variants (DIRECT, HIER, INIT) that use ResNet-50 image features. They also measure visual sensitivity by pairing test sentences with mismatched images.

With complete sentences, the multimodal systems are not significantly better than the baseline. Once textual context is reduced, they clearly outperform NMT and are hurt by mismatched images, so they do use the visual input. The work suggests that MMT needs data where image and text are complementary. It also suggests that visual grounding could make translation more robust to noisy input.

## Key points
- **Setup:** Main experiments use English→French Multi30K (about 30K training sentences, evaluated on test2017). Entity masking is also run for German and Czech using Flickr30K entity annotations. Scores are METEOR averaged over three runs, and the models are built with nmtpytorch.
- **Full data:** On unmodified data, NMT scores 70.6 METEOR and the MMT systems reach 70.7–70.9, which is not a significant difference.
- **Colour deprivation:** All MMT models become significantly better than NMT. On the 247 affected test sentences, HIER gains 1.6 METEOR. Colour accuracy rises from 32.5% to 44.5% with the attentive models, compared with 36.5% for INIT.
- **Entity masking:** Attentive models gain up to 4.2 METEOR on French, with smaller gains for German and Czech. Mismatched images cut scores by up to 9.7 points.
- **Progressive masking:** At k=0 the task effectively becomes image captioning, and MMT beats NMT by about 7 METEOR. Sensitivity to mismatched images shrinks as more source words remain. Models trained on mismatched images from the start ("blinded") perform no better than NMT.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/)

## Cited by

Cited by

75 papers of our library cite it (of the 140884 whose bibliographies we read); 2 of them have a page here, 73 not yet.

- [The Importance of Modeling Social Factors of Language: Theory and Practice](https://md.research.elamur.ai/free/en/papers/hovy-2021-importance-modeling-social-factors-language/)
- [The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes](https://md.research.elamur.ai/free/en/papers/kiela-2020-hateful-memes-challenge-detecting-hate/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{caglayan2019probing,
  title = {{Probing the Need for Visual Context in Multimodal Machine Translation}},
  author = {Caglayan, Ozan and Madhyastha, Pranava and Specia, Lucia and Barrault, Loïc},
  booktitle = {Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers)},
  year = {2019},
  doi = {10.18653/v1/n19-1422},
  url = {https://doi.org/10.18653/v1/n19-1422}
}
```

- [← previous Probing Natural Language Inference Models through Semantic Fragments](https://md.research.elamur.ai/free/en/papers/richardson-2019-probing-natural-language-inference-models/)
- [next → Probing What Different NLP Tasks Teach Machines about Function Word Comprehension](https://md.research.elamur.ai/free/en/papers/kim-2019-probing-what-different-nlp-tasks/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/)

## References

31 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. Computing Research Repository, arXiv:1409.0473. Version 7. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2014%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+Computing+Research+Repository%2C+arXiv%3A1409%2E0473%2E+Version+7%2E) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
2. Loı̈c Barrault, Fethi Bougares, Lucia Specia, Chiraag Lala, Desmond Elliott, and Stella Frank. 2018. Findings of the third shared task on multimodal machine translation. In Proceedings of the Third Conference on Machine Translation, Volume 2: Shared Task Papers, pages 308–327, Belgium, Brussels. Ass… [find](https://scholar.google.com/scholar?q=Lo%C4%B1%CC%88c+Barrault%2C+Fethi+Bougares%2C+Lucia+Specia%2C+Chiraag+Lala%2C+Desmond+Elliott%2C+and+Stella+Frank%2E+2018%2E+Findings+of+the+third+shared+task+on+multimodal+machine+translation%2E+In+Proceedings+of+the+Third+Co) [our card](https://md.research.elamur.ai/free/en/papers/barrault-2018-findings-third-shared-task-multimodal/)
3. Ozan Caglayan, Walid Aransa, Adrien Bardet, Mercedes Garcı́a-Martı́nez, Fethi Bougares, Loı̈c Barrault, Marc Masana, Luis Herranz, and Joost van de Weijer. 2017a. LIUM-CVC submissions for WMT17 multimodal translation task. In Proceedings of the Second Conference on Machine Translation, Volume 2: Sha… [find](https://scholar.google.com/scholar?q=Ozan+Caglayan%2C+Walid+Aransa%2C+Adrien+Bardet%2C+Mercedes+Garc%C4%B1%CC%81a%2DMart%C4%B1%CC%81nez%2C+Fethi+Bougares%2C+Lo%C4%B1%CC%88c+Barrault%2C+Marc+Masana%2C+Luis+Herranz%2C+and+Joost+van+de+Weijer%2E+2017a%2E+LIUM%2DCVC+submissions+for+WMT17+multim)
4. Ozan Caglayan, Adrien Bardet, Fethi Bougares, Loı̈c Barrault, Kai Wang, Marc Masana, Luis Herranz, and Joost van de Weijer. 2018. LIUM-CVC submissions for WMT18 multimodal translation task. In Proceedings of the Third Conference on Machine Translation, Volume 2: Shared Task Papers, pages 603–608, Be… [find](https://scholar.google.com/scholar?q=Ozan+Caglayan%2C+Adrien+Bardet%2C+Fethi+Bougares%2C+Lo%C4%B1%CC%88c+Barrault%2C+Kai+Wang%2C+Marc+Masana%2C+Luis+Herranz%2C+and+Joost+van+de+Weijer%2E+2018%2E+LIUM%2DCVC+submissions+for+WMT18+multimodal+translation+task%2E+In+Proceed)
5. Ozan Caglayan, Loı̈c Barrault, and Fethi Bougares. 2016. Multimodal attention for neural machine translation. Computing Research Repository, arXiv:1609.03976. — [arXiv:1609.03976](https://arxiv.org/abs/1609.03976) [find](https://scholar.google.com/scholar?q=Ozan+Caglayan%2C+Lo%C4%B1%CC%88c+Barrault%2C+and+Fethi+Bougares%2E+2016%2E+Multimodal+attention+for+neural+machine+translation%2E+Computing+Research+Repository%2C+arXiv%3A1609%2E03976%2E+%E2%80%94+%5BarXiv%3A1609%2E03976%5D%28https%3A%2F%2Farxiv%2Eorg%2Fab)
6. Ozan Caglayan, Mercedes Garcı́a-Martı́nez, Adrien Bardet, Walid Aransa, Fethi Bougares, and Loı̈c Barrault. 2017b. NMTPY: A flexible toolkit for advanced neural machine translation systems. Prague Bull. Math. Linguistics, 109:15–28. [find](https://scholar.google.com/scholar?q=Ozan+Caglayan%2C+Mercedes+Garc%C4%B1%CC%81a%2DMart%C4%B1%CC%81nez%2C+Adrien+Bardet%2C+Walid+Aransa%2C+Fethi+Bougares%2C+and+Lo%C4%B1%CC%88c+Barrault%2E+2017b%2E+NMTPY%3A+A+flexible+toolkit+for+advanced+neural+machine+translation+systems%2E+Prague+Bul)
7. Iacer Calixto, Desmond Elliott, and Stella Frank. 2016. DCU-UvA multimodal MT system report. In Proceedings of the First Conference on Machine Translation, pages 634–638, Berlin, Germany. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Iacer+Calixto%2C+Desmond+Elliott%2C+and+Stella+Frank%2E+2016%2E+DCU%2DUvA+multimodal+MT+system+report%2E+In+Proceedings+of+the+First+Conference+on+Machine+Translation%2C+pages+634%E2%80%93638%2C+Berlin%2C+Germany%2E+Association+)
8. Iacer Calixto and Qun Liu. 2017. Incorporating global visual features into attention-based neural machine translation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 992–1003, Copenhagen, Denmark. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Iacer+Calixto+and+Qun+Liu%2E+2017%2E+Incorporating+global+visual+features+into+attention%2Dbased+neural+machine+translation%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Language+Pr)
9. Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Langua… [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2C+Bart+van+Merrienboer%2C+Caglar+Gulcehre%2C+Dzmitry+Bahdanau%2C+Fethi+Bougares%2C+Holger+Schwenk%2C+and+Yoshua+Bengio%2E+2014%2E+Learning+phrase+representations+using+RNN+encoder%E2%80%93decoder+for+statistic)
10. Jonathan H. Clark, Chris Dyer, Alon Lavie, and Noah A. Smith. 2011. Better hypothesis testing for statistical machine translation: Controlling for optimizer instability. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Pap… [find](https://scholar.google.com/scholar?q=Jonathan+H%2E+Clark%2C+Chris+Dyer%2C+Alon+Lavie%2C+and+Noah+A%2E+Smith%2E+2011%2E+Better+hypothesis+testing+for+statistical+machine+translation%3A+Controlling+for+optimizer+instability%2E+In+Proceedings+of+the+49th+Ann)
11. Francis B. Colavita. 1974. Human sensory dominance. Perception & Psychophysics, 16 (2):409–412. [find](https://scholar.google.com/scholar?q=Francis+B%2E+Colavita%2E+1974%2E+Human+sensory+dominance%2E+Perception+%26+Psychophysics%2C+16+%282%29%3A409%E2%80%93412%2E)
12. J. Deng, W. Dong, R. Socher, L. Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE Conference on Computer Vision and Pattern Recognition, pages 248– 255. [find](https://scholar.google.com/scholar?q=J%2E+Deng%2C+W%2E+Dong%2C+R%2E+Socher%2C+L%2E+Li%2C+Kai+Li%2C+and+Li+Fei%2DFei%2E+2009%2E+Imagenet%3A+A+large%2Dscale+hierarchical+image+database%2E+In+2009+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+pages+248%E2%80%93+25)
13. Michael Denkowski and Alon Lavie. 2014. Meteor universal: Language specific translation evaluation for any target language. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 376–380. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Michael+Denkowski+and+Alon+Lavie%2E+2014%2E+Meteor+universal%3A+Language+specific+translation+evaluation+for+any+target+language%2E+In+Proceedings+of+the+Ninth+Workshop+on+Statistical+Machine+Translation%2C+pag) [our card](https://md.research.elamur.ai/free/en/papers/denkowski-2014-meteor-universal-language-specific-translation/)
14. Desmond Elliott. 2018. Adversarial evaluation of multimodal machine translation. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2974–2978, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Desmond+Elliott%2E+2018%2E+Adversarial+evaluation+of+multimodal+machine+translation%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+2974%E2%80%932978%2C+Brussels%2C+B)
15. Desmond Elliott, Stella Frank, Loı̈c Barrault, Fethi Bougares, and Lucia Specia. 2017. Findings of the second shared task on multimodal machine translation and multilingual image description. In Proceedings of the Second Conference on Machine Translation, Volume 2: Shared Task Papers, pages 215– 233… [find](https://scholar.google.com/scholar?q=Desmond+Elliott%2C+Stella+Frank%2C+Lo%C4%B1%CC%88c+Barrault%2C+Fethi+Bougares%2C+and+Lucia+Specia%2E+2017%2E+Findings+of+the+second+shared+task+on+multimodal+machine+translation+and+multilingual+image+description%2E+In+Proce) [our card](https://md.research.elamur.ai/free/en/papers/elliott-2017-findings-second-shared-task-multimodal/)
16. Desmond Elliott, Stella Frank, Khalil Sima’an, and Lucia Specia. 2016. Multi30K: Multilingual english-german image descriptions. In Proceedings of the 5th Workshop on Vision and Language, pages 70– 74, Berlin, Germany. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Desmond+Elliott%2C+Stella+Frank%2C+Khalil+Sima%E2%80%99an%2C+and+Lucia+Specia%2E+2016%2E+Multi30K%3A+Multilingual+english%2Dgerman+image+descriptions%2E+In+Proceedings+of+the+5th+Workshop+on+Vision+and+Language%2C+pages+70%E2%80%93+74) [our card](https://md.research.elamur.ai/free/en/papers/elliott-2016-multi30k-multilingual-english-german-image/)
17. Stig-Arne Grönroos, Benoit Huet, Mikko Kurimo, Jorma Laaksonen, Bernard Merialdo, Phu Pham, Mats Sjöberg, Umut Sulubacak, Jörg Tiedemann, Raphael Troncy, and Raúl Vázquez. 2018. The MeMAD submission to the WMT18 multimodal translation task. In Proceedings of the Third Conference on Machine Tran… [find](https://scholar.google.com/scholar?q=Stig%2DArne+Gro%CC%88nroos%2C+Benoit+Huet%2C+Mikko+Kurimo%2C+Jorma+Laaksonen%2C+Bernard+Merialdo%2C+Phu+Pham%2C+Mats+Sjo%CC%88berg%2C+Umut+Sulubacak%2C+Jo%CC%88rg+Tiedemann%2C+Raphael+Troncy%2C+and+Rau%CC%81l+Va%CC%81zquez%2E+2018%2E+The+MeMAD+submiss)
18. K. He, X. Zhang, S. Ren, and J. Sun. 2016. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778. [find](https://scholar.google.com/scholar?q=K%2E+He%2C+X%2E+Zhang%2C+S%2E+Ren%2C+and+J%2E+Sun%2E+2016%2E+Deep+residual+learning+for+image+recognition%2E+In+2016+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVPR%29%2C+pages+770%E2%80%93778%2E)
19. Jindřich Helcl, Jindřich Libovický, and Dusan Varis. 2018. CUNI system for the WMT18 multimodal translation task. In Proceedings of the Third Conference on Machine Translation, Volume 2: Shared Task Papers, pages 622–629, Belgium, Brussels. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jindr%CC%8Cich+Helcl%2C+Jindr%CC%8Cich+Libovicky%CC%81%2C+and+Dusan+Varis%2E+2018%2E+CUNI+system+for+the+WMT18+multimodal+translation+task%2E+In+Proceedings+of+the+Third+Conference+on+Machine+Translation%2C+Volume+2%3A+Shared+Tas)
20. Po-Yao Huang, Frederick Liu, Sz-Rung Shiang, Jean Oh, and Chris Dyer. 2016. Attention-based multimodal neural machine translation. In Proceedings of the First Conference on Machine Translation, pages 639–645, Berlin, Germany. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Po%2DYao+Huang%2C+Frederick+Liu%2C+Sz%2DRung+Shiang%2C+Jean+Oh%2C+and+Chris+Dyer%2E+2016%2E+Attention%2Dbased+multimodal+neural+machine+translation%2E+In+Proceedings+of+the+First+Conference+on+Machine+Translation%2C+pages+) [our card](https://md.research.elamur.ai/free/en/papers/huang-2016-attention-based-multimodal-neural-machine/)
21. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. Computing Research Repository, arXiv:1412.6980. [find](https://scholar.google.com/scholar?q=Diederik+Kingma+and+Jimmy+Ba%2E+2014%2E+Adam%3A+A+method+for+stochastic+optimization%2E+Computing+Research+Repository%2C+arXiv%3A1412%2E6980%2E)
22. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Pro… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+Chris+Dyer%2C+Ondr%CC%8Cej+Bojar%2C+Alexandra+Const)
23. Chiraag Lala, Pranava Swaroop Madhyastha, Carolina Scarton, and Lucia Specia. 2018. Sheffield submissions for WMT18 multimodal translation shared task. In Proceedings of the Third Conference on Machine Translation, Volume 2: Shared Task Papers, pages 630–637, Belgium, Brussels. Association for Compu… [find](https://scholar.google.com/scholar?q=Chiraag+Lala%2C+Pranava+Swaroop+Madhyastha%2C+Carolina+Scarton%2C+and+Lucia+Specia%2E+2018%2E+Sheffield+submissions+for+WMT18+multimodal+translation+shared+task%2E+In+Proceedings+of+the+Third+Conference+on+Machin)
24. Jindřich Libovický and Jindřich Helcl. 2017. Attention strategies for multi-source sequence-to-sequence learning. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 196–202, Vancouver, Canada. Association for Computational Li… [find](https://scholar.google.com/scholar?q=Jindr%CC%8Cich+Libovicky%CC%81+and+Jindr%CC%8Cich+Helcl%2E+2017%2E+Attention+strategies+for+multi%2Dsource+sequence%2Dto%2Dsequence+learning%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Computational+Ling)
25. Mingbo Ma, Dapeng Li, Kai Zhao, and Liang Huang. 2017. OSU multimodal machine translation system report. In Proceedings of the Second Conference on Machine Translation, Volume 2: Shared Task Papers, pages 465–469, Copenhagen, Denmark. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mingbo+Ma%2C+Dapeng+Li%2C+Kai+Zhao%2C+and+Liang+Huang%2E+2017%2E+OSU+multimodal+machine+translation+system+report%2E+In+Proceedings+of+the+Second+Conference+on+Machine+Translation%2C+Volume+2%3A+Shared+Task+Papers%2C+p)
26. Pranava Swaroop Madhyastha, Josiah Wang, and Lucia Specia. 2017. Sheffield MultiMT: Using object posterior predictions for multimodal machine translation. In Proceedings of the Second Conference on Machine Translation, Volume 2: Shared Task Papers, pages 470–476, Copenhagen, Denmark. Association for… [find](https://scholar.google.com/scholar?q=Pranava+Swaroop+Madhyastha%2C+Josiah+Wang%2C+and+Lucia+Specia%2E+2017%2E+Sheffield+MultiMT%3A+Using+object+posterior+predictions+for+multimodal+machine+translation%2E+In+Proceedings+of+the+Second+Conference+on+Ma)
27. Ofir Press and Lior Wolf. 2016. Using the output embedding to improve language models. Computing Research Repository, arXiv:1608.05859. [find](https://scholar.google.com/scholar?q=Ofir+Press+and+Lior+Wolf%2E+2016%2E+Using+the+output+embedding+to+improve+language+models%2E+Computing+Research+Repository%2C+arXiv%3A1608%2E05859%2E) [our card](https://md.research.elamur.ai/free/en/papers/press-2017-output-embedding-improve-language-models/)
28. Rico Sennrich, Orhan Firat, Kyunghyun Cho, Alexandra Birch, Barry Haddow, Julian Hitschler, Marcin Junczys-Dowmunt, Samuel Läubli, Antonio Valerio Miceli Barone, Jozef Mokry, and Maria Nadejde. 2017. Nematus: a toolkit for neural machine translation. In Proceedings of the Software Demonstrations of… [find](https://scholar.google.com/scholar?q=Rico+Sennrich%2C+Orhan+Firat%2C+Kyunghyun+Cho%2C+Alexandra+Birch%2C+Barry+Haddow%2C+Julian+Hitschler%2C+Marcin+Junczys%2DDowmunt%2C+Samuel+La%CC%88ubli%2C+Antonio+Valerio+Miceli+Barone%2C+Jozef+Mokry%2C+and+Maria+Nadejde%2E+2017%2E) [our card](https://md.research.elamur.ai/free/en/papers/sennrich-2017-nematus-toolkit-neural-machine-translation/)
29. Lucia Specia, Stella Frank, Khalil Sima’an, and Desmond Elliott. 2016. A shared task on multimodal machine translation and crosslingual image description. In Proceedings of the First Conference on Machine Translation, pages 543–553, Berlin, Germany. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Lucia+Specia%2C+Stella+Frank%2C+Khalil+Sima%E2%80%99an%2C+and+Desmond+Elliott%2E+2016%2E+A+shared+task+on+multimodal+machine+translation+and+crosslingual+image+description%2E+In+Proceedings+of+the+First+Conference+on+Mac) [our card](https://md.research.elamur.ai/free/en/papers/specia-2016-shared-task-multimodal-machine-translation/)
30. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: A simple way to prevent neural networks from overfitting. J. Mach. Learn. Res., 15(1):1929– 1958. [find](https://scholar.google.com/scholar?q=Nitish+Srivastava%2C+Geoffrey+Hinton%2C+Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Ruslan+Salakhutdinov%2E+2014%2E+Dropout%3A+A+simple+way+to+prevent+neural+networks+from+overfitting%2E+J%2E+Mach%2E+Learn%2E+Res%2E%2C+15%281%29%3A1929)
31. Peter Young, Alice Lai, Micah Hodosh, and Julia Hockenmaier. 2014. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2:67–78. [find](https://scholar.google.com/scholar?q=Peter+Young%2C+Alice+Lai%2C+Micah+Hodosh%2C+and+Julia+Hockenmaier%2E+2014%2E+From+image+descriptions+to+visual+denotations%3A+New+similarity+metrics+for+semantic+inference+over+event+descriptions%2E+Transactions+of)