---
title: "A Shared Task on Multimodal Machine — Specia 2016"
description: "Most joint language-and-vision research, such as image captioning, has worked with English only."
authors: ["Lucia Specia", "Stella Frank", "Khalil Sima'an", "Desmond Elliott"]
year: 2016
venue: "Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers"
source: "https://aclanthology.org/W16-2346/"
doi: "10.18653/v1/w16-2346"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/specia-2016-shared-task-multimodal-machine-translation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/specia-2016-shared-task-multimodal-machine-translation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/) / A Shared Task on Multimodal Machine Translation and Crosslingual Image Description

# A Shared Task on Multimodal Machine Translation and Crosslingual Image Description

**Authors:** [Lucia Specia](https://md.research.elamur.ai/free/en/authors/lucia-specia/), [Stella Frank](https://md.research.elamur.ai/free/en/authors/stella-frank/), [Khalil Sima'an](https://md.research.elamur.ai/free/en/authors/khalil-sima-an/), [Desmond Elliott](https://md.research.elamur.ai/free/en/authors/desmond-elliott/) · **Year:** 2016 · **Venue:** Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/W16-2346/) · [PDF](https://aclanthology.org/W16-2346.pdf) · [DOI 10.18653/v1/w16-2346](https://doi.org/10.18653/v1/w16-2346) · [OpenAlex](https://openalex.org/W2509282593)

**Citations:** 227 citations (openalex-snapshot · 2026-10-14) · [124](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Most joint language-and-vision research, such as image captioning, has worked with English only. This paper reports on the first shared task designed to bring multilinguality into the field, organised alongside WMT16. Both variants use English and German and the Multi30K dataset, which extends Flickr30K.

- **Task 1 (Multimodal Machine Translation):** systems translate an English image caption into German and may draw on the picture.
- **Task 2 (Crosslingual Image Description):** systems produce a German description of an image and may use English captions as extra input.

The organisers released VGG-19 image features and provided baselines: a Moses phrase-based SMT system and the GroundedTranslation model. Ten teams entered 23 systems in total.

The main outcome is that images added little. In Task 1, visual signals helped only marginally and indirectly, through reranking. In Task 2, a text-only neural system achieved the top score. This shows how difficult it is to make visual input useful and motivates more robust multimodal models and harder datasets.

## Key points
- Participation: 10 teams submitted 16 systems for Task 1 and seven for Task 2.
- Data: Multi30K covers 31,014 Flickr30K images.
  - Task 1 trains on about 29,000 professionally translated sentence pairs and is scored against one reference.
  - Task 2 trains on about 145,000 crowdsourced descriptions (via Crowdflower) and is scored against five independent German references.
- Released visual features came from the VGG-19 FC 7 (relu7) and CONV 5,4 layers.
- Task 2 results:
  - The winner was LIUM 2 TextNMT C, which used no image features (Meteor 35.1, BLEU 23.8).
  - LIUM's ResNet-50-based multimodal system came second (Meteor 32.3).
  - The GroundedTranslation baseline scored Meteor 31.2.
- English→German translation quality was surprisingly high given the small training set. The authors attribute this to simple, repetitive captions, unlike WMT News data.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Machine translation](https://md.research.elamur.ai/free/en/shelves/machine-translation/) · [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/)

## Cited by

Cited by

124 papers of our library cite it (of the 140673 whose bibliographies we read); 10 of them have a page here, 114 not yet.

- [Findings of the Third Shared Task on Multimodal Machine Translation](https://md.research.elamur.ai/free/en/papers/barrault-2018-findings-third-shared-task-multimodal/)
- [Findings of the 2016 Conference on Machine Translation](https://md.research.elamur.ai/free/en/papers/bojar-2016-findings-2016-conference-machine-translation/)
- [Probing the Need for Visual Context in Multimodal Machine Translation](https://md.research.elamur.ai/free/en/papers/caglayan-2019-probing-need-visual-context-multimodal/)
- [Findings of the Second Shared Task on Multimodal Machine Translation and Multilingual Image Description](https://md.research.elamur.ai/free/en/papers/elliott-2017-findings-second-shared-task-multimodal/)
- [Challenges and Strategies in Cross-Cultural NLP](https://md.research.elamur.ai/free/en/papers/hershcovich-2022-challenges-strategies-cross-cultural-nlp/)
- [The Hateful Memes Challenge: Detecting Hate Speech in Multimodal Memes](https://md.research.elamur.ai/free/en/papers/kiela-2020-hateful-memes-challenge-detecting-hate/)
- [Learned in Translation: Contextualized Word Vectors](https://md.research.elamur.ai/free/en/papers/mccann-2017-learned-translation-contextualized-word-vectors/)
- [Modeling Cross-Cultural Pragmatic Inference with Codenames Duet](https://md.research.elamur.ai/free/en/papers/shaikh-2023-modeling-cross-cultural-pragmatic-inference/)
- [SimVLM: Simple Visual Language Model Pretraining with Weak Supervision](https://md.research.elamur.ai/free/en/papers/wang-2021-simvlm-simple-visual-language-model/)
- [Tensor Fusion Network for Multimodal Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/zadeh-2017-tensor-fusion-network-multimodal-sentiment/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{specia2016shared,
  title = {{A Shared Task on Multimodal Machine Translation and Crosslingual Image Description}},
  author = {Specia, Lucia and Frank, Stella and Sima'an, Khalil and Elliott, Desmond},
  booktitle = {Proceedings of the First Conference on Machine Translation: Volume 2, Shared Task Papers},
  year = {2016},
  doi = {10.18653/v1/w16-2346},
  url = {https://doi.org/10.18653/v1/w16-2346}
}
```

- [← previous A research about the effect of Targums (Aramaics Old Testament) on religious literature of Hebrew-Persian](https://md.research.elamur.ai/free/en/papers/noruzi-2016-research-about-effect-targums-aramaics/)
- [next → A Stacking Gated Neural Architecture for Implicit Discourse Relation Classification](https://md.research.elamur.ai/free/en/papers/qin-2016-stacking-gated-neural-architecture-implicit/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/)

## References

34 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Desmond Elliott and Frank Keller. 2014. Comparing Automatic Evaluation Measures for Image Description. In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, pages 452–457, Baltimore, Maryland. [find](https://scholar.google.com/scholar?q=Desmond+Elliott+and+Frank+Keller%2E+2014%2E+Comparing+Automatic+Evaluation+Measures+for+Image+Description%2E+In+Proceedings+of+the+52nd+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages)
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. CoRR, abs/1409.0473. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2014%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+CoRR%2C+abs%2F1409%2E0473%2E) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
3. Raffaella Bernardi, Ruket Cakici, Desmond Elliott, Aykut Erdem, Erkut Erdem, Nazli Ikizler-Cinbis, Frank Keller, Adrian Muscat, and Barbara Plank. 2016. Automatic description generation from images: A survey of models, datasets, and evaluation measures. CoRR, abs/1601.03896. — [arXiv:1601.03896](https://arxiv.org/abs/1601.03896) [find](https://scholar.google.com/scholar?q=Raffaella+Bernardi%2C+Ruket+Cakici%2C+Desmond+Elliott%2C+Aykut+Erdem%2C+Erkut+Erdem%2C+Nazli+Ikizler%2DCinbis%2C+Frank+Keller%2C+Adrian+Muscat%2C+and+Barbara+Plank%2E+2016%2E+Automatic+description+generation+from+images%3A+A)
4. Desmond Elliott, Stella Frank, and Eva Hasler. 2015. Multi-language image description with neural sequence models. CoRR, abs/1510.04709. — [arXiv:1510.04709](https://arxiv.org/abs/1510.04709) [find](https://scholar.google.com/scholar?q=Desmond+Elliott%2C+Stella+Frank%2C+and+Eva+Hasler%2E+2015%2E+Multi%2Dlanguage+image+description+with+neural+sequence+models%2E+CoRR%2C+abs%2F1510%2E04709%2E+%E2%80%94+%5BarXiv%3A1510%2E04709%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1510%2E04709%29)
5. Desmond Elliott, Stella Frank, Khalil. Sima’an, and Lucia Specia. 2016. Multi30K: Multilingual English-German Image Descriptions. In Proceedings of the 5th Workshop on Vision and Language, Berlin, Germany. [find](https://scholar.google.com/scholar?q=Desmond+Elliott%2C+Stella+Frank%2C+Khalil%2E+Sima%E2%80%99an%2C+and+Lucia+Specia%2E+2016%2E+Multi30K%3A+Multilingual+English%2DGerman+Image+Descriptions%2E+In+Proceedings+of+the+5th+Workshop+on+Vision+and+Language%2C+Berlin%2C+Ger) [our card](https://md.research.elamur.ai/free/en/papers/elliott-2016-multi30k-multilingual-english-german-image/)
6. Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Barry Haddow, Matthias Huck, Chris Hokamp, Philipp Koehn, Varvara Logacheva, Christof Monz, Matteo Negri, Matt Post, Carolina Scarton, Lucia Specia, and Marco Turchi. 2015. Findings of the 2015 workshop on statistical machine translation. In Proc… [find](https://scholar.google.com/scholar?q=Ondr%CC%8Cej+Bojar%2C+Rajen+Chatterjee%2C+Christian+Federmann%2C+Barry+Haddow%2C+Matthias+Huck%2C+Chris+Hokamp%2C+Philipp+Koehn%2C+Varvara+Logacheva%2C+Christof+Monz%2C+Matteo+Negri%2C+Matt+Post%2C+Carolina+Scarton%2C+Lucia+Speci) [our card](https://md.research.elamur.ai/free/en/papers/bojar-2015-findings-2015-workshop-statistical-machine/)
7. Yansong Feng and Mirella Lapata. 2010. Topic models for image annotation and text illustration. In Proceedings of Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 831–839, Los Angeles, California. [find](https://scholar.google.com/scholar?q=Yansong+Feng+and+Mirella+Lapata%2E+2010%2E+Topic+models+for+image+annotation+and+text+illustration%2E+In+Proceedings+of+Human+Language+Technologies%3A+The+2010+Annual+Conference+of+the+North+American+Chapter+)
8. Ozan Caglayan, Walid Aransa, Yaxing Wang, Marc Masana, Mercedes Garcı́a-Martı́nez, Fethi Bougares, Loı̈c Barrault, and Joost van de Weijer. 2016. Does multimodality help human and machine for translation and image captioning? In Proceedings of the First Conference on Machine Translation, Berlin, Ger… [find](https://scholar.google.com/scholar?q=Ozan+Caglayan%2C+Walid+Aransa%2C+Yaxing+Wang%2C+Marc+Masana%2C+Mercedes+Garc%C4%B1%CC%81a%2DMart%C4%B1%CC%81nez%2C+Fethi+Bougares%2C+Lo%C4%B1%CC%88c+Barrault%2C+and+Joost+van+de+Weijer%2E+2016%2E+Does+multimodality+help+human+and+machine+for+translat)
9. Ruka Funaki and Hideki Nakayama. 2015. Image-mediated learning for zero-shot cross-lingual document retrieval. In Proceedings of the Conference on Empirical Methods in Natural Language Processing, pages 585–590, Lisbon, Portugal. [find](https://scholar.google.com/scholar?q=Ruka+Funaki+and+Hideki+Nakayama%2E+2015%2E+Image%2Dmediated+learning+for+zero%2Dshot+cross%2Dlingual+document+retrieval%2E+In+Proceedings+of+the+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pag)
10. Haoyuan Gao, Junhua Mao, Jie Zhou, Zhiheng Huang, Lei Wang, and Wei Xu. 2015. Are you talking to a machine? dataset and methods for multilingual image question answering. Advances in Neural Information Processing Systems, pages 2287–2295. [find](https://scholar.google.com/scholar?q=Haoyuan+Gao%2C+Junhua+Mao%2C+Jie+Zhou%2C+Zhiheng+Huang%2C+Lei+Wang%2C+and+Wei+Xu%2E+2015%2E+Are+you+talking+to+a+machine%3F+dataset+and+methods+for+multilingual+image+question+answering%2E+Advances+in+Neural+Informatio)
11. Iacer Calixto, Desmond Elliott, and Stella Frank. 2016. Dcu-uva multimodal mt system report. In Proceedings of the First Conference on Machine Translation, Berlin, Germany. [find](https://scholar.google.com/scholar?q=Iacer+Calixto%2C+Desmond+Elliott%2C+and+Stella+Frank%2E+2016%2E+Dcu%2Duva+multimodal+mt+system+report%2E+In+Proceedings+of+the+First+Conference+on+Machine+Translation%2C+Berlin%2C+Germany%2E)
12. Michael Grubinger, Paul D. Clough, Henning Muller, and Thomas Desealers. 2006. The IAPR TC-12 benchmark: A new evaluation resource for visual information systems. In Proceedings of the Language Resources and Evaluation Conference. [find](https://scholar.google.com/scholar?q=Michael+Grubinger%2C+Paul+D%2E+Clough%2C+Henning+Muller%2C+and+Thomas+Desealers%2E+2006%2E+The+IAPR+TC%2D12+benchmark%3A+A+new+evaluation+resource+for+visual+information+systems%2E+In+Proceedings+of+the+Language+Resour)
13. Jonathan H. Clark, Chris Dyer, Alon Lavie, and Noah A. Smith. 2011. Better hypothesis testing for statistical machine translation: Controlling for optimizer instability. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies: Short Pap… [find](https://scholar.google.com/scholar?q=Jonathan+H%2E+Clark%2C+Chris+Dyer%2C+Alon+Lavie%2C+and+Noah+A%2E+Smith%2E+2011%2E+Better+hypothesis+testing+for+statistical+machine+translation%3A+Controlling+for+optimizer+instability%2E+In+Proceedings+of+the+49th+Ann)
14. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2015. Deep residual learning for image recognition. CoRR, abs/1512.03385. [find](https://scholar.google.com/scholar?q=Kaiming+He%2C+Xiangyu+Zhang%2C+Shaoqing+Ren%2C+and+Jian+Sun%2E+2015%2E+Deep+residual+learning+for+image+recognition%2E+CoRR%2C+abs%2F1512%2E03385%2E)
15. Julian Hitschler, Shigehiko Schamoni, and Stefan Riezler. 2016. Multimodal Pivots for Image Caption Translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany. [find](https://scholar.google.com/scholar?q=Julian+Hitschler%2C+Shigehiko+Schamoni%2C+and+Stefan+Riezler%2E+2016%2E+Multimodal+Pivots+for+Image+Caption+Translation%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Linguist)
16. Michael Denkowski and Alon Lavie. 2014. Meteor universal: Language specific translation evaluation for any target language. In Proceedings of the Ninth Workshop on Statistical Machine Translation, pages 376–380, Baltimore, Maryland. [find](https://scholar.google.com/scholar?q=Michael+Denkowski+and+Alon+Lavie%2E+2014%2E+Meteor+universal%3A+Language+specific+translation+evaluation+for+any+target+language%2E+In+Proceedings+of+the+Ninth+Workshop+on+Statistical+Machine+Translation%2C+pag) [our card](https://md.research.elamur.ai/free/en/papers/denkowski-2014-meteor-universal-language-specific-translation/)
17. Chris Hokamp and Iacer Calixto. 2016. Multimodal neural machine translation using minimum risk training. [https://www.github.com/c](https://www.github.com/c) hrishokamp/multimodal_nmt. [↗](https://www.github.com/c)
18. Karen Simonyan and Andrew Zisserman. 2015. Very deep convolutional networks for large-scale image recognition. In Proceedings of the International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Karen+Simonyan+and+Andrew+Zisserman%2E+2015%2E+Very+deep+convolutional+networks+for+large%2Dscale+image+recognition%2E+In+Proceedings+of+the+International+Conference+on+Learning+Representations%2E) [our card](https://md.research.elamur.ai/free/en/papers/simonyan-2014-very-deep-convolutional-networks-large/)
19. Po-Yao Huang, Frederick Liu, Sz-Rung Shiang, Jean Oh, and Chris Dyer. 2016. Attention-based multimodal neural machine translation. In Proceedings of the First Conference on Machine Translation, Berlin, Germany. [find](https://scholar.google.com/scholar?q=Po%2DYao+Huang%2C+Frederick+Liu%2C+Sz%2DRung+Shiang%2C+Jean+Oh%2C+and+Chris+Dyer%2E+2016%2E+Attention%2Dbased+multimodal+neural+machine+translation%2E+In+Proceedings+of+the+First+Conference+on+Machine+Translation%2C+Berlin) [our card](https://md.research.elamur.ai/free/en/papers/huang-2016-attention-based-multimodal-neural-machine/)
20. Matthew Snover, Bonnie Dorr, and Richard Schwartz. 2006. A study of translation edit rate with targeted human annotation. In Proceedings of Association for Machine Translation in the Americas, Cambridge, Massachusetts. [find](https://scholar.google.com/scholar?q=Matthew+Snover%2C+Bonnie+Dorr%2C+and+Richard+Schwartz%2E+2006%2E+A+study+of+translation+edit+rate+with+targeted+human+annotation%2E+In+Proceedings+of+Association+for+Machine+Translation+in+the+Americas%2C+Cambrid) [our card](https://md.research.elamur.ai/free/en/papers/snover-2006-study-translation-edit-rate-targeted/)
21. Karen Spärck Jones. 1972. A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation, 28:11–21. [find](https://scholar.google.com/scholar?q=Karen+Spa%CC%88rck+Jones%2E+1972%2E+A+statistical+interpretation+of+term+specificity+and+its+application+in+retrieval%2E+Journal+of+Documentation%2C+28%3A11%E2%80%9321%2E)
22. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, et al. 2007. Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual meeting of Association for Comp… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+et+al%2E+2007%2E+Moses%3A+Open+source+toolkit+fo)
23. Darlene Stewart, Roland Kuhn, Eric Joanis, and George Foster. 2014. Coarse split and lump bilingual language models for richer source information in SMT. In Proceedings of the Eleventh Conference of the Association for Machine Translation in the Americas, pages 28–41, Vancouver, Canada. [find](https://scholar.google.com/scholar?q=Darlene+Stewart%2C+Roland+Kuhn%2C+Eric+Joanis%2C+and+George+Foster%2E+2014%2E+Coarse+split+and+lump+bilingual+language+models+for+richer+source+information+in+SMT%2E+In+Proceedings+of+the+Eleventh+Conference+of+t)
24. Jindřich Libovický, Jindřich Helcl, Marek Tlustý, Ondřej Bojar, and Pavel Pecina. 2016. Cuni system for wmt16 automatic post-editing and multimodal translation tasks. In Proceedings of the First Conference on Machine Translation, Berlin, Germany. [find](https://scholar.google.com/scholar?q=Jindr%CC%8Cich+Libovicky%CC%81%2C+Jindr%CC%8Cich+Helcl%2C+Marek+Tlusty%CC%81%2C+Ondr%CC%8Cej+Bojar%2C+and+Pavel+Pecina%2E+2016%2E+Cuni+system+for+wmt16+automatic+post%2Dediting+and+multimodal+translation+tasks%2E+In+Proceedings+of+the+First+)
25. Oriol Vinyals, Alexander Toshev, Samy Bengio, and Dumitru Erhan. 2015. Show and tell: A neural image caption generator. In The IEEE Conference on Computer Vision and Pattern Recognition. [find](https://scholar.google.com/scholar?q=Oriol+Vinyals%2C+Alexander+Toshev%2C+Samy+Bengio%2C+and+Dumitru+Erhan%2E+2015%2E+Show+and+tell%3A+A+neural+image+caption+generator%2E+In+The+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition%2E) [our card](https://md.research.elamur.ai/free/en/papers/vinyals-2014-show-tell-neural-image-caption/)
26. Zichao Yang, Xiaodong He, Jianfeng Gao, Li Deng, and Alexander J. Smola. 2015. Stacked attention networks for image question answering. CoRR, abs/1511.02274. [find](https://scholar.google.com/scholar?q=Zichao+Yang%2C+Xiaodong+He%2C+Jianfeng+Gao%2C+Li+Deng%2C+and+Alexander+J%2E+Smola%2E+2015%2E+Stacked+attention+networks+for+image+question+answering%2E+CoRR%2C+abs%2F1511%2E02274%2E)
27. Kishore Papineni, Salim Roukos, Todd Ard, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ard%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+Annual+Meeting+of+the+Association+for+Compu) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
28. Peter Young, Alice Lai, Micha Hodosh, and Julia Hockenmaier. 2014. From image descriptions to visual denotations: New similarity metrics for semantic inference over event descriptions. Transactions of the Association for Computational Linguistics, 2:67–78. [find](https://scholar.google.com/scholar?q=Peter+Young%2C+Alice+Lai%2C+Micha+Hodosh%2C+and+Julia+Hockenmaier%2E+2014%2E+From+image+descriptions+to+visual+denotations%3A+New+similarity+metrics+for+semantic+inference+over+event+descriptions%2E+Transactions+of)
29. Arnau Ramisa, Fei Yan, Francesc Moreno-Noguer, and Krystian Mikolajczyk. 2016. Breakingnews: Article annotation by image and text processing. CoRR, abs/1603.07141. — [arXiv:1603.07141](https://arxiv.org/abs/1603.07141) [find](https://scholar.google.com/scholar?q=Arnau+Ramisa%2C+Fei+Yan%2C+Francesc+Moreno%2DNoguer%2C+and+Krystian+Mikolajczyk%2E+2016%2E+Breakingnews%3A+Article+annotation+by+image+and+text+processing%2E+CoRR%2C+abs%2F1603%2E07141%2E+%E2%80%94+%5BarXiv%3A1603%2E07141%5D%28https%3A%2F%2Farxiv%2Eo)
30. Sergio Rodrı́guez Guasch and Marta R. Costa-jussà. 2016. [find](https://scholar.google.com/scholar?q=Sergio+Rodr%C4%B1%CC%81guez+Guasch+and+Marta+R%2E+Costa%2Djussa%CC%80%2E+2016%2E)
31. Wmt 2016 multimodal translation system description based on bidirectional recurrent neural networks with double-embeddings. In Proceedings of the First Conference on Machine Translation, Berlin, Germany. [find](https://scholar.google.com/scholar?q=Wmt+2016+multimodal+translation+system+description+based+on+bidirectional+recurrent+neural+networks+with+double%2Dembeddings%2E+In+Proceedings+of+the+First+Conference+on+Machine+Translation%2C+Berlin%2C+Germa)
32. Olga Russakovsky, Jia Deng, Hao Su, Jonathan Krause, Sanjeev Satheesh, Sean Ma, Zhiheng Huang, Andrej Karpathy, Aditya Khosla, Michael S. Bernstein, Alexander C. Berg, and Fei-Fei Li. 2014. Imagenet large scale visual recognition challenge. CoRR, abs/1409.0575. [find](https://scholar.google.com/scholar?q=Olga+Russakovsky%2C+Jia+Deng%2C+Hao+Su%2C+Jonathan+Krause%2C+Sanjeev+Satheesh%2C+Sean+Ma%2C+Zhiheng+Huang%2C+Andrej+Karpathy%2C+Aditya+Khosla%2C+Michael+S%2E+Bernstein%2C+Alexander+C%2E+Berg%2C+and+Fei%2DFei+Li%2E+2014%2E+Imagenet+l) [our card](https://md.research.elamur.ai/free/en/papers/russakovsky-2014-imagenet-large-scale-visual-recognition/)
33. Kashif Shah, Josiah Wang, and Lucia Specia. 2016. Shef-multimodal: Grounding machine translation on images. In Proceedings of the First Conference on Machine Translation, Berlin, Germany. [find](https://scholar.google.com/scholar?q=Kashif+Shah%2C+Josiah+Wang%2C+and+Lucia+Specia%2E+2016%2E+Shef%2Dmultimodal%3A+Grounding+machine+translation+on+images%2E+In+Proceedings+of+the+First+Conference+on+Machine+Translation%2C+Berlin%2C+Germany%2E)
34. Shiqi Shen, Yong Cheng, Zhongjun He, Wei He, Hua Wu, Maosong Sun, and Yang Liu. 2016. Minimum Risk Training for Neural Machine Translation. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Germany. [find](https://scholar.google.com/scholar?q=Shiqi+Shen%2C+Yong+Cheng%2C+Zhongjun+He%2C+Wei+He%2C+Hua+Wu%2C+Maosong+Sun%2C+and+Yang+Liu%2E+2016%2E+Minimum+Risk+Training+for+Neural+Machine+Translation%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association) [our card](https://md.research.elamur.ai/free/en/papers/shen-2015-minimum-risk-training-neural-machine/)