--- title: "Extension Multi30K — Saichyshyna 2023" description: "Authors from Kharkiv National University of Radio Electronics built a Ukrainian version of the multimodal corpus Multi30K." authors: ["Nataliia Saichyshyna", "Daniil Maksymenko", "Oleksii Turuta", "Andriy Yerokhin", "Andrii Babii", "Olena Turuta"] year: 2023 venue: "Proceedings of the Second Ukrainian Natural Language Processing Workshop (UNLP)" source: "https://aclanthology.org/2023.unlp-1.7/" doi: "10.18653/v1/2023.unlp-1.7" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/papers/saichyshyna-2023-extension-multi30k-multimodal-dataset-integrated/" --- [Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/saichyshyna-2023-extension-multi30k-multimodal-dataset-integrated/) [home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2023](https://md.research.elamur.ai/free/papers/2023/page/2/) / Extension Multi30K: Multimodal Dataset for Integrated Vision and Language Research in Ukrainian # Extension Multi30K: Multimodal Dataset for Integrated Vision and Language Research in Ukrainian **Authors:** Nataliia Saichyshyna, Daniil Maksymenko, Oleksii Turuta, Andriy Yerokhin, Andrii Babii, Olena Turuta · **Year:** 2023 · **Venue:** Proceedings of the Second Ukrainian Natural Language Processing Workshop (UNLP) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2023.unlp-1.7/) · [PDF](https://aclanthology.org/2023.unlp-1.7.pdf) · [DOI 10.18653/v1/2023.unlp-1.7](https://doi.org/10.18653/v1/2023.unlp-1.7) · [OpenAlex](https://openalex.org/W4386566354) **Citations:** 6 citations (openalex-snapshot · 2026-10-14) · 9 in our library, of the 140673 papers whose bibliographies we read ## Summary Authors from Kharkiv National University of Radio Electronics built a Ukrainian version of the multimodal corpus Multi30K. The corpus consists of images from Flickr30K and their text descriptions. The work aims to give Ukrainian, a low-resource language, high-quality parallel data for machine translation and for tasks that combine vision and language. The English descriptions were first translated automatically with Google Cloud Translator. A team of eight students and lecturers then proofread the translations, checking them against both the English original and the image itself. About half of the sentences needed corrections. The corpus was used in three directions. The first was fine-tuning MarianMT for translation from English. The second was testing multilingual semantic image search based on CLIP. The third was evaluating translation quality without a reference text, using cosine similarity of embeddings. The text model aligned with visual embeddings handled idioms and slang noticeably better than a text-only model. The work's value lies in an open Ukrainian multimodal resource. It also shows what existing multilingual models can do for Ukrainian and where they make mistakes. ## Key points - **Corpus size:** 31 014 sentences. Token counts: English 357 172, German 333 833, Ukrainian 276 520. There are fewer Ukrainian tokens partly because the language has no articles. About 51% of the machine translations were corrected by hand. The corpus is open: Hugging Face `turuta/Multi30k-uk` and GitHub `researchlabs/multi30k-uk`. - **Quality check:** cosine similarity was computed with the distiluse-base-multilingual-cased-v2 model on a sample of 5000 pairs. Only 3 pairs scored below 0.4. After the manual corrections, the share of pairs with high similarity increased. - **Machine translation:** MarianMT (Helsinki NLP, OPUS) was fine-tuned on Multi30K together with other collected datasets. On 5000 TATOEBA texts, the model achieved METEOR 0.3810 and BERTScore F1 0.9232. Multi30K on its own did not substantially improve quality, but it improved generalization. - **Semantic search** (CLIP + DistilBERT, `clip-ViT-B-32-multilingual-v1`): the correct image was found for 44.5% of English descriptions and only 29.55% of Ukrainian ones. The model recognizes objects well but often confuses actions and background details. - **Reference-free translation evaluation:** the model aligned with CLIP gave no pair a score below 0.5. For "walkie talkie" → «рація», the score rose from 0.32 to 0.7456. For idioms such as "Keep the change", it rose from about 0.25 to 0.95. The authors stress that this approach still needs to be validated. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) Topics: [Machine translation](https://md.research.elamur.ai/free/topics/machine-translation/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{saichyshyna2023extension, title = {{Extension Multi30K: Multimodal Dataset for Integrated Vision and Language Research in Ukrainian}}, author = {Saichyshyna, Nataliia and Maksymenko, Daniil and Turuta, Oleksii and Yerokhin, Andriy and Babii, Andrii and Turuta, Olena}, booktitle = {Proceedings of the Second Ukrainian Natural Language Processing Workshop (UNLP)}, year = {2023}, doi = {10.18653/v1/2023.unlp-1.7}, url = {https://doi.org/10.18653/v1/2023.unlp-1.7} } ``` - [← previous Exploring the Relationship between LLM Hallucinations and Prompt Linguistic Nuances: Readability, Formality, and Concreteness](https://md.research.elamur.ai/free/papers/rawte-2023-exploring-relationship-between-llm-hallucinations/) - [next → FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios](https://md.research.elamur.ai/free/papers/chern-2023-factool-factuality-detection-generative-ai/) - [↑ back to papers of 2023](https://md.research.elamur.ai/free/papers/2023/page/2/) ## References 3 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Loïc Barrault, Fethi Bougares, Lucia Specia, Chiraag Lala, Desmond Elliott, and Stella Frank. 2018. Findings of the third shared task on multimodal machine translation. In Proceedings of the Third Conference on Machine Translation: Shared Task Papers, pages 304–323, Belgium, Brussels. Association fo… [find](https://scholar.google.com/scholar?q=Lo%C3%AFc+Barrault%2C+Fethi+Bougares%2C+Lucia+Specia%2C+Chiraag+Lala%2C+Desmond+Elliott%2C+and+Stella+Frank%2E+2018%2E+Findings+of+the+third+shared+task+on+multimodal+machine+translation%2E+In+Proceedings+of+the+Third+Con) [our card](https://md.research.elamur.ai/free/papers/barrault-2018-findings-third-shared-task-multimodal/) 2. Begum Citamak, Ozan Caglayan, Menekse Kuyu, Erkut Erdem, Aykut Erdem, Pranava Madhyastha, and Lucia Specia. 2020. Msvd-turkish: A comprehensive multimodal dataset for integrated vision and language research in turkish. [find](https://scholar.google.com/scholar?q=Begum+Citamak%2C+Ozan+Caglayan%2C+Menekse+Kuyu%2C+Erkut+Erdem%2C+Aykut+Erdem%2C+Pranava+Madhyastha%2C+and+Lucia+Specia%2E+2020%2E+Msvd%2Dturkish%3A+A+comprehensive+multimodal+dataset+for+integrated+vision+and+language+re) 3. Desmond Elliott, Stella Frank, Loïc Barrault, Fethi Bougares, and Lucia Specia. 2017. Findings of the second shared task on multimodal machine translation and multilingual image description. In Proceedings of the Second Conference on Machine Translation, pages 215–233, Copenhagen, Denmark. Associati… [find](https://scholar.google.com/scholar?q=Desmond+Elliott%2C+Stella+Frank%2C+Lo%C3%AFc+Barrault%2C+Fethi+Bougares%2C+and+Lucia+Specia%2E+2017%2E+Findings+of+the+second+shared+task+on+multimodal+machine+translation+and+multilingual+image+description%2E+In+Procee) [our card](https://md.research.elamur.ai/free/papers/elliott-2017-findings-second-shared-task-multimodal/)