--- title: "Findings of the Second WMT Shared Task on Sign — Müller 2023" description: "WMT-SLT23, the second edition of the WMT shared task on sign language translation, asks systems to translate between signed and spoken languages." authors: ["Mathias Müller", "Malihe Alikhani", "Eleftherios Avramidis", "Richard Bowden", "Annelies Braffort", "Necati Cihan Camgöz", "Sarah Ebling", "Cristina España-Bonet", "Anne Göhring", "Roman Grundkiewicz", "Mert Inan", "Zifan Jiang", "Oscar Koller", "Amit Moryossef", "Annette Rios", "Dimitar Shterionov", "Sandra Sidler-Miserez", "Katja Tissi", "Davy Van Landuyt"] year: 2023 venue: "Proceedings of the Eighth Conference on Machine Translation" source: "https://aclanthology.org/2023.wmt-1.4/" doi: "10.18653/v1/2023.wmt-1.4" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/papers/muller-2023-findings-second-wmt-shared-task/" --- [Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/muller-2023-findings-second-wmt-shared-task/) [home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2023](https://md.research.elamur.ai/free/papers/2023/page/3/) / Findings of the Second WMT Shared Task on Sign Language Translation (WMT-SLT23) # Findings of the Second WMT Shared Task on Sign Language Translation (WMT-SLT23) **Authors:** [Mathias Müller](https://md.research.elamur.ai/free/authors/mathias-muller/), [Malihe Alikhani](https://md.research.elamur.ai/free/authors/malihe-alikhani/), [Eleftherios Avramidis](https://md.research.elamur.ai/free/authors/eleftherios-avramidis/), Richard Bowden, Annelies Braffort, Necati Cihan Camgöz, Sarah Ebling, [Cristina España-Bonet](https://md.research.elamur.ai/free/authors/cristina-espana-bonet/), Anne Göhring, [Roman Grundkiewicz](https://md.research.elamur.ai/free/authors/roman-grundkiewicz/), Mert Inan, Zifan Jiang et al. (19 authors) · **Year:** 2023 · **Venue:** Proceedings of the Eighth Conference on Machine Translation · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2023.wmt-1.4/) · [PDF](https://aclanthology.org/2023.wmt-1.4.pdf) · [DOI 10.18653/v1/2023.wmt-1.4](https://doi.org/10.18653/v1/2023.wmt-1.4) · [OpenAlex](https://openalex.org/W4389519214) **Citations:** 16 citations (openalex-snapshot · 2026-10-14) · [23](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary WMT-SLT23, the second edition of the WMT shared task on sign language translation, asks systems to translate between signed and spoken languages. This requires handling visual input such as video or pose estimates, so the problem goes beyond standard text-to-text MT. The organizers built on the 2022 edition in three ways: they grew from one language pair to three (DSGS, LSF-CH and LIS-CH paired with German, French and Italian), released far more DSGS training data, and placed more weight on spoken-to-signed translation by supplying official baselines for that direction. Four teams took part, one of them building a baseline, and every team entered only the DSGS-to-German track. Both automatic and human evaluation showed very poor quality: the best system averaged about 1/100 in human scores, which is far from practical use. The authors attribute this to three gaps: too little usable training data, the shift in modality from text, and missing basic NLP tools for sign languages. The task still leaves useful resources behind, including new corpora, reproducible baselines, public system outputs and additional human judgments for SLT. ## Key points - The task had four tracks: DSGS→German, German→DSGS, French→LSF-CH and Italian→LIS-CH. No team submitted to any track with a sign language as the target. - DSGS training data rose to 437 hours from 16 the previous year. Unlike 2022, the data was released without manual correction. - The SRF23 training corpus behaves like a noisy comparable corpus. Its German subtitles follow the broadcast audio rather than the delayed live interpretation, and no participant attempted to fix the alignment. - TTIC and CASIA scored higher on the SRF test data than on Signsuisse. In contrast, human reference translations rated poorly on SRF, partly because of bad video cuts and missing discourse context. - For future editions, the organizers propose shared tasks on more basic SLP problems, namely alignment, segmentation and parallel-corpus filtering. They also suggest moving away from interpreted news material. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/shelves/machine-translation/) ## Cited by Cited by 23 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 22 not yet. - [A Critical Study of Automatic Evaluation in Sign Language Translation](https://md.research.elamur.ai/free/papers/yazdani-2026-critical-study-automatic-evaluation-sign/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{muller2023findings, title = {{Findings of the Second WMT Shared Task on Sign Language Translation (WMT-SLT23)}}, author = {Müller, Mathias and Alikhani, Malihe and Avramidis, Eleftherios and Bowden, Richard and Braffort, Annelies and Camgöz, Necati Cihan and Ebling, Sarah and España-Bonet, Cristina and Göhring, Anne and Grundkiewicz, Roman and Inan, Mert and Jiang, Zifan and Koller, Oscar and Moryossef, Amit and Rios, Annette and Shterionov, Dimitar and Sidler-Miserez, Sandra and Tissi, Katja and Landuyt, Davy Van}, booktitle = {Proceedings of the Eighth Conference on Machine Translation}, year = {2023}, doi = {10.18653/v1/2023.wmt-1.4}, url = {https://doi.org/10.18653/v1/2023.wmt-1.4} } ``` - [← previous Findings of the Second Shared Task on Multilingual Coreference Resolution](https://md.research.elamur.ai/free/papers/abokrtsky-2023-findings-second-shared-task-multilingual/) - [next → Findings of the WMT 2023 Shared Task on Low-Resource Indic Language Translation](https://md.research.elamur.ai/free/papers/pal-2023-findings-wmt-2023-shared-task/) - [↑ back to papers of 2023](https://md.research.elamur.ai/free/papers/2023/page/3/) ## References 2 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Alan Agresti. 1996. An introduction to categorical data analysis, volume 135. Wiley New York. [find](https://scholar.google.com/scholar?q=Alan+Agresti%2E+1996%2E+An+introduction+to+categorical+data+analysis%2C+volume+135%2E+Wiley+New+York%2E) 2. Farhad Akhbardeh, Arkady Arkhangorodsky, Magdalena Biesialska, Ondřej Bojar, Rajen Chatterjee, Vishrav Chaudhary, Marta R. Costa-jussa, Cristina España-Bonet, Angela Fan, Christian Federmann, Markus Freitag, Yvette Graham, Roman Grundkiewicz, Barry Haddow, Leonie Harter, Kenneth Heafield, Christoph… [find](https://scholar.google.com/scholar?q=Farhad+Akhbardeh%2C+Arkady+Arkhangorodsky%2C+Magdalena+Biesialska%2C+Ondr%CC%8Cej+Bojar%2C+Rajen+Chatterjee%2C+Vishrav+Chaudhary%2C+Marta+R%2E+Costa%2Djussa%2C+Cristina+Espa%C3%B1a%2DBonet%2C+Angela+Fan%2C+Christian+Federmann%2C+Markus+) [our card](https://md.research.elamur.ai/free/papers/akhbardeh-2021-findings-2021-conference-machine-translation/)