--- title: "OpenSubtitles2016 — Lison 2016" description: "The paper introduces OpenSubtitles2016, a much larger and cleaner version of the OpenSubtitles parallel corpus collection." authors: ["Pierre Lison", "Jörg Tiedemann"] year: 2016 venue: "Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)" source: "https://aclanthology.org/L16-1147/" licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)" licence_url: "https://creativecommons.org/licenses/by-nc/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/lison-2016-opensubtitles2016-extracting-large-parallel-corpora/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lison-2016-opensubtitles2016-extracting-large-parallel-corpora/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/3/) / OpenSubtitles2016: Extracting Large Parallel Corpora from Movie and TV Subtitles # OpenSubtitles2016: Extracting Large Parallel Corpora from Movie and TV Subtitles **Authors:** [Pierre Lison](https://md.research.elamur.ai/free/en/authors/pierre-lison/), [Jörg Tiedemann](https://md.research.elamur.ai/free/en/authors/jorg-tiedemann/) · **Year:** 2016 · **Venue:** Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/) **Links:** [Publisher's page](https://aclanthology.org/L16-1147/) · [PDF](https://aclanthology.org/L16-1147.pdf) · [OpenAlex](https://openalex.org/W2419539795) **Citations:** 656 citations (openalex-snapshot · 2026-10-14) · [441](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary The paper introduces OpenSubtitles2016, a much larger and cleaner version of the OpenSubtitles parallel corpus collection. The authors built it from a dump of the OpenSubtitles.org database, about 3.36 million subtitle files in more than 60 languages. Their pipeline has several stages: - It detects each file's character encoding using per-language candidate lists and the chardet library. - It turns timed subtitle blocks into tokenised sentences with language-specific rules. - It fixes OCR mistakes and misplaced accents using a noisy-channel model with a bigram language model. - It stores each subtitle as an XML file. Subtitles for the same movie or episode are then aligned across languages by how much their timings overlap. When several alternative subtitles exist, metadata-based quality scores choose the best pair. Alternative subtitles in the same language are also aligned with each other. The result is 1689 bitexts covering 60 languages. The data grew about 39% over the previous release, and Moses SMT systems trained on it gain BLEU for nearly all tested pairs. This makes it one of the largest freely available parallel resources for conversational language (on OPUS). ## Key points - **Scale:** 2.6 billion sentences (17.2 billion tokens) from about 2.8M converted subtitles, covering 152,939 distinct movies and TV episodes by IMDb identifier. New languages are Breton, Esperanto, Georgian and Tagalog. - **Largest bitext:** English–Spanish, with about 62.2K aligned documents and 49.2M sentence pairs. - **OCR and accent correction:** applied to 11 languages, fixing 9.04 million words, about 3.2 corrections per subtitle on average. - **Extrinsic evaluation:** Moses phrase-based SMT with MERT tuning on 18 language pairs. The 2016 data mostly beats the 2012+2013 release, for example 35.49 → 39.15 BLEU on the top pair. - **Ukrainian:** relatively small, with about 1.0K subtitle files, 0.9K covered IMDbs, 0.8M sentences and 5.4M tokens. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) Topics: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) ## Cited by Cited by 441 papers of our library cite it (of the 140673 whose bibliographies we read); 38 of them have a page here, 403 not yet. - [Pre-Training BERT on Arabic Tweets: Practical Considerations](https://md.research.elamur.ai/free/en/papers/abdelali-2021-pre-training-bert-arabic-tweets/) - [FINDINGS OF THE IWSLT 2023 EVALUATION CAMPAIGN](https://md.research.elamur.ai/free/en/papers/agarwal-2023-findings-iwslt-2023-evaluation-campaign/) - [JW300: A Wide-Coverage Parallel Corpus for Low-Resource Languages](https://md.research.elamur.ai/free/en/papers/agic-2019-jw300-wide-coverage-parallel-corpus/) - [The Arabic Parallel Gender Corpus 2.0: Extensions and Analyses](https://md.research.elamur.ai/free/en/papers/alhafni-2022-arabic-parallel-gender-corpus-2/) - [The User-Aware Arabic Gender Rewriter](https://md.research.elamur.ai/free/en/papers/alhafni-2023-user-aware-arabic-gender-rewriter/) - [Findings of the Third Shared Task on Multimodal Machine Translation](https://md.research.elamur.ai/free/en/papers/barrault-2018-findings-third-shared-task-multimodal/) - [Evaluating Discourse Phenomena in Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/bawden-2017-evaluating-discourse-phenomena-neural-machine/) - [GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://md.research.elamur.ai/free/en/papers/black-2022-gpt-neox-20b-open-source/) - [Evaluating the Evaluation Metrics for Style Transfer: A Case Study in Multilingual Formality Transfer](https://md.research.elamur.ai/free/en/papers/briakou-2021-evaluating-evaluation-metrics-style-transfer/) - [Effective Domain Mixing for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/britz-2017-effective-domain-mixing-neural-machine/) - [Findings of the WMT 2020 Shared Task on Chat Translation](https://md.research.elamur.ai/free/en/papers/farajian-2020-findings-wmt-2020-shared-task/) - [Expectations over Unspoken Alternatives Predict Pragmatic Inferences](https://md.research.elamur.ai/free/en/papers/hu-2023-expectations-over-unspoken-alternatives-predict/) - [BlonDe: An Automatic Evaluation Metric for Document-level Machine Translation](https://md.research.elamur.ai/free/en/papers/jiang-2022-blonde-automatic-evaluation-metric-document/) - [Harnessing Sequence Labeling for Sarcasm Detection in Dialogue from TV Series ‘Friends'](https://md.research.elamur.ai/free/en/papers/joshi-2016-harnessing-sequence-labeling-sarcasm-detection/) - [On the Impact of Various Types of Noise on Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/khayrallah-2018-impact-various-types-noise-neural/) - [An Arabic-Hebrew parallel corpus of TED talks](https://md.research.elamur.ai/free/en/papers/langstream-2016-cettolo-arabic-hebrew-parallel-corpus-ted/) - [Arabic Sentence Segmentation Across Genres and Punctuation Conditions](https://md.research.elamur.ai/free/en/papers/langstream-2026-elkholy-arabic-sentence-segmentation-across-genres/) - [The BigScience ROOTS Corpus: A 1.6TB Composite Multilingual Dataset](https://md.research.elamur.ai/free/en/papers/lauren-2023-ccon-bigscience-roots-corpus-1-6tb/) - [OpenSubtitles2018: Statistical Rescoring of Sentence Alignments in Large, Noisy Parallel Corpora](https://md.research.elamur.ai/free/en/papers/lison-2018-opensubtitles2018-statistical-rescoring-sentence-alignments/) - [Document-Level Neural Machine Translation with Hierarchical Attention Networks](https://md.research.elamur.ai/free/en/papers/miculicich-2018-document-level-neural-machine-translation/) - [Domain Robustness in Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/muller-2020-domain-robustness-neural-machine-translation/) - [Large-scale text pre-training helps with dialogue act recognition, but not without fine-tuning](https://md.research.elamur.ai/free/en/papers/noble-2021-large-scale-text-pre-training/) - [Fixing Translation Divergences in Parallel Corpora for Neural MT](https://md.research.elamur.ai/free/en/papers/pham-2018-fixing-translation-divergences-parallel-corpora/) - [Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/post-2018-fast-lexically-constrained-decoding-dynamic/) - [EgyBERT: A Large Language Model Pretrained on Egyptian Dialect Corpora](https://md.research.elamur.ai/free/en/papers/qarah-2024-egybert-large-language-model-pretrained/) - [SaudiBERT: A Large Language Model Pretrained on Saudi Dialect Corpora](https://md.research.elamur.ai/free/en/papers/qarah-2024-saudibert-large-language-model-pretrained/) - [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://md.research.elamur.ai/free/en/papers/reimers-2020-making-monolingual-sentence-embeddings-multilingual/) - [Bifixer and Bicleaner: two open-source tools to clean your parallel data](https://md.research.elamur.ai/free/en/papers/sanchez-2020-bifixer-bicleaner-two-open-source/) - [Gender Bias in Machine Translation](https://md.research.elamur.ai/free/en/papers/savoldi-2021-gender-bias-machine-translation/) - [Cross-Lingual Alignment of Contextual Word Embeddings, with Applications to Zero-shot Dependency Parsing](https://md.research.elamur.ai/free/en/papers/schuster-2019-cross-lingual-alignment-contextual-word/) - [CCMatrix: Mining Billions of High-Quality Parallel Sentences on the WEB](https://md.research.elamur.ai/free/en/papers/schwenk-2019-ccmatrix-mining-billions-high-quality/) - [WikiMatrix: Mining 135M Parallel Sentences in 1620 Language Pairs from Wikipedia](https://md.research.elamur.ai/free/en/papers/schwenk-2019-wikimatrix-mining-135m-parallel-sentences/) - [Rebelòt: Datasets and Token-Level Language Identification for Lombard-Italian-English Code-Mixing](https://md.research.elamur.ai/free/en/papers/signoroni-2026-rebelot-datasets-token-level-language/) - [OpenSubtitles2024: A Massively Parallel Dataset of Movie Subtitles for MT Development and Evaluation](https://md.research.elamur.ai/free/en/papers/tiedemann-2026-opensubtitles2024-massively-parallel-dataset-movie/) - [Leveraging Rule-Based Machine Translation Knowledge for Under-Resourced Neural Machine Translation Models](https://md.research.elamur.ai/free/en/papers/torregrosa-2019-leveraging-rule-based-machine-translation/) - [Quality and Appropriateness of Large Text Datasets for Irish NLP](https://md.research.elamur.ai/free/en/papers/walsh-2026-quality-appropriateness-large-text-datasets/) - [On Exposure Bias, Hallucination and Domain Shift in Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/wang-2020-exposure-bias-hallucination-domain-shift/) - [IndoNLU: Benchmark and Resources for Evaluating Indonesian Natural Language Understanding](https://md.research.elamur.ai/free/en/papers/wilie-2020-indonlu-benchmark-resources-evaluating-indonesian/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{lison2016opensubtitles2016, title = {{OpenSubtitles2016: Extracting Large Parallel Corpora from Movie and TV Subtitles}}, author = {Lison, Pierre and Tiedemann, Jörg}, booktitle = {Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)}, year = {2016}, doi = {10.63317/3fi26b3nobqg}, url = {https://doi.org/10.63317/3fi26b3nobqg} } ``` - [← previous Neural versus Phrase-Based Machine Translation Quality: a Case Study](https://md.research.elamur.ai/free/en/papers/bentivogli-2016-neural-versus-phrase-based-machine/) - [next → OPT: Oslo–Potsdam–Teesside. Pipelining Rules, Rankers, and Classifier Ensembles for Shallow Discourse Parsing](https://md.research.elamur.ai/free/en/papers/oepen-2016-opt-oslo-potsdam-teesside-pipelining/) - [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/3/) ## References 12 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Aziz, W., de Sousa, S. C. M., and Specia, L. (2012). Cross-lingual sentence compression for subtitles. In 16th Annual Conference of the European Association for Machine Translation (EAMT 2012), pages 103–110, Trento, Italy. [find](https://scholar.google.com/scholar?q=Aziz%2C+W%2E%2C+de+Sousa%2C+S%2E+C%2E+M%2E%2C+and+Specia%2C+L%2E+%282012%29%2E+Cross%2Dlingual+sentence+compression+for+subtitles%2E+In+16th+Annual+Conference+of+the+European+Association+for+Machine+Translation+%28EAMT+2012%29%2C+pages+) 2. Brants, T. and Franz, A. (2006). Web 1T 5-gram corpus version 1. Technical report, Google Research. [find](https://scholar.google.com/scholar?q=Brants%2C+T%2E+and+Franz%2C+A%2E+%282006%29%2E+Web+1T+5%2Dgram+corpus+version+1%2E+Technical+report%2C+Google+Research%2E) 3. Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., Cowan, B., Shen, W., Moran, C., Zens, R., Dyer, C. J., Bojar, O., Constantin, A., and Herbst, E. (2007). Moses: Open source toolkit for statistical machine translation. In Proceedings of the 45th Annual Meeting of the A… [find](https://scholar.google.com/scholar?q=Koehn%2C+P%2E%2C+Hoang%2C+H%2E%2C+Birch%2C+A%2E%2C+Callison%2DBurch%2C+C%2E%2C+Federico%2C+M%2E%2C+Bertoldi%2C+N%2E%2C+Cowan%2C+B%2E%2C+Shen%2C+W%2E%2C+Moran%2C+C%2E%2C+Zens%2C+R%2E%2C+Dyer%2C+C%2E+J%2E%2C+Bojar%2C+O%2E%2C+Constantin%2C+A%2E%2C+and+Herbst%2C+E%2E+%282007%29%2E+Moses%3A+Open+so) 4. Li, S. and Momoi, K. (2001). A composite approach to language/encoding detection. In 19th International Unicode Conference (IUC 2001), San Jose, California. [find](https://scholar.google.com/scholar?q=Li%2C+S%2E+and+Momoi%2C+K%2E+%282001%29%2E+A+composite+approach+to+language%2Fencoding+detection%2E+In+19th+International+Unicode+Conference+%28IUC+2001%29%2C+San+Jose%2C+California%2E) 5. Lui, M. and Baldwin, T. (2012). Langid.py: An off-the-shelf language identification tool. In Proceedings of the 50th Annual Meeting of the Association for Computational Linguistics (ACL 2012 System Demonstrations), pages 25–30, Jeju Island, Korea. [find](https://scholar.google.com/scholar?q=Lui%2C+M%2E+and+Baldwin%2C+T%2E+%282012%29%2E+Langid%2Epy%3A+An+off%2Dthe%2Dshelf+language+identification+tool%2E+In+Proceedings+of+the+50th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28ACL+2012+System+De) [our card](https://md.research.elamur.ai/free/en/papers/lui-2012-langid-py-off-shelf-language/) 6. Neubig, G., Nakata, Y., and Mori, S. (2011). Pointwise prediction for robust, adaptable japanese morphological analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics (ACL 2011), pages 529–533, Portland, Oregon, USA. [find](https://scholar.google.com/scholar?q=Neubig%2C+G%2E%2C+Nakata%2C+Y%2E%2C+and+Mori%2C+S%2E+%282011%29%2E+Pointwise+prediction+for+robust%2C+adaptable+japanese+morphological+analysis%2E+In+Proceedings+of+the+49th+Annual+Meeting+of+the+Association+for+Computational+) [our card](https://md.research.elamur.ai/free/en/papers/neubig-2011-pointwise-prediction-robust-adaptable-japanese/) 7. Och, F. J. (2003). Minimum Error Rate Training in Statistical Machine Translation. In Proceedings of the 41st Annual Meeting on Association for Computational Linguistics (ACL 2003), pages 160–167, Sapporo, Japan. [find](https://scholar.google.com/scholar?q=Och%2C+F%2E+J%2E+%282003%29%2E+Minimum+Error+Rate+Training+in+Statistical+Machine+Translation%2E+In+Proceedings+of+the+41st+Annual+Meeting+on+Association+for+Computational+Linguistics+%28ACL+2003%29%2C+pages+160%E2%80%93167%2C+Sap) 8. Tiedemann, J. (2007). Improved sentence alignment for movie subtitles. In Proceedings of the Conference on Recent Advances in Natural Language Processing (RANLP 2007), Borovets, Bulgaria. [find](https://scholar.google.com/scholar?q=Tiedemann%2C+J%2E+%282007%29%2E+Improved+sentence+alignment+for+movie+subtitles%2E+In+Proceedings+of+the+Conference+on+Recent+Advances+in+Natural+Language+Processing+%28RANLP+2007%29%2C+Borovets%2C+Bulgaria%2E) 9. Tiedemann, J. (2008). Synchronizing translated movie subtitles. In Proceedings of the 6th International Conference on Language Resources and Evaluation (LREC 2008), pages 1902–1906, Marrakesh, Marocco. [find](https://scholar.google.com/scholar?q=Tiedemann%2C+J%2E+%282008%29%2E+Synchronizing+translated+movie+subtitles%2E+In+Proceedings+of+the+6th+International+Conference+on+Language+Resources+and+Evaluation+%28LREC+2008%29%2C+pages+1902%E2%80%931906%2C+Marrakesh%2C+Marocco) 10. Tiedemann, J. (2012). Parallel Data, Tools and Interfaces in OPUS. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC 2012), pages 2214–2218, Istanbul, Turkey. [find](https://scholar.google.com/scholar?q=Tiedemann%2C+J%2E+%282012%29%2E+Parallel+Data%2C+Tools+and+Interfaces+in+OPUS%2E+In+Proceedings+of+the+Eight+International+Conference+on+Language+Resources+and+Evaluation+%28LREC+2012%29%2C+pages+2214%E2%80%932218%2C+Istanbul%2C+Tur) 11. Tiedemann, J. (2016). Finding alternative translations in a large corpus of movie subtitles. In Proceedings of the 10th International Conference on Language Resources and Evaluation (LREC 2016), Portorož, Slovenia. [find](https://scholar.google.com/scholar?q=Tiedemann%2C+J%2E+%282016%29%2E+Finding+alternative+translations+in+a+large+corpus+of+movie+subtitles%2E+In+Proceedings+of+the+10th+International+Conference+on+Language+Resources+and+Evaluation+%28LREC+2016%29%2C+Porto) 12. Zhang, S., Ling, W., and Dyer, C. (2014). Dual subtitles as parallel corpora. In Proceedings of the 9th International Conference on Language Resources and Evaluation (LREC 2014), pages 1869–1874, Reykjavik, Iceland. [find](https://scholar.google.com/scholar?q=Zhang%2C+S%2E%2C+Ling%2C+W%2E%2C+and+Dyer%2C+C%2E+%282014%29%2E+Dual+subtitles+as+parallel+corpora%2E+In+Proceedings+of+the+9th+International+Conference+on+Language+Resources+and+Evaluation+%28LREC+2014%29%2C+pages+1869%E2%80%931874%2C+Reyk)