---
title: "A Character-level Convolutional Neural Network for"
description: "The paper tackles automatic identification of closely related languages and dialects, a task that is… 18 references, cited by 15 papers here."
authors: ["Yonatan Belinkov", "James Glass"]
year: 2016
venue: "Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3)"
source: "https://aclanthology.org/W16-4819/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/belinkov-2016-character-level-convolutional-neural-network/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/belinkov-2016-character-level-convolutional-neural-network/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/4/) / A Character-level Convolutional Neural Network for Distinguishing Similar Languages and Dialects

# A Character-level Convolutional Neural Network for Distinguishing Similar Languages and Dialects

**Authors:** [Yonatan Belinkov](https://md.research.elamur.ai/free/en/authors/yonatan-belinkov/) and [James Glass](https://md.research.elamur.ai/free/en/authors/james-glass/) · **Year:** 2016 · **Venue:** [Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3)](https://md.research.elamur.ai/free/en/venues/vardial/) ([VarDial 2016](https://md.research.elamur.ai/free/en/venues/vardial/2016/)) · **Publisher:** [International Committee on Computational Linguistics (ICCL)](https://md.research.elamur.ai/free/en/publishers/international-committee-on-computational-linguistics-iccl/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/W16-4819/) · [PDF](https://aclanthology.org/W16-4819.pdf) · [OpenAlex](https://openalex.org/W2526744500)

**Citations:** 7 citations (OpenAlex, 2026-10-14) · [15](https://md.research.elamur.ai/free/en/papers/belinkov-2016-character-level-convolutional-neural-network/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper tackles automatic identification of closely related languages and dialects, a task that is much harder than separating unrelated languages. Belinkov and Glass entered the DSL 2016 shared task (closed track). Sub-task 1 covered groups such as Bosnian/Croatian/Serbian, Malay/Indonesian, and national varieties of Portuguese, Spanish and French in news text. Sub-task 2 covered Arabic dialects (Levantine, Gulf, Egyptian, North African, MSA) in speech-recognizer transcripts. Their central question was how far a model can get using characters alone, with no notion of words. The system learns character embeddings, runs several convolutions of different widths in parallel, applies max-over-time pooling, and classifies through a hidden layer and a softmax, all trained end to end. On Arabic, a 10-model voting ensemble reached a weighted F1 of 0.4834, ranking 6th of 18. The untuned system ranked near the bottom on sub-task 1. The error analysis is the most valuable part. It shows which dialect cues a character model cannot see, such as word order, morphological patterns and code-mixing, and how ASR transcription hides phonological differences between dialects.

## Key points
- Architecture: trainable character embeddings → dropout → parallel CNN filters (widths 1–7; best set {1∗50, 2∗50, 3∗100, 4∗100, 5∗100, 6∗100, 7∗100}) with ReLU → max-over-time pooling → fully-connected ReLU layer with dropout → softmax. Implemented in Keras with the TensorFlow backend; code is at github.com/boknilev/dsl-char-cnn.
- Training: Adam with default settings, cross-entropy loss, mini-batches of 16, and early stopping with a patience of 10 epochs. Dropout, maximum text length, embedding size and hidden size were tuned on a held-out portion of the Arabic training data.
- Tuning observations:
  - A single filter width of 5 with 250 filters already worked reasonably well.
  - Mixing several widths gave a small gain.
  - Dropping the fully-connected layer slightly hurt performance.
- Arabic test set C results:
  - Run 3, the plurality vote of 10 models trained on different random 90/10 splits, was best: accuracy 0.4851, weighted F1 0.4834.
  - The top system in the task reached a weighted F1 of 0.5132.
  - MSA was the easiest class to detect (F1 0.60); the other dialects scored roughly 0.47–0.50.
- Sub-task 1 test set A: best accuracy was 0.8307, against 0.8938 for the winning system. This system was not tuned for that sub-task and placed second to last.

## Interesting ideas
- When there is no official dev set, train several copies of the model on different random train/dev splits and let them vote; this used all the data and gave the best run, beating both a single split and full-data training with borrowed stopping epoch.
- Running character filters of many widths side by side lets one layer detect affix-sized and stem-sized patterns at the same time, which suits morphology-heavy cues.
- The error analysis sorts misclassifications by the linguistic level needed to fix them, such as word order (syntax), derivational patterns (morphology) and register mixing, which shows exactly where a surface model hits its ceiling.
- To explain a single wrong prediction, the authors check how often the decisive clue word occurred with each label in training (one word appeared five times under one dialect and never under the other).
- The authors suggest a hierarchical decision, first separating the standard variety (MSA) from the dialects and only then picking the specific dialect, a scheme earlier DSL work found helpful.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): When a gold dev set is missing for a new domain, train gparse or tagger variants on rotated splits and use their agreement as an extra witness when building silver data.
- **I2** (tokens): A small multi-width character scorer could back off for unknown words in the en tagger, proposing POS and lemma candidates from suffix and prefix shapes while keeping the rule-based path as the explanation.
- **I3** (learn): Label each failure in our evaluation by the earliest stage that could have prevented it, so that repair rules and negative controls are aimed at the correct layer.
- **I4** (world): Store per-label frequency evidence for compiled knowledge entries, so that when a verb-class or ontology decision goes wrong, the trace shows which training counts drove it.
- **I5** (tree): In UD dialect conversion, first decide which annotation convention or genre a treebank follows, then apply the specific conversion rules for that family.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Massachusetts Institute of Technology](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/) — University, Cambridge, United States · 211 works · 69,000 citations
- [MIT Computer Science and Artificial Intelligence Laboratory](https://md.research.elamur.ai/free/en/labs/mit-computer-science-and-artificial-intelligence-laboratory/) — Research institute, Cambridge, United States · 69 works · 30,953 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

## Cite

Cite as: Yonatan Belinkov and James Glass. 2016. A Character-level Convolutional Neural Network for Distinguishing Similar Languages and Dialects. Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{belinkov2016character,
  title = {{A Character-level Convolutional Neural Network for Distinguishing Similar Languages and Dialects}},
  author = {Yonatan Belinkov and James Glass},
  booktitle = {Proceedings of the Third Workshop on NLP for Similar Languages, Varieties and Dialects (VarDial3)},
  year = {2016},
  url = {https://aclanthology.org/W16-4819/}
}
```

- [← previous LSIS at SemEval-2016 Task 7: Using Web Search Engines for English and Arabic Unsupervised Sentiment Intensity Prediction](https://md.research.elamur.ai/free/en/papers/htait-2016-lsis-semeval-2016-task-7/)
- [next → Moving Beyond the Turing Test with the Allen AI Science Challenge](https://md.research.elamur.ai/free/en/papers/schoenick-2016-moving-turing-test-allen/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/4/)

## References

18 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenberg, Dan Mané, Rajat Monga, Sherry Moore, Derek Murray, Chris Olah, Mike Schuster, Jonathon Shlens, Benoit Steiner, Ilya Sutskever, Kunal Talwar, Paul Tucker, Vincent Vanhoucke, Vijay Vasudevan, Fernanda Viégas, Oriol Vinyals, Pete Warden, Martin Wattenberg, Martin Wicke, Yuan Yu, and Xiaoqiang Zheng. 2015. TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. Software available from tensorflow.org. [find](https://scholar.google.com/scholar?q=Mart%C3%ADn+Abadi%2C+Ashish+Agarwal%2C+Paul+Barham%2C+Eugene+Brevdo%2C+Zhifeng+Chen%2C+Craig+Citro%2C+Greg+S%2E+Corrado%2C+Andy+Davis%2C+Jeffrey+Dean%2C+Matthieu+Devin%2C+Sanjay+Ghemawat%2C+Ian+Goodfellow%2C+Andrew+Harp%2C+Geoffrey+I)
2. Ahmed Ali, Najim Dehak, Patrick Cardinal, Sameer Khurana, Sree Harsha Yella, James Glass, Peter Bell, and Steve Renals. 2016. [Automatic Dialect Detection in Arabic Broadcast Speech](https://md.research.elamur.ai/free/en/papers/ali-2015-automatic-dialect-detection-arabic-broadcast/). In Interspeech 2016, pages 2934–2938.
3. Fadi Biadsy, Julia Hirschberg, and Nizar Habash. 2009. [Spoken Arabic Dialect Identification Using Phonotactic Modeling](https://md.research.elamur.ai/free/en/papers/biadsy-2009-spoken-arabic-dialect-identification-phonotactic/). In Proceedings of the EACL 2009 Workshop on Computational Approaches to Semitic Languages, pages 53–61, Athens, Greece.
4. François Chollet. 2015. Keras. [https://github.com/fchollet/keras](https://github.com/fchollet/keras). [↗](https://github.com/fchollet/keras)
5. Marc Franco-Salvador, Paolo Rosso, and Francisco Rangel. 2015. Distributed Representations of Words and Documents for Discriminating Similar Languages. In Proceedings of the Joint Workshop on Language Technology for Closely Related Languages, Varieties and Dialects (LT4VarDial), pages 11–16, Hissar, Bulgaria. [find](https://scholar.google.com/scholar?q=Marc+Franco%2DSalvador%2C+Paolo+Rosso%2C+and+Francisco+Rangel%2E+2015%2E+Distributed+Representations+of+Words+and+Documents+for+Discriminating+Similar+Languages%2E+In+Proceedings+of+the+Joint+Workshop+on+Language)
6. Xavier Glorot, Antoine Bordes, and Yoshua Bengio. 2011. Deep Sparse Rectifier Neural Networks. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics (AISTATS-11), volume 15, pages 315–323. [find](https://scholar.google.com/scholar?q=Xavier+Glorot%2C+Antoine+Bordes%2C+and+Yoshua+Bengio%2E+2011%2E+Deep+Sparse+Rectifier+Neural+Networks%2E+In+Proceedings+of+the+Fourteenth+International+Conference+on+Artificial+Intelligence+and+Statistics+%28AIST)
7. Cyril Goutte, Serge LÃl’ger, Shervin Malmasi, and Marcos Zampieri. 2016. Discriminating Similar Languages: Evaluations and Explorations. In Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC 2016), Portoroz̆, Slovenia. [find](https://scholar.google.com/scholar?q=Cyril+Goutte%2C+Serge+L%C3%83l%E2%80%99ger%2C+Shervin+Malmasi%2C+and+Marcos+Zampieri%2E+2016%2E+Discriminating+Similar+Languages%3A+Evaluations+and+Explorations%2E+In+Proceedings+of+the+Tenth+International+Conference+on+Languag)
8. Yoon Kim, Yacine Jernite, David Sontag, and Alexander Rush. 2016. [Character-Aware Neural Language Models](https://md.research.elamur.ai/free/en/papers/kim-2015-character-aware-neural-language-models/). In AAAI Conference on Artificial Intelligence.
9. Yoon Kim. 2014. Convolutional Neural Networks for Sentence Classification. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1746–1751, Doha, Qatar. Diederik Kingma and Jimmy Ba. arXiv:1412.6980. 2014. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). arXiv preprint Minh-Thang Luong and Christopher D Manning. 2016. Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models. arXiv preprint arXiv:1604.00788. [↗](https://arxiv.org/abs/1412.6980)
10. Shervin Malmasi, Eshrag Refaee, and Mark Dras. 2015. Arabic Dialect Identification using a Parallel Multidialectal Corpus. In Proceedings of the 14th Conference of the Pacific Association for Computational Linguistics (PACLING 2015), pages 209–217, Bali, Indonesia. [find](https://scholar.google.com/scholar?q=Shervin+Malmasi%2C+Eshrag+Refaee%2C+and+Mark+Dras%2E+2015%2E+Arabic+Dialect+Identification+using+a+Parallel+Multidialectal+Corpus%2E+In+Proceedings+of+the+14th+Conference+of+the+Pacific+Association+for+Computat)
11. Shervin Malmasi, Marcos Zampieri, Nikola Ljubešić, Preslav Nakov, Ahmed Ali, and Jörg Tiedemann. 2016. [Discriminating between Similar Languages and Arabic Dialect Identification: A Report on the Third DSL Shared Task](https://md.research.elamur.ai/free/en/papers/malmasi-2016-discriminating-between-similar-languages/). In Proceedings of the 3rd Workshop on Language Technology for Closely Related Languages, Varieties and Dialects (VarDial), Osaka, Japan.
12. Jordi Porta and José-Luis Sancho. 2014. Using Maximum Entropy Models to Discriminate between Similar Languages and Varieties. In Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects (VarDial), pages 120–128, Dublin, Ireland. [find](https://scholar.google.com/scholar?q=Jordi+Porta+and+Jos%C3%A9%2DLuis+Sancho%2E+2014%2E+Using+Maximum+Entropy+Models+to+Discriminate+between+Similar+Languages+and+Varieties%2E+In+Proceedings+of+the+First+Workshop+on+Applying+NLP+Tools+to+Similar+Lang)
13. Cicero D Santos and Bianca Zadrozny. 2014. Learning character-level representations for part-of-speech tagging. In Proceedings of the 31st International Conference on Machine Learning (ICML-14), pages 1818–1826. [find](https://scholar.google.com/scholar?q=Cicero+D+Santos+and+Bianca+Zadrozny%2E+2014%2E+Learning+character%2Dlevel+representations+for+part%2Dof%2Dspeech+tagging%2E+In+Proceedings+of+the+31st+International+Conference+on+Machine+Learning+%28ICML%2D14%29%2C+pages)
14. Liling Tan, Marcos Zampieri, Nikola Ljubešic, and Jörg Tiedemann. 2014. Merging Comparable Data Sources for the Discrimination of Similar Languages: The DSL Corpus Collection. In Proceedings of the 7th Workshop on Building and Using Comparable Corpora (BUCC), pages 11–15, Reykjavik, Iceland. [find](https://scholar.google.com/scholar?q=Liling+Tan%2C+Marcos+Zampieri%2C+Nikola+Ljube%C5%A1ic%2C+and+J%C3%B6rg+Tiedemann%2E+2014%2E+Merging+Comparable+Data+Sources+for+the+Discrimination+of+Similar+Languages%3A+The+DSL+Corpus+Collection%2E+In+Proceedings+of+the+7t)
15. Omar F. Zaidan and Chris Callison-Burch. 2014. [Arabic Dialect Identification](https://md.research.elamur.ai/free/en/papers/zaidan-2014-arabic-dialect-identification/). Comput. Linguist., 40(1):171–202. [↗](https://doi.org/10.1162/coli_a_00169)
16. Marcos Zampieri, Liling Tan, Nikola Ljubešić, and Jörg Tiedemann. 2014. [A Report on the DSL Shared Task 2014](https://md.research.elamur.ai/free/en/papers/zampieri-2014-report-dsl-shared-task/). In Proceedings of the First Workshop on Applying NLP Tools to Similar Languages, Varieties and Dialects (VarDial), pages 58–67, Dublin, Ireland.
17. Marcos Zampieri, Liling Tan, Nikola Ljubešić, Jörg Tiedemann, and Preslav Nakov. 2015. [Overview of the DSL Shared Task 2015](https://md.research.elamur.ai/free/en/papers/zampieri-2015-overview-dsl-shared-task/). In Proceedings of the Joint Workshop on Language Technology for Closely Related Languages, Varieties and Dialects (LT4VarDial), pages 1–9, Hissar, Bulgaria.
18. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015. [Character-level Convolutional Networks for Text Classification](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/). In C. Cortes, N. D. Lawrence, D. D. Lee, M. Sugiyama, and R. Garnett, editors, Advances in Neural Information Processing Systems 28, pages 649–657. Curran Associates, Inc. [↗](https://doi.org/10.48550/arxiv.1509.01626)

*Updated 1 November 2026*