---
title: "Discriminating between Similar Languages and — Malmasi 2016"
description: "The third DSL shared task, held at VarDial 2016 (co-located with COLING 2016), tested how well automatic systems can tell apart closely related…"
authors: ["Shervin Malmasi", "Marcos Zampieri", "Nikola Ljubešić", "Preslav Nakov", "Ahmed Ali", "Jörg Tiedemann"]
year: 2016
venue: "VarDial 2016"
source: "https://aclanthology.org/W16-4801/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/malmasi-2016-discriminating-between-similar-languages/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/malmasi-2016-discriminating-between-similar-languages/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/2/) / Discriminating between Similar Languages and Arabic Dialect Identification: A Report on the Third DSL Shared Task

# Discriminating between Similar Languages and Arabic Dialect Identification: A Report on the Third DSL Shared Task

**Authors:** [Shervin Malmasi](https://md.research.elamur.ai/free/en/authors/shervin-malmasi/), [Marcos Zampieri](https://md.research.elamur.ai/free/en/authors/marcos-zampieri/), [Nikola Ljubešić](https://md.research.elamur.ai/free/en/authors/nikola-ljubesic/), [Preslav Nakov](https://md.research.elamur.ai/free/en/authors/preslav-nakov/), [Ahmed Ali](https://md.research.elamur.ai/free/en/authors/ahmed-ali/), [Jörg Tiedemann](https://md.research.elamur.ai/free/en/authors/jorg-tiedemann/) · **Year:** 2016 · **Venue:** VarDial 2016 · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/W16-4801/) · [PDF](https://aclanthology.org/W16-4801.pdf) · [OpenAlex](https://openalex.org/W2561747913)

**Citations:** 164 citations (openalex-snapshot · 2026-10-14) · [122](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
The third DSL shared task, held at VarDial 2016 (co-located with COLING 2016), tested how well automatic systems can tell apart closely related languages, language varieties and dialects. It had two subtasks.

- **Subtask 1** used a new release of the DSL corpus collection, DSLCC v3.0. It contains short news texts in twelve languages and varieties, including Hexagonal and Canadian French for the first time. Systems were tested on in-domain news and on out-of-domain Twitter data.
- **Subtask 2** introduced Arabic dialect identification on speech transcripts from the Ali et al. (2016) corpus.

Both subtasks had a closed track (organizer data only) and an open track (any external data). Participation reached a record level for the series.

Across both subtasks, the strongest systems relied on character n-grams of high order combined with conventional classifiers such as SVM and logistic regression. Ensembles and hierarchical classifiers also did well. Most deep learning entries fell behind these simpler methods. The tweet test sets proved harder than news, and Arabic dialects remained difficult, with top F1 scores near 0.5.

The report gives a common benchmark and practical guidance for language identification among close varieties.

## Key points
- 37 teams registered, 24 submitted results and 20 wrote system papers, up from 8 submissions in 2014 and 10 in 2015.
- DSLCC v3.0 provides 20,000 instances per variety: 18,000 for training and 2,000 for development. Test set A has 1,000 news instances per variety. Out-of-domain sets B1 and B2 each cover 100 Twitter users per variety. Mexican Spanish was also added, while easy pairs such as Czech–Slovak and Bulgarian–Macedonian were dropped.
- Besides MSA, the Arabic data includes four regional varieties: Gulf (GLF), Egyptian (EGY), North African (NOR) and Levantine (LAV).
- Arabic closed-track results were led by MAZA, an ensemble of word and character n-grams (F1 0.513). The best open-track result came from ASIREM (F1 0.527), which added 18,000 social media documents collected by native speakers.
- Character n-grams beat word n-grams, especially orders 5–7. Deep models were costly to train, with one team reporting ten days of training.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) · [Computing & GPUs](https://md.research.elamur.ai/free/en/shelves/computing/)

## Cited by

Cited by

122 papers of our library cite it (of the 140673 whose bibliographies we read); 12 of them have a page here, 110 not yet.

- [Arabic Dialect Identification in the Wild](https://md.research.elamur.ai/free/en/papers/abdelali-2020-arabic-dialect-identification-wild/)
- [Camelira: An Arabic Multi-Dialect Morphological Disambiguator](https://md.research.elamur.ai/free/en/papers/aclbulk-2022-obeid-camelira-arabic-multi-dialect-morphological/)
- [Speech Recognition Challenge in the Wild: Arabic MGB-3](https://md.research.elamur.ai/free/en/papers/ali-2017-speech-recognition-challenge-wild-arabic/)
- [Automatic Arabic Dialect Identification Systems for Written Texts: A Survey](https://md.research.elamur.ai/free/en/papers/althobaiti-2020-automatic-arabic-dialect-identification-systems/)
- [Hierarchical Aggregation of Dialectal Data for Arabic Dialect Identification](https://md.research.elamur.ai/free/en/papers/baimukan-2022-hierarchical-aggregation-dialectal-data-arabic/)
- [ADI-20: Arabic Dialect Identification dataset and models](https://md.research.elamur.ai/free/en/papers/elleuch-2025-adi-20-arabic-dialect-identification/)
- [Automatic Language Identification in Texts: A Survey](https://md.research.elamur.ai/free/en/papers/jauhiainen-2019-automatic-language-identification-texts/)
- [Optimizing the Size of Subword Vocabularies in Dialect Classification](https://md.research.elamur.ai/free/en/papers/kanjirangat-2023-optimizing-size-subword-vocabularies-dialect/)
- [DADA: Dialect Adaptation via Dynamic Aggregation of Linguistic Rules](https://md.research.elamur.ai/free/en/papers/liu-2023-dada-dialect-adaptation-dynamic-aggregation/)
- [Fine-Grained Arabic Dialect Identification](https://md.research.elamur.ai/free/en/papers/salameh-2018-fine-grained-arabic-dialect-identification/)
- [Findings of the VarDial Evaluation Campaign 2017](https://md.research.elamur.ai/free/en/papers/zampieri-2017-findings-vardial-evaluation-campaign/)
- [Language Identification and Morphosyntactic Tagging: The Second VarDial Evaluation Campaign](https://md.research.elamur.ai/free/en/papers/zampieri-2018-language-identification-morphosyntactic-tagging-second/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{malmasi2016discriminating,
  title = {{Discriminating between Similar Languages and Arabic Dialect Identification: A Report on the Third DSL Shared Task}},
  author = {Malmasi, Shervin and Zampieri, Marcos and Ljubešić, Nikola and Nakov, Preslav and Ali, Ahmed and Tiedemann, Jörg},
  booktitle = {VarDial 2016},
  year = {2016},
  url = {https://aclanthology.org/W16-4801/}
}
```

- [← previous Discourse Structure and Dialogue Acts in Multiparty Dialogue: the STAC Corpus](https://md.research.elamur.ai/free/en/papers/asher-2016-discourse-structure-dialogue-acts-multiparty/)
- [next → Diverse Beam Search: Decoding Diverse Solutions from Neural Sequence Models](https://md.research.elamur.ai/free/en/papers/vijayakumar-2016-diverse-beam-search-decoding-diverse/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/2/)

## References

61 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Judit Ács, László Grad-Gyenge, and Thiago Bruno Rodrigues de Rezende Oliveira. 2015. A Two-level Classifier for Discriminating Similar Languages. In Proceedings of the LT4VarDial Workshop. [find](https://scholar.google.com/scholar?q=Judit+A%CC%81cs%2C+La%CC%81szlo%CC%81+Grad%2DGyenge%2C+and+Thiago+Bruno+Rodrigues+de+Rezende+Oliveira%2E+2015%2E+A+Two%2Dlevel+Classifier+for+Discriminating+Similar+Languages%2E+In+Proceedings+of+the+LT4VarDial+Workshop%2E)
2. Wafia Adouane, Nasredine Semmar, Richard Johansson, and Victoria Bobicev. 2016. ASIREM Participation at the Discriminating Similar Languages Shared Task 2016. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Wafia+Adouane%2C+Nasredine+Semmar%2C+Richard+Johansson%2C+and+Victoria+Bobicev%2E+2016%2E+ASIREM+Participation+at+the+Discriminating+Similar+Languages+Shared+Task+2016%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
3. Ahmed Ali, Najim Dehak, Patrick Cardinal, Sameer Khurana, Sree Harsha Yella, James Glass, Peter Bell, and Steve Renals. 2016. Automatic Dialect Detection in Arabic Broadcast Speech. In Proceedings of Interspeech. [find](https://scholar.google.com/scholar?q=Ahmed+Ali%2C+Najim+Dehak%2C+Patrick+Cardinal%2C+Sameer+Khurana%2C+Sree+Harsha+Yella%2C+James+Glass%2C+Peter+Bell%2C+and+Steve+Renals%2E+2016%2E+Automatic+Dialect+Detection+in+Arabic+Broadcast+Speech%2E+In+Proceedings+of+)
4. Areej Alshutayri, Eric Atwell, Abdulrahman Alosaimy, James Dickins, Michale Ingleby, and Janet Watson. 2016. Arabic Language WEKA-Based Dialect Classifier for Arabic Automatic Speech Recognition Transcripts. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Areej+Alshutayri%2C+Eric+Atwell%2C+Abdulrahman+Alosaimy%2C+James+Dickins%2C+Michale+Ingleby%2C+and+Janet+Watson%2E+2016%2E+Arabic+Language+WEKA%2DBased+Dialect+Classifier+for+Arabic+Automatic+Speech+Recognition+Trans)
5. Timothy Baldwin and Marco Lui. 2010. Multilingual Language Identification: ALTW 2010 Shared Task Data. In Proceedings of ALTA. [find](https://scholar.google.com/scholar?q=Timothy+Baldwin+and+Marco+Lui%2E+2010%2E+Multilingual+Language+Identification%3A+ALTW+2010+Shared+Task+Data%2E+In+Proceedings+of+ALTA%2E)
6. Adrien Barbaresi. 2016. An Unsupervised Morphological Criterion for Discriminating Similar Languages. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Adrien+Barbaresi%2E+2016%2E+An+Unsupervised+Morphological+Criterion+for+Discriminating+Similar+Languages%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
7. Yonatan Belinkov and James Glass. 2016. A Character-level Convolutional Neural Network for Distinguishing Similar Languages and Dialects. In Proceedings of the VarDial Workshop. [↗](https://doi.org/10.48550/arxiv.1609.07568)
8. Johannes Bjerva. 2016. Byte-based Language Identification with Deep Convolutional Networks. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Johannes+Bjerva%2E+2016%2E+Byte%2Dbased+Language+Identification+with+Deep+Convolutional+Networks%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
9. Victoria Bobicev. 2015. LT4VarDial Workshop. Discriminating between Similar Languages Using PPM. In Proceedings of the Ondrej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, et al. 2016. Findings of the 2016 Conference on Machine Translation. In Proceedings of WMT. [find](https://scholar.google.com/scholar?q=Victoria+Bobicev%2E+2015%2E+LT4VarDial+Workshop%2E+Discriminating+between+Similar+Languages+Using+PPM%2E+In+Proceedings+of+the+Ondrej+Bojar%2C+Rajen+Chatterjee%2C+Christian+Federmann%2C+Yvette+Graham%2C+Barry+Haddow%2C) [our card](https://md.research.elamur.ai/free/en/papers/bojar-2016-findings-2016-conference-machine-translation/)
10. Çağrı Çöltekin and Taraka Rama. 2016. Discriminating Similar Languages with Linear SVMs and Neural Networks. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=C%CC%A7ag%CC%86r%C4%B1+C%CC%A7o%CC%88ltekin+and+Taraka+Rama%2E+2016%2E+Discriminating+Similar+Languages+with+Linear+SVMs+and+Neural+Networks%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
11. Andre Cianflone and Leila Kosseim. 2016. N-gram and Neural Language Models for Discriminating Similar Languages. In Proceedings of the VarDial Workshop. [↗](https://doi.org/10.48550/arxiv.1708.03421)
12. Alina Maria Ciobanu, Sergiu Nisioi, and Liviu P. Dinu. 2016. Vanilla Classifiers for Distinguishing between Similar Languages. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Alina+Maria+Ciobanu%2C+Sergiu+Nisioi%2C+and+Liviu+P%2E+Dinu%2E+2016%2E+Vanilla+Classifiers+for+Distinguishing+between+Similar+Languages%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
13. Kareem Darwish, Hassan Sajjad, and Hamdy Mubarak. 2014. Verifiably Effective Arabic Dialect Identification. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Kareem+Darwish%2C+Hassan+Sajjad%2C+and+Hamdy+Mubarak%2E+2014%2E+Verifiably+Effective+Arabic+Dialect+Identification%2E+In+Proceedings+of+EMNLP%2E)
14. Mohamed Eldesouki, Fahim Dalvi, Hassan Sajjad, and Kareem Darwish. 2016. QCRI @ DSL 2016: Spoken Arabic Dialect Identification Using Textual Features. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Mohamed+Eldesouki%2C+Fahim+Dalvi%2C+Hassan+Sajjad%2C+and+Kareem+Darwish%2E+2016%2E+QCRI+%40+DSL+2016%3A+Spoken+Arabic+Dialect+Identification+Using+Textual+Features%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
15. Heba Elfardy and Mona Diab. 2014. Sentence Level Dialect Identification in Arabic. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Heba+Elfardy+and+Mona+Diab%2E+2014%2E+Sentence+Level+Dialect+Identification+in+Arabic%2E+In+Proceedings+of+ACL%2E)
16. Raül Fabra-Boluda, Francisco Rangel, and Paolo Rosso. 2015. NLEL UPV Autoritas participation at Discrimination between Similar Languages (DSL) 2015 shared task. In Proceedings of the LT4VarDial Workshop. [find](https://scholar.google.com/scholar?q=Rau%CC%88l+Fabra%2DBoluda%2C+Francisco+Rangel%2C+and+Paolo+Rosso%2E+2015%2E+NLEL+UPV+Autoritas+participation+at+Discrimination+between+Similar+Languages+%28DSL%29+2015+shared+task%2E+In+Proceedings+of+the+LT4VarDial+Works)
17. Hector-Hugo Franco-Penya and Liliana Mamani Sanchez. 2016. Tuning Bayes Baseline for Dialect Detection. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Hector%2DHugo+Franco%2DPenya+and+Liliana+Mamani+Sanchez%2E+2016%2E+Tuning+Bayes+Baseline+for+Dialect+Detection%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
18. Marc Franco-Salvador, Paolo Rosso, and Francisco Rangel. 2015. Distributed Representations of Words and Documents for Discriminating Similar Languages. In Proceedings of the LT4VarDial Workshop. [find](https://scholar.google.com/scholar?q=Marc+Franco%2DSalvador%2C+Paolo+Rosso%2C+and+Francisco+Rangel%2E+2015%2E+Distributed+Representations+of+Words+and+Documents+for+Discriminating+Similar+Languages%2E+In+Proceedings+of+the+LT4VarDial+Workshop%2E)
19. Pablo Gamallo, Iñaki Alegria, and José Ramom Pichel. 2016. Comparing two Basic Methods for Discriminating Between Similar Languages and Varieties. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Pablo+Gamallo%2C+In%CC%83aki+Alegria%2C+and+Jose%CC%81+Ramom+Pichel%2E+2016%2E+Comparing+two+Basic+Methods+for+Discriminating+Between+Similar+Languages+and+Varieties%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
20. Cyril Goutte and Serge Léger. 2015. Experiments in Discriminating Similar Languages. In Proceedings of the LT4VarDial Workshop. [find](https://scholar.google.com/scholar?q=Cyril+Goutte+and+Serge+Le%CC%81ger%2E+2015%2E+Experiments+in+Discriminating+Similar+Languages%2E+In+Proceedings+of+the+LT4VarDial+Workshop%2E)
21. Cyril Goutte and Serge Léger. 2016. Advances in Ngram-based Discrimination of Similar Languages. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Cyril+Goutte+and+Serge+Le%CC%81ger%2E+2016%2E+Advances+in+Ngram%2Dbased+Discrimination+of+Similar+Languages%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
22. Cyril Goutte, Serge Léger, and Marine Carpuat. 2014. The NRC System for Discriminating Similar Languages. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Cyril+Goutte%2C+Serge+Le%CC%81ger%2C+and+Marine+Carpuat%2E+2014%2E+The+NRC+System+for+Discriminating+Similar+Languages%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
23. Cyril Goutte, Serge Léger, Shervin Malmasi, and Marcos Zampieri. 2016. Discriminating Similar Languages: Evaluations and Explorations. In Proceedings of LREC. [find](https://scholar.google.com/scholar?q=Cyril+Goutte%2C+Serge+Le%CC%81ger%2C+Shervin+Malmasi%2C+and+Marcos+Zampieri%2E+2016%2E+Discriminating+Similar+Languages%3A+Evaluations+and+Explorations%2E+In+Proceedings+of+LREC%2E)
24. Cyril Grouin, Dominic Forest, Lyne Da Sylva, Patrick Paroubek, and Pierre Zweigenbaum. 2010. Présentation et Résultats du Défi Fouille de Texte DEFT2010 Où et Quand un Article de Presse a-t-il Été Écrit? In Proceedings of DEFT. [find](https://scholar.google.com/scholar?q=Cyril+Grouin%2C+Dominic+Forest%2C+Lyne+Da+Sylva%2C+Patrick+Paroubek%2C+and+Pierre+Zweigenbaum%2E+2010%2E+Pre%CC%81sentation+et+Re%CC%81sultats+du+De%CC%81fi+Fouille+de+Texte+DEFT2010+Ou%CC%80+et+Quand+un+Article+de+Presse+a%2Dt%2Dil+E%CC%81t)
25. Chinnappa Guggilla. 2016. Discrimination between Similar Languages, Varieties and Dialects using CNN and LSTM-based Deep Neural Networks. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Chinnappa+Guggilla%2E+2016%2E+Discrimination+between+Similar+Languages%2C+Varieties+and+Dialects+using+CNN+and+LSTM%2Dbased+Deep+Neural+Networks%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
26. Bo Han, Afshin Rahimi, Leon Derczynski, and Timothy Baldwin. 2016. Twitter Geolocation Prediction Shared Task of the 2016 Workshop on Noisy User-generated Text. In Proceedings of the W-NUT Workshop. [find](https://scholar.google.com/scholar?q=Bo+Han%2C+Afshin+Rahimi%2C+Leon+Derczynski%2C+and+Timothy+Baldwin%2E+2016%2E+Twitter+Geolocation+Prediction+Shared+Task+of+the+2016+Workshop+on+Noisy+User%2Dgenerated+Text%2E+In+Proceedings+of+the+W%2DNUT+Workshop%2E)
27. Abualsoud Hanani, Aziz Qaroush, and Stephen Taylor. 2016. Classifying ASR Transcriptions According to Arabic Dialect. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Abualsoud+Hanani%2C+Aziz+Qaroush%2C+and+Stephen+Taylor%2E+2016%2E+Classifying+ASR+Transcriptions+According+to+Arabic+Dialect%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
28. Ondr̆ej Herman, Vit Suchomel, Vı́t Baisa, and Pavel Pavel Rychlý. 2016. DSL Shared task 2016: Perfect Is The Enemy of Good Language Discrimination Through Expectation–Maximization and Chunk-based Language Model. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Ondr%CC%86ej+Herman%2C+Vit+Suchomel%2C+V%C4%B1%CC%81t+Baisa%2C+and+Pavel+Pavel+Rychly%CC%81%2E+2016%2E+DSL+Shared+task+2016%3A+Perfect+Is+The+Enemy+of+Good+Language+Discrimination+Through+Expectation%E2%80%93Maximization+and+Chunk%2Dbased+Lan)
29. Chu-ren Huang and Lung-hao Lee. 2008. Contrastive Approach towards Text Source Classification based on Top-Bag-of-Word Similarity. In Proceedings of PACLIC. [find](https://scholar.google.com/scholar?q=Chu%2Dren+Huang+and+Lung%2Dhao+Lee%2E+2008%2E+Contrastive+Approach+towards+Text+Source+Classification+based+on+Top%2DBag%2Dof%2DWord+Similarity%2E+In+Proceedings+of+PACLIC%2E)
30. Radu Tudor Ionescu and Marius Popescu. 2016. UnibucKernel: An Approach for Arabic Dialect Identification based on Multiple String Kernels. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Radu+Tudor+Ionescu+and+Marius+Popescu%2E+2016%2E+UnibucKernel%3A+An+Approach+for+Arabic+Dialect+Identification+based+on+Multiple+String+Kernels%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
31. Radu Tudor Ionescu, Marius Popescu, and Aoife Cahill. 2016. String Kernels for Native Language Identification: Insights from Behind the Curtains. Computational Linguistics, 43(3):491–525. [↗](https://doi.org/10.1162/coli_a_00256)
32. Tommi Jauhiainen, Heidi Jauhiainen, and Krister Lindén. 2015. Discriminating Similar Languages with Token-based Backoff. In Proceedings of the LT4VarDial Workshop. [find](https://scholar.google.com/scholar?q=Tommi+Jauhiainen%2C+Heidi+Jauhiainen%2C+and+Krister+Linde%CC%81n%2E+2015%2E+Discriminating+Similar+Languages+with+Token%2Dbased+Backoff%2E+In+Proceedings+of+the+LT4VarDial+Workshop%2E)
33. Tommi Jauhiainen, Krister Lindén, and Heidi Jauhiainen. 2016. HeLI, a Word-Based Backoff Method for Language Identification. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Tommi+Jauhiainen%2C+Krister+Linde%CC%81n%2C+and+Heidi+Jauhiainen%2E+2016%2E+HeLI%2C+a+Word%2DBased+Backoff+Method+for+Language+Identification%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
34. Ben King, Dragomir Radev, and Steven Abney. 2014. Experiments in Sentence Language Identification with Groups of Similar Languages. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Ben+King%2C+Dragomir+Radev%2C+and+Steven+Abney%2E+2014%2E+Experiments+in+Sentence+Language+Identification+with+Groups+of+Similar+Languages%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
35. Nikola Ljubešić and Denis Kranjčić. 2015. Discriminating Between Closely Related Languages on Twitter. Informatica, 39(1). [find](https://scholar.google.com/scholar?q=Nikola+Ljubes%CC%8Cic%CC%81+and+Denis+Kranjc%CC%8Cic%CC%81%2E+2015%2E+Discriminating+Between+Closely+Related+Languages+on+Twitter%2E+Informatica%2C+39%281%29%2E)
36. Nikola Ljubešić, Nives Mikelic, and Damir Boras. 2007. Language identification: How to distinguish similar languages? In Proceedings of ITI. [find](https://scholar.google.com/scholar?q=Nikola+Ljubes%CC%8Cic%CC%81%2C+Nives+Mikelic%2C+and+Damir+Boras%2E+2007%2E+Language+identification%3A+How+to+distinguish+similar+languages%3F+In+Proceedings+of+ITI%2E)
37. Nikola Ljubešić, Darja Fišer, and Tomaž Erjavec. 2014. TweetCaT: a Tool for Building Twitter Corpora of Smaller Languages. In Proceedings of LREC. [find](https://scholar.google.com/scholar?q=Nikola+Ljubes%CC%8Cic%CC%81%2C+Darja+Fis%CC%8Cer%2C+and+Tomaz%CC%8C+Erjavec%2E+2014%2E+TweetCaT%3A+a+Tool+for+Building+Twitter+Corpora+of+Smaller+Languages%2E+In+Proceedings+of+LREC%2E)
38. Nikola Ljubešić, Tanja Samardžić, and Curdin Derungs. 2016. TweetGeo – A Tool for Collecting, Processing and Analysing Geo-encoded Data. In Proceedings of COLING. [find](https://scholar.google.com/scholar?q=Nikola+Ljubes%CC%8Cic%CC%81%2C+Tanja+Samardz%CC%8Cic%CC%81%2C+and+Curdin+Derungs%2E+2016%2E+TweetGeo+%E2%80%93+A+Tool+for+Collecting%2C+Processing+and+Analysing+Geo%2Dencoded+Data%2E+In+Proceedings+of+COLING%2E)
39. Marco Lui and Paul Cook. 2013. Classifying English Documents by National Dialect. In Proceedings of ALTA. [find](https://scholar.google.com/scholar?q=Marco+Lui+and+Paul+Cook%2E+2013%2E+Classifying+English+Documents+by+National+Dialect%2E+In+Proceedings+of+ALTA%2E)
40. Marco Lui, Ned Letcher, Oliver Adams, Long Duong, Paul Cook, and Timothy Baldwin. 2014. Exploring Methods and Resources for Discriminating Similar Languages. In Proceedings of VarDial. [find](https://scholar.google.com/scholar?q=Marco+Lui%2C+Ned+Letcher%2C+Oliver+Adams%2C+Long+Duong%2C+Paul+Cook%2C+and+Timothy+Baldwin%2E+2014%2E+Exploring+Methods+and+Resources+for+Discriminating+Similar+Languages%2E+In+Proceedings+of+VarDial%2E)
41. Shervin Malmasi and Mark Dras. 2015a. Automatic Language Identification for Persian and Dari Texts. In Proceedings of PACLING. [find](https://scholar.google.com/scholar?q=Shervin+Malmasi+and+Mark+Dras%2E+2015a%2E+Automatic+Language+Identification+for+Persian+and+Dari+Texts%2E+In+Proceedings+of+PACLING%2E)
42. Shervin Malmasi and Mark Dras. 2015b. Language Identification using Classifier Ensembles. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Shervin+Malmasi+and+Mark+Dras%2E+2015b%2E+Language+Identification+using+Classifier+Ensembles%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
43. Shervin Malmasi and Marcos Zampieri. 2016. Arabic Dialect Identification in Speech Transcripts. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Shervin+Malmasi+and+Marcos+Zampieri%2E+2016%2E+Arabic+Dialect+Identification+in+Speech+Transcripts%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
44. Paul McNamee. 2016. Language and Dialect Discrimination Using Compression-Inspired Language Models. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Paul+McNamee%2E+2016%2E+Language+and+Dialect+Discrimination+Using+Compression%2DInspired+Language+Models%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
45. Jordi Porta and José-Luis Sancho. 2014. Using Maximum Entropy Models to Discriminate between Similar Languages and Varieties. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Jordi+Porta+and+Jose%CC%81%2DLuis+Sancho%2E+2014%2E+Using+Maximum+Entropy+Models+to+Discriminate+between+Similar+Languages+and+Varieties%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
46. Matthew Purver. 2014. A Simple Baseline for Discriminating Similar Language. In Proceedings of VarDial Workshop. [find](https://scholar.google.com/scholar?q=Matthew+Purver%2E+2014%2E+A+Simple+Baseline+for+Discriminating+Similar+Language%2E+In+Proceedings+of+VarDial+Workshop%2E)
47. Bali Ranaivo-Malançon. 2006. Automatic identification of close languages - case study: Malay and Indonesian. ECTI Transactions on Computer and Information Technology, 2:126–134. [find](https://scholar.google.com/scholar?q=Bali+Ranaivo%2DMalanc%CC%A7on%2E+2006%2E+Automatic+identification+of+close+languages+%2D+case+study%3A+Malay+and+Indonesian%2E+ECTI+Transactions+on+Computer+and+Information+Technology%2C+2%3A126%E2%80%93134%2E)
48. Abdulhadi Shoufan and Sumaya Al-Ameri. 2015. Natural Language Processing for Dialectical Arabic: A Survey. In Proceedings of the Arabic NLP Workshop. [find](https://scholar.google.com/scholar?q=Abdulhadi+Shoufan+and+Sumaya+Al%2DAmeri%2E+2015%2E+Natural+Language+Processing+for+Dialectical+Arabic%3A+A+Survey%2E+In+Proceedings+of+the+Arabic+NLP+Workshop%2E)
49. Thamar Solorio, Elizabeth Blair, Suraj Maharjan, Steven Bethard, Mona Diab, Mahmoud Ghoneim, Abdelati Hawwari, Fahad AlGhamdi, Julia Hirschberg, Alison Chang, and Pascale Fung. 2014. Overview for the First Shared Task on Language Identification in Code-Switched Data. In Proceedings of the CodeSwitch Workshop. [find](https://scholar.google.com/scholar?q=Thamar+Solorio%2C+Elizabeth+Blair%2C+Suraj+Maharjan%2C+Steven+Bethard%2C+Mona+Diab%2C+Mahmoud+Ghoneim%2C+Abdelati+Hawwari%2C+Fahad+AlGhamdi%2C+Julia+Hirschberg%2C+Alison+Chang%2C+and+Pascale+Fung%2E+2014%2E+Overview+for+the+) [our card](https://md.research.elamur.ai/free/en/papers/solorio-2014-overview-first-shared-task-language/)
50. Liling Tan, Marcos Zampieri, Nikola Ljubešić, and Jörg Tiedemann. 2014. Merging Comparable Data Sources for the Discrimination of Similar Languages: The DSL Corpus Collection. In Proceedings of the BUCC Workshop. [find](https://scholar.google.com/scholar?q=Liling+Tan%2C+Marcos+Zampieri%2C+Nikola+Ljubes%CC%8Cic%CC%81%2C+and+Jo%CC%88rg+Tiedemann%2E+2014%2E+Merging+Comparable+Data+Sources+for+the+Discrimination+of+Similar+Languages%3A+The+DSL+Corpus+Collection%2E+In+Proceedings+of+the)
51. Christoph Tillmann, Saab Mansour, and Yaser Al-Onaizan. 2014. Improved Sentence-Level Arabic Dialect Classification. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Christoph+Tillmann%2C+Saab+Mansour%2C+and+Yaser+Al%2DOnaizan%2E+2014%2E+Improved+Sentence%2DLevel+Arabic+Dialect+Classification%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
52. Samantha Wray and Ahmed Ali. 2015. Crowdsource a little to label a lot: labeling a speech corpus of dialectal Arabic. In Proceedings of Interspeech. [find](https://scholar.google.com/scholar?q=Samantha+Wray+and+Ahmed+Ali%2E+2015%2E+Crowdsource+a+little+to+label+a+lot%3A+labeling+a+speech+corpus+of+dialectal+Arabic%2E+In+Proceedings+of+Interspeech%2E)
53. Omar F. Zaidan and Chris Callison-Burch. 2011. Crowdsourcing translation: Professional quality from non-professionals. In Proceedings of ACL-HLT. [find](https://scholar.google.com/scholar?q=Omar+F%2E+Zaidan+and+Chris+Callison%2DBurch%2E+2011%2E+Crowdsourcing+translation%3A+Professional+quality+from+non%2Dprofessionals%2E+In+Proceedings+of+ACL%2DHLT%2E) [our card](https://md.research.elamur.ai/free/en/papers/zaidan-2011-crowdsourcing-translation-professional-quality-non/)
54. Omar F Zaidan and Chris Callison-Burch. 2014. Arabic Dialect Identification. Computational Linguistics, 40(1):171–202. [↗](https://doi.org/10.1162/coli_a_00169) [our card](https://md.research.elamur.ai/free/en/papers/zaidan-2014-arabic-dialect-identification/)
55. Marcos Zampieri and Binyam Gebrekidan Gebre. 2012. Automatic Identification of Language Varieties: The Case of Portuguese. In Proceedings of KONVENS. [find](https://scholar.google.com/scholar?q=Marcos+Zampieri+and+Binyam+Gebrekidan+Gebre%2E+2012%2E+Automatic+Identification+of+Language+Varieties%3A+The+Case+of+Portuguese%2E+In+Proceedings+of+KONVENS%2E)
56. Marcos Zampieri, Liling Tan, Nikola Ljubešić, and Jörg Tiedemann. 2014. A Report on the DSL Shared Task 2014. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Marcos+Zampieri%2C+Liling+Tan%2C+Nikola+Ljubes%CC%8Cic%CC%81%2C+and+Jo%CC%88rg+Tiedemann%2E+2014%2E+A+Report+on+the+DSL+Shared+Task+2014%2E+In+Proceedings+of+the+VarDial+Workshop%2E) [our card](https://md.research.elamur.ai/free/en/papers/zampieri-2014-report-dsl-shared-task/)
57. Marcos Zampieri, Binyam Gebrekidan Gebre, Hernani Costa, and Josef van Genabith. 2015a. Comparing Approaches to the Identification of Similar Languages. In Proceedings of the VarDial Workshop. [↗](https://doi.org/10.13140/rg.2.1.4315.9528)
58. Marcos Zampieri, Liling Tan, Nikola Ljubešić, Jörg Tiedemann, and Preslav Nakov. 2015b. Overview of the DSL Shared Task 2015. In Proceedings of the LT4VarDial Workshop. [find](https://scholar.google.com/scholar?q=Marcos+Zampieri%2C+Liling+Tan%2C+Nikola+Ljubes%CC%8Cic%CC%81%2C+Jo%CC%88rg+Tiedemann%2C+and+Preslav+Nakov%2E+2015b%2E+Overview+of+the+DSL+Shared+Task+2015%2E+In+Proceedings+of+the+LT4VarDial+Workshop%2E) [our card](https://md.research.elamur.ai/free/en/papers/zampieri-2015-overview-dsl-shared-task/)
59. Ayah Zirikly, Bart Desmet, and Mona Diab. 2016. The GW/LT3 VarDial 2016 Shared Task System for Dialects and Similar Languages Detection. In Proceedings of the VarDial Workshop. [find](https://scholar.google.com/scholar?q=Ayah+Zirikly%2C+Bart+Desmet%2C+and+Mona+Diab%2E+2016%2E+The+GW%2FLT3+VarDial+2016+Shared+Task+System+for+Dialects+and+Similar+Languages+Detection%2E+In+Proceedings+of+the+VarDial+Workshop%2E)
60. Arkaitz Zubiaga, Inaki San Vicente, Pablo Gamallo, José Ramom Pichel, Inaki Alegria, Nora Aranberri, Aitzol Ezeiza, and Vıctor Fresno. 2014. Overview of TweetLID: Tweet Language Identification at SEPLN 2014. In Proceedings of the TweetLID Workshop. [↗](https://doi.org/10.13140/2.1.4326.8482)
61. Arkaitz Zubiaga, Iñaki San Vicente, Pablo Gamallo, José Ramom Pichel, Iñaki Alegria, Nora Aranberri, Aitzol Ezeiza, and Vı́ctor Fresno. 2015. TweetLID: A Benchmark for Tweet Language Identification. Language Resources and Evaluation, pages 1–38. [↗](https://doi.org/10.1007/s10579-015-9317-4)