---
title: "Experiments in Cuneiform Language Identification — Paetzold 2019"
description: "The paper addresses language identification for texts in cuneiform, one of the oldest writing systems. 18.2 F1. 26 references, cited by 1 paper here."
authors: ["Gustavo Henrique Paetzold", "Marcos Zampieri"]
year: 2019
venue: "Proceedings of the Sixth Workshop on NLP for Similar Languages, Varieties and Dialects"
source: "https://aclanthology.org/W19-1423/"
doi: "10.18653/v1/w19-1423"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/paetzold-2019-experiments-cuneiform-language-identification/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/paetzold-2019-experiments-cuneiform-language-identification/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/) / Experiments in Cuneiform Language Identification

# Experiments in Cuneiform Language Identification

**Authors:** [Gustavo Henrique Paetzold](https://md.research.elamur.ai/free/en/authors/gustavo-paetzold/) and [Marcos Zampieri](https://md.research.elamur.ai/free/en/authors/marcos-zampieri/) · **Year:** 2019 · **Venue:** Proceedings of the Sixth Workshop on NLP for Similar Languages, Varieties and Dialects · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/W19-1423/) · [PDF](https://aclanthology.org/W19-1423.pdf) · [DOI 10.18653/v1/w19-1423](https://doi.org/10.18653/v1/w19-1423) · [OpenAlex](https://openalex.org/W2941883773)

**Citations:** 5 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/paetzold-2019-experiments-cuneiform-language-identification/cited-by/) in our library, of the 140,884 papers whose bibliographies we read

## Summary
The paper addresses language identification for texts in cuneiform, one of the oldest writing systems. It was written for the Cuneiform Language Identification (CLI) shared task at VarDial 2019. The task has seven classes: Sumerian and six Akkadian dialects. The data comes from Jauhiainen et al. (2019), who took transliterated texts from Oracc and converted them back into cuneiform signs. The team PZ adapted an existing VarDial meta-classifier. Several linear SVMs are each trained on a separate family of character n-gram or skip-gram features, and a bagging ensemble then learns from their outputs. Trained only on the official data, the system reached 0.738 macro F1 and placed fourth of eight teams, a few points behind the top three. The authors also compared it with a character-level compositional GRU network from a related team. The SVM ensemble beat the network by a wide margin, especially on closely related dialects. The paper matters because it is an early study of identifying historical languages. It also supports the view that well-engineered n-gram models remain strong baselines against neural approaches.

## Key points
- **Data:**
  - Source: dataset of Jauhiainen et al. (2019), built from Oracc.
  - Size: about 13,006 texts, 278,837 lines and roughly 1.9 million signs.
  - Classes: Sumerian (SUX) and six Akkadian varieties, namely Neo Assyrian, Late Babylonian, Neo Babylonian, Standard Babylonian, peripheral Middle Babylonian and Old Babylonian.
- **Features:** character n-grams of order 1–5, plus 1-, 2- and 3-skip character bigrams and trigrams. Each feature family trains its own LIBLINEAR SVM, and a bagging meta-classifier is fit on the SVM outputs.
- **Ranking (macro F1):**
  - NRC-CNRC 0.769 (first; the authors guess a BERT-like neural method may explain this)
  - tearsofjoy 0.763
  - Twist Bytes 0.743
  - PZ 0.738 (fourth)
  - ghpaetzold neural model 0.556 (sixth)
- **Neural baseline (ghpaetzold):**
  - 25-dimensional character embeddings feed a 2-layer GRU that builds word vectors.
  - A second 2-layer GRU combines the word vectors into a sentence vector, followed by a softmax layer.
  - Settings: 30 units per layer and 50% dropout; the epoch checkpoint with the lowest dev error was submitted.
- **Error analysis:** Neo Assyrian was the hardest class to separate from both Standard Babylonian and Neo Babylonian. Both systems struggled with these pairs, but the meta-classifier made far fewer errors. Overall it scored 18.2 F1 points above the neural model.

## Interesting ideas
- Train one simple linear model per feature family instead of one model on all features, then let a second-level learner decide how much to trust each; this keeps each base model interpretable while still combining evidence.
- Character skip-grams capture discontinuous letter patterns that ordinary contiguous n-grams miss, which helps separate very close varieties from short strings.
- Comparing two models from entirely different families on the same split, and reading their confusion matrices side by side, shows exactly which class pairs cause the gap rather than just a single score.
- Saving a checkpoint after every epoch and submitting the one with the lowest development error is a cheap way to avoid overtraining a neural model.
- Character-level features can also tell apart close varieties of the same language, not just separate languages.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Instead of a fixed majority vote among witness parsers for silver data, a small stacked learner could learn per-witness, per-relation reliability while keeping each witness's vote inspectable.
- **I2** (tokens): Skip-gram features could strengthen the guesser for out-of-vocabulary words (POS, lemma, morphology) in crate en, especially for affixed or misspelled forms.
- **I3** (learn): Evaluation could report pairwise confusions (for example, which UD relations the perceptron parser and gparse each mix up) to direct repair rules at the true weak spots.
- **I4** (tree): gparse training could keep per-epoch checkpoints and select by dev LAS, logging which epoch won for explainability.
- **I5** (tree): A light character-feature classifier could flag which treebank convention or English variety an input sentence resembles before UD dialect conversion is applied.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [𒀭 Akkadian and cuneiform NLP](https://md.research.elamur.ai/free/en/topics/akkadian-cuneiform-nlp/)

## Institutions

- [University of Wolverhampton](https://md.research.elamur.ai/free/en/universities/university-of-wolverhampton/) — University, Wolverhampton, United Kingdom · 27 works · 5,947 citations
- [Universidade Tecnológica Federal do Paraná](https://md.research.elamur.ai/free/en/universities/universidade-tecnologica-federal-do-parana/) — University, Curitiba, Brazil · 4 works · 442 citations

## Cite

Cite as: Gustavo Henrique Paetzold and Marcos Zampieri. 2019. Experiments in Cuneiform Language Identification. Proceedings of the Sixth Workshop on NLP for Similar Languages, Varieties and Dialects.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{paetzold2019experiments,
  title = {{Experiments in Cuneiform Language Identification}},
  author = {Gustavo Henrique Paetzold and Marcos Zampieri},
  booktitle = {Proceedings of the Sixth Workshop on NLP for Similar Languages, Varieties and Dialects},
  year = {2019},
  doi = {10.18653/v1/w19-1423},
  url = {https://doi.org/10.18653/v1/w19-1423}
}
```

- [← previous Constrained Sequence-to-sequence Semitic Root Extraction for Enriching Word Embeddings](https://md.research.elamur.ai/free/en/papers/kishky-2019-constrained-sequence-sequence-semitic-root/)
- [next → Subjective Sentiment Analysis for Arabic Newswire Comments](https://md.research.elamur.ai/free/en/papers/bessou-2019-subjective-sentiment-analysis-arabic-newswire/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/5/)

## References

26 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Andrei Butnaru and Radu Tudor Ionescu. 2019. MO-ROCO: The Moldavian and Romanian Dialectal Corpus. arXiv preprint arXiv:1901.06543. [↗](https://arxiv.org/abs/1901.06543)
2. Christian Chiarcos, Émilie Pagé-Perron, Ilya Khait, Niko Schenk, and Lucas Reckling. 2018. Towards a Linked Open Data Edition of Sumerian Corpora. In Proceedings of LREC. [find](https://scholar.google.com/scholar?q=Christian+Chiarcos%2C+E%CC%81milie+Page%CC%81%2DPerron%2C+Ilya+Khait%2C+Niko+Schenk%2C+and+Lucas+Reckling%2E+2018%2E+Towards+a+Linked+Open+Data+Edition+of+Sumerian+Corpora%2E+In+Proceedings+of+LREC%2E)
3. Alina Maria Ciobanu and Liviu P Dinu. 2016. A computational perspective on the romanian dialects. In Proceedings of LREC. [find](https://scholar.google.com/scholar?q=Alina+Maria+Ciobanu+and+Liviu+P+Dinu%2E+2016%2E+A+computational+perspective+on+the+romanian+dialects%2E+In+Proceedings+of+LREC%2E)
4. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. BERT: Pre-training of Hanna Sababa and Athena Stassopoulou. 2018. A Classifier to Distinguish Between Cypriot Greek and Standard Modern Greek. In Proceedings of SNAMS. Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805. [↗](https://arxiv.org/abs/1810.04805) [our card](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/)
5. Rong-En Fan, Kai-Wei Chang, Cho-Jui Hsieh, Xiang-Rui Wang, and Chih-Jen Lin. 2008. LIBLINEAR: A Library for Large Linear Classification. Journal of Machine Learning Research, 9(Aug):1871–1874. [find](https://scholar.google.com/scholar?q=Rong%2DEn+Fan%2C+Kai%2DWei+Chang%2C+Cho%2DJui+Hsieh%2C+Xiang%2DRui+Wang%2C+and+Chih%2DJen+Lin%2E+2008%2E+LIBLINEAR%3A+A+Library+for+Large+Linear+Classification%2E+Journal+of+Machine+Learning+Research%2C+9%28Aug%29%3A1871%E2%80%931874%2E)
6. Octavia-Maria Sulea, Marcos Zampieri, Shervin Malmasi, Mihaela Vela, Liviu P Dinu, and Josef van Genabith. 2017. Exploring the use of Text Classification in the Legal Domain. Proceedings of ASAIL. [find](https://scholar.google.com/scholar?q=Octavia%2DMaria+Sulea%2C+Marcos+Zampieri%2C+Shervin+Malmasi%2C+Mihaela+Vela%2C+Liviu+P+Dinu%2C+and+Josef+van+Genabith%2E+2017%2E+Exploring+the+use+of+Text+Classification+in+the+Legal+Domain%2E+Proceedings+of+ASAIL%2E)
7. Christophe Giraud-Carrier, Ricardo Vilalta, and Pavel Brazdil. 2004. Introduction to the Special Issue on Meta-learning. Machine learning, 54(3):187–193. [find](https://scholar.google.com/scholar?q=Christophe+Giraud%2DCarrier%2C+Ricardo+Vilalta%2C+and+Pavel+Brazdil%2E+2004%2E+Introduction+to+the+Special+Issue+on+Meta%2Dlearning%2E+Machine+learning%2C+54%283%29%3A187%E2%80%93193%2E)
8. Liling Tan, Marcos Zampieri, Nikola Ljubešić, and Jörg Tiedemann. 2014. Merging Comparable Data Sources for the Discrimination of Similar Languages: The DSL Corpus Collection. In Proceedings BUCC. [find](https://scholar.google.com/scholar?q=Liling+Tan%2C+Marcos+Zampieri%2C+Nikola+Ljubes%CC%8Cic%CC%81%2C+and+Jo%CC%88rg+Tiedemann%2E+2014%2E+Merging+Comparable+Data+Sources+for+the+Discrimination+of+Similar+Languages%3A+The+DSL+Corpus+Collection%2E+In+Proceedings+BUCC%2E)
9. Tommi Jauhiainen, Heidi Jauhiainen, Tero Alstola, and Krister Lindén. 2019. Language and Dialect Identification of Cuneiform Texts. In Proceedings of Var-Dial. [find](https://scholar.google.com/scholar?q=Tommi+Jauhiainen%2C+Heidi+Jauhiainen%2C+Tero+Alstola%2C+and+Krister+Linde%CC%81n%2E+2019%2E+Language+and+Dialect+Identification+of+Cuneiform+Texts%2E+In+Proceedings+of+Var%2DDial%2E)
10. Tommi Jauhiainen, Marco Lui, Marcos Zampieri, Timothy Baldwin, and Krister Lindén. 2018. Automatic language identification in texts: A survey. arXiv preprint arXiv:1804.08186. [↗](https://arxiv.org/abs/1804.08186) [our card](https://md.research.elamur.ai/free/en/papers/jauhiainen-2019-automatic-language-identification-texts/)
11. Dolf Trieschnigg, Djoerd Hiemstra, Mariët Theune, Franciska Jong, and Theo Meder. 2012. An Exploration of Language Identification Techniques in the Dutch Folktale Database. In Proceedings of the Workshop on Adaptation of Language Resources and Tools for Processing Cultural Heritage. [find](https://scholar.google.com/scholar?q=Dolf+Trieschnigg%2C+Djoerd+Hiemstra%2C+Marie%CC%88t+Theune%2C+Franciska+Jong%2C+and+Theo+Meder%2E+2012%2E+An+Exploration+of+Language+Identification+Techniques+in+the+Dutch+Folktale+Database%2E+In+Proceedings+of+the+Work)
12. Martin Kroon, Masha Medvedeva, and Barbara Plank. 2018. When Simple N-gram Models Outperform Syntactic Approaches: Discriminating between Dutch and Flemish. In Proceedings of Var-Dial. [find](https://scholar.google.com/scholar?q=Martin+Kroon%2C+Masha+Medvedeva%2C+and+Barbara+Plank%2E+2018%2E+When+Simple+N%2Dgram+Models+Outperform+Syntactic+Approaches%3A+Discriminating+between+Dutch+and+Flemish%2E+In+Proceedings+of+Var%2DDial%2E)
13. Marcos Zampieri, Shervin Malmasi, Nikola Ljubešić, Preslav Nakov, Ahmed Ali, Jörg Tiedemann, Yves Scherrer, and Noëmi Aepli. 2017. Findings of the VarDial Evaluation Campaign 2017. In Proceedings of VarDial. [our card](https://md.research.elamur.ai/free/en/papers/zampieri-2017-findings-vardial-evaluation-campaign/)
14. Wang Ling, Tiago Luı́s, Luı́s Marujo, Ramón Fernandez Astudillo, Silvio Amir, Chris Dyer, Alan W Black, and Isabel Trancoso. 2015. Finding function in form: Compositional character models for open vocabulary word representation. arXiv preprint arXiv:1508.02096. [↗](https://arxiv.org/abs/1508.02096) [our card](https://md.research.elamur.ai/free/en/papers/ling-2015-finding-function-form-compositional-character/)
15. Marcos Zampieri, Shervin Malmasi, Preslav Nakov, Ahmed Ali, Suwon Shuon, James Glass, Yves Scherrer, Tanja Samardžić, Nikola Ljubešić, Jörg Tiedemann, Chris van der Lee, Stefan Grondelaers, Nelleke Oostdijk, Antal van den Bosch, Ritesh Kumar, Bornini Lahiri, and Mayank Jain. 2018. Language Identification and Morphosyntactic Tagging: The Second VarDial Evaluation Campaign. In Proceedings of VarDial. [our card](https://md.research.elamur.ai/free/en/papers/zampieri-2018-language-identification-morphosyntactic-tagging-second/)
16. Shervin Malmasi and Marcos Zampieri. 2017a. Arabic Dialect Identification Using iVectors and ASR Transcripts. In Proceedings of VarDial. [find](https://scholar.google.com/scholar?q=Shervin+Malmasi+and+Marcos+Zampieri%2E+2017a%2E+Arabic+Dialect+Identification+Using+iVectors+and+ASR+Transcripts%2E+In+Proceedings+of+VarDial%2E)
17. Marcos Zampieri, Shervin Malmasi, Yves Scherrer, Tanja Samardžić, Francis Tyers, Miikka Silfverberg, Natalia Klyueva, Tung-Le Pan, Chu-Ren Huang, Radu Tudor Ionescu, Andrei Butnaru, and Tommi Jauhiainen. 2019. A Report on the Third VarDial Evaluation Campaign. In Proceedings of VarDial. [find](https://scholar.google.com/scholar?q=Marcos+Zampieri%2C+Shervin+Malmasi%2C+Yves+Scherrer%2C+Tanja+Samardz%CC%8Cic%CC%81%2C+Francis+Tyers%2C+Miikka+Silfverberg%2C+Natalia+Klyueva%2C+Tung%2DLe+Pan%2C+Chu%2DRen+Huang%2C+Radu+Tudor+Ionescu%2C+Andrei+Butnaru%2C+and+Tommi+Jauhia)
18. Shervin Malmasi and Marcos Zampieri. 2017b. German Dialect Identification in Interview Transcriptions. In Proceedings of VarDial. [find](https://scholar.google.com/scholar?q=Shervin+Malmasi+and+Marcos+Zampieri%2E+2017b%2E+German+Dialect+Identification+in+Interview+Transcriptions%2E+In+Proceedings+of+VarDial%2E)
19. Shervin Malmasi, Marcos Zampieri, and Mark Dras. 2016a. Predicting Post Severity in Mental Health Forums. In Proceedings of CLPsych. [find](https://scholar.google.com/scholar?q=Shervin+Malmasi%2C+Marcos+Zampieri%2C+and+Mark+Dras%2E+2016a%2E+Predicting+Post+Severity+in+Mental+Health+Forums%2E+In+Proceedings+of+CLPsych%2E)
20. Marcos Zampieri, Liling Tan, Nikola Ljubešić, and Jörg Tiedemann. 2014. A report on the DSL shared task 2014. In Proceedings of VarDial. [our card](https://md.research.elamur.ai/free/en/papers/zampieri-2014-report-dsl-shared-task/)
21. Shervin Malmasi, Marcos Zampieri, Nikola Ljubešić, Preslav Nakov, Ahmed Ali, and Jörg Tiedemann. 2016b. Discriminating between Similar Languages and Arabic Dialect Identification: A Report on the Third DSL Shared Task. In Proceedings of VarDial. [our card](https://md.research.elamur.ai/free/en/papers/malmasi-2016-discriminating-between-similar-languages/)
22. Marcos Zampieri, Liling Tan, Nikola Ljubešić, Jörg Tiedemann, and Preslav Nakov. 2015. Overview of the dsl shared task 2015. In Proceedings of LT4VarDial. [our card](https://md.research.elamur.ai/free/en/papers/zampieri-2015-overview-dsl-shared-task/)
23. Maria Medvedeva, Martin Kroon, and Barbara Plank. 2017. When Sparse Traditional Models Outperform Dense Neural Networks: The Curious Case of Discriminating between Similar Languages. In Proceedings of VarDial. [find](https://scholar.google.com/scholar?q=Maria+Medvedeva%2C+Martin+Kroon%2C+and+Barbara+Plank%2E+2017%2E+When+Sparse+Traditional+Models+Outperform+Dense+Neural+Networks%3A+The+Curious+Case+of+Discriminating+between+Similar+Languages%2E+In+Proceedings+of)
24. Arkaitz Zubiaga, Inaki San Vicente, Pablo Gamallo, José Ramom Pichel, Inaki Alegria, Nora Aranberri, Aitzol Ezeiza, and Vı́ctor Fresno. 2016. Tweetlid: A Benchmark for Tweet Language Identification. Language Resources and Evaluation, 50(4):729–766. [find](https://scholar.google.com/scholar?q=Arkaitz+Zubiaga%2C+Inaki+San+Vicente%2C+Pablo+Gamallo%2C+Jose%CC%81+Ramom+Pichel%2C+Inaki+Alegria%2C+Nora+Aranberri%2C+Aitzol+Ezeiza%2C+and+V%C4%B1%CC%81ctor+Fresno%2E+2016%2E+Tweetlid%3A+A+Benchmark+for+Tweet+Language+Identification%2E+)
25. Gustavo Paetzold. 2018. UTFPR at IEST 2018: Exploring Character-to-Word Composition for Emotion Analysis. In Proceedings of WASSA. [find](https://scholar.google.com/scholar?q=Gustavo+Paetzold%2E+2018%2E+UTFPR+at+IEST+2018%3A+Exploring+Character%2Dto%2DWord+Composition+for+Emotion+Analysis%2E+In+Proceedings+of+WASSA%2E)
26. Gustavo Paetzold. 2019. UTFPR at SemEval-2019 Task 6: Relying on Compositionality to Find Offense. In Proceedings of SemEval. [find](https://scholar.google.com/scholar?q=Gustavo+Paetzold%2E+2019%2E+UTFPR+at+SemEval%2D2019+Task+6%3A+Relying+on+Compositionality+to+Find+Offense%2E+In+Proceedings+of+SemEval%2E)