---
title: "Syntax-Ignorant N-gram Embeddings for Sentiment Analysis of"
description: "Arabic dialects vary in syntax and allow free word order. 25 references, cited by 2 papers here."
authors: ["Hala Mulki", "Hatem Haddad", "Mourad Gridach", "Ismail Babaoğlu"]
year: 2019
venue: "Proceedings of the Fourth Arabic Natural Language Processing Workshop"
source: "https://aclanthology.org/W19-4604/"
doi: "10.18653/v1/w19-4604"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mulki-2019-syntax-ignorant-n-gram-embeddings/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mulki-2019-syntax-ignorant-n-gram-embeddings/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/7/) / Syntax-Ignorant N-gram Embeddings for Sentiment Analysis of Arabic Dialects

# Syntax-Ignorant N-gram Embeddings for Sentiment Analysis of Arabic Dialects

**Authors:** [Hala Mulki](https://md.research.elamur.ai/free/en/authors/hala-mulki/), [Hatem Haddad](https://md.research.elamur.ai/free/en/authors/hatem-haddad/), [Mourad Gridach](https://md.research.elamur.ai/free/en/authors/mourad-gridach/) and [Ismail Babaoğlu](https://md.research.elamur.ai/free/en/authors/ismail-babaoglu/) · **Year:** 2019 · **Venue:** [Proceedings of the Fourth Arabic Natural Language Processing Workshop](https://md.research.elamur.ai/free/en/venues/wanlp/) ([WANLP 2019](https://md.research.elamur.ai/free/en/venues/wanlp/2019/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/W19-4604/) · [PDF](https://aclanthology.org/W19-4604.pdf) · [DOI 10.18653/v1/w19-4604](https://doi.org/10.18653/v1/w19-4604) · [OpenAlex](https://openalex.org/W2971281071)

**Citations:** 11 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/mulki-2019-syntax-ignorant-n-gram-embeddings/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Arabic dialects vary in syntax and allow free word order. A sentiment classifier that relies on ordered, syntax-aware sentence composition, such as a CNN or RecNN, may therefore work well for one dialect and poorly for another. Mulki et al. address this with Tw-StAR, a shallow feed-forward network that learns n-gram embeddings directly from sentiment-labelled data.

Each sentence is cut into fixed-length n-grams with a sliding window, and every n-gram inherits the polarity of its source sentence. The word vectors of an n-gram are then added together (SOWE), so word order and syntax are deliberately ignored. The network has a single hidden layer with hard-sigmoid activation and a softmax output for positive versus negative.

The model was tested on five datasets covering different dialects, including the Tunisian TSAC and the multi-dialect MDT. The learned embeddings beat word2vec, both doc2vec variants and hand-crafted baselines, and they came close to deeper neural systems. This suggests that cheap, order-free, sentiment-aware features can bridge dialect differences without heavy architectures or huge corpora.

## Key points
- **Tw-StAR layout:** embedding lookup → lambda layer (SOWE sum) → one hidden layer with hard sigmoid → softmax over two labels, encoded as [1,0] for positive and [0,1] for negative.
- **No pretrained vectors:** embeddings start from Glorot uniform initialization. The authors avoided existing Arabic vectors (Zahran et al.; Al-Rfou et al.) because these were built from MSA/Egyptian text and would cause OOV problems for Tunisian and Moroccan words.
- **Whole n-grams as input:** unlike SSWE or CBOW, the model is fed complete n-grams rather than corrupted ones, because the goal is semantic and sentiment relations, not syntactic context.
- **Baselines and results:**
  - Compared against word2vec, doc2vec (PV-DM and PV-DBoW) and hand-crafted systems.
  - Tw-StAR did not beat the baseline on MDT (three dialects), but it performed reasonably without the very large training corpus that baseline relied on.
- **Qualitative check:** t-SNE plots showed Tw-StAR placing positive Tunisian words near each other. PV-DBoW, by contrast, put some positive words next to negative ones.

## Interesting ideas
- Every fixed-window fragment inherits its sentence's label, which turns a few labelled sentences into many training examples cheaply.
- Training vectors with a supervised sentiment signal pulls apart antonyms that occur in identical contexts, which purely distributional vectors tend to merge.
- A deliberately order-blind additive model serves as a strong, cheap reference point for how much syntax actually contributes to a task.
- The authors declined pretrained resources built on a different language variety, anticipating OOV damage for the dialects they target.
- Inspecting nearest neighbours in the learned space exposed misplaced words that aggregate scores hide, and the authors propose building a lexicon from those distances.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Sentence-level silver labels could be pushed down to subtrees or clauses, giving the trainer more local examples while flagging the inheritance as weak evidence.
- **I2** (frames): Polarity and evaluation cues used for irony or implicature should come from labelled contrastive pairs (interesting/boring in the same frame), not from context similarity alone.
- **I3** (learn): Use an unordered sum-of-words scorer as a negative control, and accept a tree-based component only if it clearly beats this baseline on the same test.
- **I4** (tokens): Before reusing a lexicon or tagger trained on one English register (news vs. tales or school texts), measure its OOV rate on the target domain and train locally if coverage is poor.
- **I5** (world): Neighbour inspection could audit compiled knowledge such as verb classes or value words, surfacing entries whose learned neighbours contradict their ontology labels.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (3): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 1,035 works · [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works · [📐 Syntax & grammar formalisms](https://md.research.elamur.ai/free/en/topics/syntax-grammar/) · 221 works

💡 Concepts (1): [💡 N-gram](https://md.research.elamur.ai/free/en/concepts/n-gram/) · 26 works

## Cite

Cite as: Hala Mulki, Hatem Haddad, Mourad Gridach and Ismail Babaoğlu. 2019. Syntax-Ignorant N-gram Embeddings for Sentiment Analysis of Arabic Dialects. Proceedings of the Fourth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mulki2019syntax,
  title = {{Syntax-Ignorant N-gram Embeddings for Sentiment Analysis of Arabic Dialects}},
  author = {Hala Mulki, Hatem Haddad, Mourad Gridach and Ismail Babaoğlu},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Workshop},
  year = {2019},
  doi = {10.18653/v1/w19-4604},
  url = {https://doi.org/10.18653/v1/w19-4604}
}
```

- [← previous Sentiment Analysis for Arabic in Social Media Network: A Systematic Mapping Study](https://md.research.elamur.ai/free/en/papers/abo-2019-sentiment-analysis-arabic-social-media/)
- [next → Target Based Speech Act Classification in Political Campaign Text](https://md.research.elamur.ai/free/en/papers/subramanian-2019-target-based-speech-act-classification/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/7/)

## References

25 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pages 249–256. [find](https://scholar.google.com/scholar?q=Xavier+Glorot+and+Yoshua+Bengio%2E+2010%2E+Understanding+the+difficulty+of+training+deep+feedforward+neural+networks%2E+In+Proceedings+of+the+Thirteenth+International+Conference+on+Artificial+Intelligence+a)
2. Nawaf A. Abdulla, Nizar A. Ahmed, Mohammed A. Shehab, and Mahmoud Al-Ayyoub. 2013. Arabic sentiment analysis: Lexicon-based and corpus-based. In 2013 IEEE Jordan Conference on Applied Electrical Engineering and Computing Technologies (AEECT), pages 1–6. IEEE. [find](https://scholar.google.com/scholar?q=Nawaf+A%2E+Abdulla%2C+Nizar+A%2E+Ahmed%2C+Mohammed+A%2E+Shehab%2C+and+Mahmoud+Al%2DAyyoub%2E+2013%2E+Arabic+sentiment+analysis%3A+Lexicon%2Dbased+and+corpus%2Dbased%2E+In+2013+IEEE+Jordan+Conference+on+Applied+Electrical+Engin)
3. Mourad Gridach, Hatem Haddad, and Hala Mulki. 2017. Empirical evaluation of word representations on arabic sentiment analysis. In International Conference on Arabic Language Processing (ICALP), pages 147–158. Springer. [find](https://scholar.google.com/scholar?q=Mourad+Gridach%2C+Hatem+Haddad%2C+and+Hala+Mulki%2E+2017%2E+Empirical+evaluation+of+word+representations+on+arabic+sentiment+analysis%2E+In+International+Conference+on+Arabic+Language+Processing+%28ICALP%29%2C+pages+)
4. Sadam Al-Azani and El-Sayed M. El-Alfy. 2017. Hybrid deep learning for sentiment polarity determination of arabic microblogs. In International Conference on Neural Information Processing, pages 491– 500. Springer. [find](https://scholar.google.com/scholar?q=Sadam+Al%2DAzani+and+El%2DSayed+M%2E+El%2DAlfy%2E+2017%2E+Hybrid+deep+learning+for+sentiment+polarity+determination+of+arabic+microblogs%2E+In+International+Conference+on+Neural+Information+Processing%2C+pages+491%E2%80%93+5)
5. Caglar Gulcehre, Marcin Moczulski, Misha Denil, and Yoshua Bengio. 2016. Noisy activation functions. In International Conference on Machine Learning, pages 3059–3068. [find](https://scholar.google.com/scholar?q=Caglar+Gulcehre%2C+Marcin+Moczulski%2C+Misha+Denil%2C+and+Yoshua+Bengio%2E+2016%2E+Noisy+activation+functions%2E+In+International+Conference+on+Machine+Learning%2C+pages+3059%E2%80%933068%2E)
6. Rami Al-Rfou, Bryan Perozzi, and Steven Skiena. 2013. [Polyglot: Distributed word representations for multilingual nlp](https://md.research.elamur.ai/free/en/papers/rfou-2013-polyglot-distributed-word-representations-multilingual/). arXiv preprint arXiv:1307.1662. [↗](https://arxiv.org/abs/1307.1662)
7. Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III. 2015. [Deep unordered composition rivals syntactic methods for text classification](https://md.research.elamur.ai/free/en/papers/iyyer-2015-deep-unordered-composition-rivals-syntactic/). In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), volume 1, pages 1681–1691.
8. Ahmad Al Sallab, Hazem Hajj, Gilbert Badaro, Ramy Baly, Wassim El Hajj, and Khaled Bashir Shaban. 2015. [Deep learning models for sentiment analysis in arabic](https://md.research.elamur.ai/free/en/papers/sallab-2015-deep-learning-models-sentiment-analysis/). In Proceedings of the Second Workshop on Arabic Natural Language Processing, pages 9– 17.
9. Diederik P. Kingma and Jimmy Ba. 2014. [A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). abs/1412.6980. [↗](https://arxiv.org/abs/1412.6980)
10. Aziz A. Altowayan and Lixin Tao. 2016. Word embeddings for arabic sentiment analysis. In 2016 IEEE International Conference on Big Data (Big Data), pages 3820–3825. IEEE. Adam: CoRR, Quoc Le and Tomas Mikolov. 2014. [Distributed representations of sentences and documents](https://md.research.elamur.ai/free/en/papers/le-2014-distributed-representations-sentences-documents/). In International Conference on Machine Learning, pages 1188–1196.
11. Kristen E. Brustad. 2000. The Syntax of Spoken Arabic: A Comparative Study of Moroccan, Egyptian, Syrian, and Kuwaiti Dialects. ERIC. [find](https://scholar.google.com/scholar?q=Kristen+E%2E+Brustad%2E+2000%2E+The+Syntax+of+Spoken+Arabic%3A+A+Comparative+Study+of+Moroccan%2C+Egyptian%2C+Syrian%2C+and+Kuwaiti+Dialects%2E+ERIC%2E)
12. Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of machine learning research, 9(Nov):2579–2605. [find](https://scholar.google.com/scholar?q=Laurens+van+der+Maaten+and+Geoffrey+Hinton%2E+2008%2E+Visualizing+data+using+t%2Dsne%2E+Journal+of+machine+learning+research%2C+9%28Nov%29%3A2579%E2%80%932605%2E)
13. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. [Natural language processing (almost) from scratch](https://md.research.elamur.ai/free/en/papers/collobert-2011-natural-language-processing-almost-scratch/). Journal of Machine Learning Research, 12(Aug):2493–2537.
14. Salima Medhaffar, Fethi Bougares, Yannick Esteve, and Lamia Hadrich-Belguith. 2017. [Sentiment analysis of tunisian dialects: Linguistic ressources and experiments](https://md.research.elamur.ai/free/en/papers/medhaffar-2017-sentiment-analysis-tunisian-dialects-linguistic/). In Proceedings of the Third Arabic Natural Language Processing Workshop, pages 55– 61.
15. Abdelghani Dahou, Shengwu Xiong, Junwei Zhou, Mohamed Houcine Haddoud, and Pengfei Duan. 2016. [Word embeddings and convolutional neural network for arabic sentiment classification](https://md.research.elamur.ai/free/en/papers/dahou-2016-word-embeddings-convolutional-neural-network/). In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2418–2427.
16. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. [Distributed representations of words and phrases and their compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). In Advances in neural information processing systems, pages 3111–3119.
17. Samhaa R. El-Beltagy, Mona El kalamawy, and Abu Bakr Soliman. 2017. [Niletmrg at semeval-2017 task 4: Arabic sentiment analysis](https://md.research.elamur.ai/free/en/papers/beltagy-2017-niletmrg-semeval-2017-task-4/). In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 790–795. Association for Computational Linguistics.
18. Jeff Mitchell and Mirella Lapata. 2010. Composition in distributional models of semantics. Cognitive science, 34(8):1388–1429. [find](https://scholar.google.com/scholar?q=Jeff+Mitchell+and+Mirella+Lapata%2E+2010%2E+Composition+in+distributional+models+of+semantics%2E+Cognitive+science%2C+34%288%29%3A1388%E2%80%931429%2E)
19. Ahmed Mourad and Kareem Darwish. 2013. [Subjectivity and sentiment analysis of modern standard arabic and arabic microblogs](https://md.research.elamur.ai/free/en/papers/mourad-2013-subjectivity-sentiment-analysis-modern-standard/). In Proceedings of the 4th workshop on computational approaches to subjectivity, sentiment and social media analysis, pages 55–64.
20. Abdeljalil Elouardighi, Mohcine Maghfour, Hafdalla Hammia, and Fatima-zahra Aazi. 2017. A machine learning approach for sentiment analysis in the standard or dialectal arabic facebook comments. In 3rd International Conference of Cloud Computing Technologies and Applications (CloudTech), pages 1–8. IEEE. [find](https://scholar.google.com/scholar?q=Abdeljalil+Elouardighi%2C+Mohcine+Maghfour%2C+Hafdalla+Hammia%2C+and+Fatima%2Dzahra+Aazi%2E+2017%2E+A+machine+learning+approach+for+sentiment+analysis+in+the+standard+or+dialectal+arabic+facebook+comments%2E+In+3rd)
21. Mahmoud Nabil, Mohamed Aly, and Amir Atiya. 2015. [Astd: Arabic sentiment tweets dataset](https://md.research.elamur.ai/free/en/papers/nabil-2015-astd-arabic-sentiment-tweets-dataset/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2515–2519.
22. Karim Sayadi, Marcus Liwicki, Rolf Ingold, and Marc Bui. 2016. Tunisian dialect and modern standard arabic dataset for sentiment analysis : Tunisian election context. In To appear in the ACLing 2016 IEEE proceedings. CICLING. [find](https://scholar.google.com/scholar?q=Karim+Sayadi%2C+Marcus+Liwicki%2C+Rolf+Ingold%2C+and+Marc+Bui%2E+2016%2E+Tunisian+dialect+and+modern+standard+arabic+dataset+for+sentiment+analysis+%3A+Tunisian+election+context%2E+In+To+appear+in+the+ACLing+2016+I)
23. Duyu Tang, Furu Wei, Nan Yang, Ming Zhou, Ting Liu, and Bing Qin. 2014. [Learning sentiment-specific word embedding for twitter sentiment classification](https://md.research.elamur.ai/free/en/papers/tang-2014-learning-sentiment-specific-word-embedding/). In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 1555– 1565.
24. Lyndon White, Roberto Togneri, Wei Liu, and Mohammed Bennamoun. 2015. How well sentence embeddings capture meaning. In Proceedings of the 20th Australasian Document Computing Symposium, page 9. ACM. [find](https://scholar.google.com/scholar?q=Lyndon+White%2C+Roberto+Togneri%2C+Wei+Liu%2C+and+Mohammed+Bennamoun%2E+2015%2E+How+well+sentence+embeddings+capture+meaning%2E+In+Proceedings+of+the+20th+Australasian+Document+Computing+Symposium%2C+page+9%2E+ACM%2E)
25. Mohamed A. Zahran, Ahmed Magooda, Ashraf Y. Mahgoub, Hazem Raafat, Mohsen Rashwan, and Amir Atyia. 2015. Word representations in vector space and their applications for arabic. In International Conference on Intelligent Text Processing and Computational Linguistics, pages 430–443. Springer. [find](https://scholar.google.com/scholar?q=Mohamed+A%2E+Zahran%2C+Ahmed+Magooda%2C+Ashraf+Y%2E+Mahgoub%2C+Hazem+Raafat%2C+Mohsen+Rashwan%2C+and+Amir+Atyia%2E+2015%2E+Word+representations+in+vector+space+and+their+applications+for+arabic%2E+In+International+Confer)

*Updated 2 November 2026*