---
title: "Improved Generalization of Arabic Text Classifiers — Khaddaj 2019"
description: "The authors present this as the first domain adaptation work for Arabic and suggest it could help bridge… 25 references, cited by 1 paper here."
authors: ["Alaa Khaddaj", "Hazem Hajj", "Wassim El-Hajj"]
year: 2019
venue: "Proceedings of the Fourth Arabic Natural Language Processing Workshop"
source: "https://aclanthology.org/W19-4618/"
doi: "10.18653/v1/w19-4618"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/khaddaj-2019-improved-generalization-arabic-text-classifiers/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/khaddaj-2019-improved-generalization-arabic-text-classifiers/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/7/) / Improved Generalization of Arabic Text Classifiers

# Improved Generalization of Arabic Text Classifiers

**Authors:** [Alaa Khaddaj](https://md.research.elamur.ai/free/en/authors/alaa-khaddaj/), [Hazem Hajj](https://md.research.elamur.ai/free/en/authors/hazem-hajj/) and [Wassim El-Hajj](https://md.research.elamur.ai/free/en/authors/wassim-el-hajj/) · **Year:** 2019 · **Venue:** [Proceedings of the Fourth Arabic Natural Language Processing Workshop](https://md.research.elamur.ai/free/en/venues/wanlp/) ([WANLP 2019](https://md.research.elamur.ai/free/en/venues/wanlp/2019/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/W19-4618/) · [PDF](https://aclanthology.org/W19-4618.pdf) · [DOI 10.18653/v1/w19-4618](https://doi.org/10.18653/v1/w19-4618) · [OpenAlex](https://openalex.org/W2970689370)

**Citations:** 9 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/khaddaj-2019-improved-generalization-arabic-text-classifiers/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses a gap in Arabic NLP: classifiers trained on one domain of text lose accuracy on another, and transfer techniques that are common for English had barely been tried for Arabic. The authors frame this as domain adaptation for sentiment analysis. A labelled source domain and an unlabelled target domain feed one network. They compare a linear SVM, a plain network, the adversarial DANN, and their own variant. The variant adds a denoising autoencoder branch to DANN, so the shared features must also be able to rebuild a masked version of the input. Experiments use ArSentD-LEV, a Levantine tweet corpus with 5-point sentiment labels. Domains are defined either by country of origin or by topic. Domain shift is measured with proxy A-distance (PAD). The autoencoder-augmented model generally beats DANN, and it gains most when the source and target domains are far apart. When training data is very small, the linear SVM still wins. The authors present this as the first domain adaptation work for Arabic and suggest it could help bridge Arabic dialects.

## Key points
- **Training objective.** The model minimises L_task + λ·L_domain + µ·L_recon, with λ and µ in [0, 1].
  - µ is kept small because the reconstruction error is orders of magnitude larger than the other two losses.
  - Both the task and domain losses use binary cross-entropy.
  - A Gradient Reversal Layer flips the domain gradient during backpropagation.
- **Input corruption.** The noisy input is produced by masking: each element is zeroed with probability p, and the decoder must recover the clean input.
- **Data and tasks.** ArSentD-LEV (Baly et al., 2018) is used in two settings:
  - cross-country adaptation;
  - cross-topic adaptation over politics, personal, religious, sports and other, giving 20 source→target tasks.
- **Cross-topic accuracy is low and uneven, and no model wins everywhere.**
  - On Politics→Personal, the proposed model scores 33.3, against 28.7 for DANN and 29.5 for SVM.
  - On Other→Religious, SVM reaches 53.2, ahead of DANN (41.8) and the proposed model (43.0).
- **Data-size study (Politics→Personal).** As the training set grows, the proposed model stays above DANN at every size, and both improve with more data.

## Interesting ideas
- Measuring domain distance with PAD, computed from how well a classifier can tell the two domains apart, makes it possible to predict where adaptation will pay off; here the gain grew with the distance.
- Adding a denoising reconstruction loss beside the main task acts as a regulariser, keeping representations general instead of narrowly task-tuned, and it helped most with little data.
- A single batch can mix labelled and unlabelled examples: only the labelled slice reaches the task head, while every example reaches the domain head.
- The authors report openly that a linear SVM beats deep models when data is scarce, which makes a simple baseline a necessary control in every adaptation table.
- Defining domains in two orthogonal ways over the same corpus, once by origin and once by topic, shows which kind of shift actually hurts a model.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Compute PAD between treebank genres and our target texts (fables, school tests, word problems) to rank where silver data or adaptation effort is most needed.
- **I2** (tree): gparse could add a masked-token reconstruction head trained on unlabelled in-domain text, so the parser adapts to new genres without new gold trees.
- **I3** (learn): Gold/silver sentences could train the parser head while raw target-genre sentences train only a domain or reconstruction head, using one pipeline for both.
- **I4** (learn): Keep the perceptron parser as a mandatory control next to gparse on each small domain, and report where the cheaper model wins.
- **I5** (answer): Split evaluation sets by both source genre and question type to see whether reader failures come from text style or from task kind.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [American University of Beirut](https://md.research.elamur.ai/free/en/universities/american-university-of-beirut/) — University, Beirut, Lebanon · 34 works · 2,561 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works

## Cite

Cite as: Alaa Khaddaj, Hazem Hajj and Wassim El-Hajj. 2019. Improved Generalization of Arabic Text Classifiers. Proceedings of the Fourth Arabic Natural Language Processing Workshop.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{khaddaj2019improved,
  title = {{Improved Generalization of Arabic Text Classifiers}},
  author = {Alaa Khaddaj, Hazem Hajj and Wassim El-Hajj},
  booktitle = {Proceedings of the Fourth Arabic Natural Language Processing Workshop},
  year = {2019},
  doi = {10.18653/v1/w19-4618},
  url = {https://doi.org/10.18653/v1/w19-4618}
}
```

- [← previous Historical Text Normalization with Delayed Rewards](https://md.research.elamur.ai/free/en/papers/flachs-2019-historical-text-normalization-delayed-rewards/)
- [next → Universal dependencies for Scottish Gaelic: syntax](https://md.research.elamur.ai/free/en/papers/batchelor-2019-universal-dependencies-scottish-gaelic-syntax/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/7/)

## References

25 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Gilbert Badaro, Ramy Baly, Hazem Hajj, Wassim El-Hajj, Khaled Shaban, Nizar Habash, Ahmad Sallab, and Ali Hamdi. 2019. A survey of opinion mining in arabic: A comprehensive system perspective covering challenges and advances in tools, resources, models, applications and visualizations. ACM Transactions on Asian Language Information Processing, 18. [find](https://scholar.google.com/scholar?q=Gilbert+Badaro%2C+Ramy+Baly%2C+Hazem+Hajj%2C+Wassim+El%2DHajj%2C+Khaled+Shaban%2C+Nizar+Habash%2C+Ahmad+Sallab%2C+and+Ali+Hamdi%2E+2019%2E+A+survey+of+opinion+mining+in+arabic%3A+A+comprehensive+system+perspective+covering)
2. Konstantinos Bousmalis, George Trigeorgis, Nathan Silberman, Dilip Krishnan, and Dumitru Erhan. 2016. Domain separation networks. In D. D. Lee, M. Sugiyama, U. V. Luxburg, I. Guyon, and R. Garnett, editors, Advances in Neural Information Processing Systems 29, pages 343–351. Curran Associates, Inc. [↗](https://doi.org/10.48550/arxiv.1608.06019)
3. Pierre Baldi. 2012. Autoencoders, unsupervised learning, and deep architectures. In Proceedings of ICML Workshop on Unsupervised and Transfer Learning, volume 27 of Proceedings of Machine Learning Research, pages 37–49, Bellevue, Washington, USA. PMLR. [find](https://scholar.google.com/scholar?q=Pierre+Baldi%2E+2012%2E+Autoencoders%2C+unsupervised+learning%2C+and+deep+architectures%2E+In+Proceedings+of+ICML+Workshop+on+Unsupervised+and+Transfer+Learning%2C+volume+27+of+Proceedings+of+Machine+Learning+Res)
4. Minmin Chen, Zhixiang Xu, Kilian Q. Weinberger, and Fei Sha. 2012. Marginalized denoising autoencoders for domain adaptation. In Proceedings of the 29th International Coference on International Conference on Machine Learning, ICML’12, pages 1627–1634, USA. Omnipress. [↗](https://doi.org/10.48550/arxiv.1206.4683)
5. Ramy Baly, Gilbert Badaro, Georges El-Khoury, Rawan Moukalled, Rita Aoun, Hazem Hajj, Wassim El-Hajj, Nizar Habash, and Khaled Shaban. 2017. [A characterization study of Arabic twitter data with a benchmarking for state-of-the-art opinion mining models](https://md.research.elamur.ai/free/en/papers/baly-2017-characterization-study-arabic-twitter-data/). In Proceedings of the Third Arabic Natural Language Processing Workshop, pages 110–118, Valencia, Spain. Association for Computational Linguistics.
6. Corinna Cortes and Vladimir Vapnik. 1995. [Support-vector networks](https://md.research.elamur.ai/free/en/papers/cortes-1995-support-vector-networks/). In Machine Learning, pages 273– 297. [↗](https://doi.org/10.1007/bf00994018)
7. Ramy Baly, Alaa Khaddaj, Hazem Hajj, Wassim El-Hajj, and Khaled Shaban. 2018. [Arsentd-lev: A multi-topic corpus for target-based sentiment analysis in arabic levantine tweets](https://md.research.elamur.ai/free/en/papers/baly-2019-arsentd-lev-multi-topic-corpus/). In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018), Paris, France. European Language Resources Association (ELRA). [↗](https://doi.org/10.48550/arxiv.1906.01830)
8. Yaroslav Ganin, Evgeniya Ustinova, Hana Ajakan, Pascal Germain, Hugo Larochelle, François Laviolette, Mario Marchand, and Victor Lempitsky. 2016. Domain-adversarial training of neural networks. J. Mach. Learn. Res., 17(1):2096–2030. [↗](https://doi.org/10.1007/978-3-319-58347-1_10)
9. Hal Daume III. 2007. [Frustratingly easy domain adaptation](https://md.research.elamur.ai/free/en/papers/iii-2007-frustratingly-easy-domain-adaptation/). In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 256–263, Prague, Czech Republic. Association for Computational Linguistics.
10. Xavier Glorot, Antoine Bordes, and Yoshua Bengio. 2011. Domain adaptation for large-scale sentiment classification: A deep learning approach. In Proceedings of the 28th International Conference on International Conference on Machine Learning, ICML’11, pages 513–520, USA. Omnipress. [find](https://scholar.google.com/scholar?q=Xavier+Glorot%2C+Antoine+Bordes%2C+and+Yoshua+Bengio%2E+2011%2E+Domain+adaptation+for+large%2Dscale+sentiment+classification%3A+A+deep+learning+approach%2E+In+Proceedings+of+the+28th+International+Conference+on+Int)
11. Shai Ben-David, John Blitzer, Koby Crammer, Alex Kulesza, Fernando Pereira, and Jennifer Wortman Vaughan. 2010. A theory of learning from different domains. Machine Learning, 79(1):151–175. [↗](https://doi.org/10.1007/s10994-009-5152-4)
12. Boqing Gong, Kristen Grauman, and Fei Sha. 2013. Connecting the dots with landmarks: Discriminatively learning domain-invariant features for unsupervised domain adaptation. In Proceedings of the 30th International Conference on Machine Learning, volume 28 of Proceedings of Machine Learning Research, pages 222–230, Atlanta, Georgia, USA. PMLR. [find](https://scholar.google.com/scholar?q=Boqing+Gong%2C+Kristen+Grauman%2C+and+Fei+Sha%2E+2013%2E+Connecting+the+dots+with+landmarks%3A+Discriminatively+learning+domain%2Dinvariant+features+for+unsupervised+domain+adaptation%2E+In+Proceedings+of+the+30th+)
13. Shai Ben-David, John Blitzer, Koby Crammer, and Fernando Pereira. 2007. Analysis of representations for domain adaptation. In B. Schölkopf, J. C. Platt, and T. Hoffman, editors, Advances in Neural Information Processing Systems 19, pages 137–144. MIT Press. [↗](https://doi.org/10.7551/mitpress/7503.003.0022)
14. Eric Tzeng, Judy Hoffman, Kate Saenko, and Trevor Darrell. 2017. Adversarial discriminative domain adaptation. In The IEEE Conference on Computer Vision and Pattern Recognition (CVPR). [↗](https://doi.org/10.48550/arxiv.1702.05464)
15. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. [http://www.deeplearningbook.org](http://www.deeplearningbook.org). [↗](http://www.deeplearningbook.org)
16. Ian J. Goodfellow, Jean Pouget-Abadie, Mehdi Mirza, Bing Xu, David Warde-Farley, Sherjil Ozair, Aaron Courville, and Yoshua Bengio. 2014. Generative adversarial nets. In Proceedings of the 27th International Conference on Neural Information Processing Systems - Volume 2, NIPS’14, pages 2672–2680, Cambridge, MA, USA. MIT Press. [find](https://scholar.google.com/scholar?q=Ian+J%2E+Goodfellow%2C+Jean+Pouget%2DAbadie%2C+Mehdi+Mirza%2C+Bing+Xu%2C+David+Warde%2DFarley%2C+Sherjil+Ozair%2C+Aaron+Courville%2C+and+Yoshua+Bengio%2E+2014%2E+Generative+adversarial+nets%2E+In+Proceedings+of+the+27th+Intern)
17. Pascal Vincent, Hugo Larochelle, Isabelle Lajoie, Yoshua Bengio, and Pierre-Antoine Manzagol. 2010. Stacked denoising autoencoders: Learning useful representations in a deep network with a local denoising criterion. J. Mach. Learn. Res., 11:3371– 3408. [find](https://scholar.google.com/scholar?q=Pascal+Vincent%2C+Hugo+Larochelle%2C+Isabelle+Lajoie%2C+Yoshua+Bengio%2C+and+Pierre%2DAntoine+Manzagol%2E+2010%2E+Stacked+denoising+autoencoders%3A+Learning+useful+representations+in+a+deep+network+with+a+local+denoi)
18. Katarzyna Janocha and Wojciech Czarnecki. 2017. On loss functions for deep neural networks in classification. 25. [↗](https://doi.org/10.4467/20838476si.16.004.6185)
19. Serena Jeblee, Weston Feely, Houda Bouamor, Alon Lavie, Nizar Habash, and Kemal Oflazer. 2014. [Domain and dialect adaptation for machine translation into egyptian arabic](https://md.research.elamur.ai/free/en/papers/jeblee-2014-domain-dialect-adaptation-machine-translation/). In ANLP@EMNLP.
20. Shafiq Joty, Preslav Nakov, Llus Mrquez, and Israa Jaradat. 2017. Cross-language learning with adversarial neural networks. pages 226–237. [find](https://scholar.google.com/scholar?q=Shafiq+Joty%2C+Preslav+Nakov%2C+Llus+Mrquez%2C+and+Israa+Jaradat%2E+2017%2E+Cross%2Dlanguage+learning+with+adversarial+neural+networks%2E+pages+226%E2%80%93237%2E)
21. Diederik Kingma and Jimmy Ba. 2014. method for stochastic optimization. Adam: A Mingsheng Long, Zhangjie Cao, Jianmin Wang, and Michael I Jordan. 2018. Conditional adversarial domain adaptation. In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, Advances in Neural Information Processing Systems 31, pages 1640–1650. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Diederik+Kingma+and+Jimmy+Ba%2E+2014%2E+method+for+stochastic+optimization%2E+Adam%3A+A+Mingsheng+Long%2C+Zhangjie+Cao%2C+Jianmin+Wang%2C+and+Michael+I+Jordan%2E+2018%2E+Conditional+adversarial+domain+adaptation%2E+In+S%2E)
22. Will Monroe, Spence Green, and Christopher D. Manning. 2014. [Word segmentation of informal arabic with domain adaptation](https://md.research.elamur.ai/free/en/papers/monroe-2014-word-segmentation-informal-arabic-domain/). In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 206–211. Association for Computational Linguistics.
23. Hong-Wei Ng, Viet Dung Nguyen, Vassilios Vonikakis, and Stefan Winkler. 2015. Deep learning for emotion recognition on small datasets using transfer learning. In Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, ICMI ’15, pages 443–449, New York, NY, USA. ACM. [find](https://scholar.google.com/scholar?q=Hong%2DWei+Ng%2C+Viet+Dung+Nguyen%2C+Vassilios+Vonikakis%2C+and+Stefan+Winkler%2E+2015%2E+Deep+learning+for+emotion+recognition+on+small+datasets+using+transfer+learning%2E+In+Proceedings+of+the+2015+ACM+on+Interna)
24. Sinno Jialin Pan, Xiaochuan Ni, Jian-Tao Sun, Qiang Yang, and Zheng Chen. 2010. Cross-domain sentiment classification via spectral feature alignment. In Proceedings of the 19th International Conference on World Wide Web, WWW ’10, pages 751–760, New York, NY, USA. ACM. [find](https://scholar.google.com/scholar?q=Sinno+Jialin+Pan%2C+Xiaochuan+Ni%2C+Jian%2DTao+Sun%2C+Qiang+Yang%2C+and+Zheng+Chen%2E+2010%2E+Cross%2Ddomain+sentiment+classification+via+spectral+feature+alignment%2E+In+Proceedings+of+the+19th+International+Conferenc)
25. David E. Rumelhart, Geoffrey E. Hinton, and Ronald J. Williams. 1988. Neurocomputing: Foundations of research. chapter Learning Representations by Back-propagating Errors, pages 696–699. MIT Press, Cambridge, MA, USA. [find](https://scholar.google.com/scholar?q=David+E%2E+Rumelhart%2C+Geoffrey+E%2E+Hinton%2C+and+Ronald+J%2E+Williams%2E+1988%2E+Neurocomputing%3A+Foundations+of+research%2E+chapter+Learning+Representations+by+Back%2Dpropagating+Errors%2C+pages+696%E2%80%93699%2E+MIT+Press%2C+Ca)

*Updated 1 November 2026*