--- title: "The ASMAT project – Arabic social media analysis tools (2014)" description: "Existing Arabic NLP tools handle Modern Standard Arabic (MSA) reasonably well but break down on dialects and social media text." year: 2014 venue: "Proceedings of the 17th Annual Conference of the European Association for Machine Translation" source: "https://aclanthology.org/2014.eamt-1.27/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/anon-2014-asmat-project-arabic-social-media/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/anon-2014-asmat-project-arabic-social-media/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2014](https://md.research.elamur.ai/free/en/papers/2014/page/3/) / The ASMAT project – Arabic social media analysis tools # The ASMAT project – Arabic social media analysis tools **Year:** 2014 · **Venue:** [Proceedings of the 17th Annual Conference of the European Association for Machine Translation](https://md.research.elamur.ai/free/en/venues/eamt/) ([EAMT 2014](https://md.research.elamur.ai/free/en/venues/eamt/2014/)) · **Publisher:** [European Association for Machine Translation](https://md.research.elamur.ai/free/en/publishers/european-association-for-machine-translation/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/2014.eamt-1.27/) · [PDF](https://aclanthology.org/2014.eamt-1.27.pdf) ## Summary Existing Arabic NLP tools handle Modern Standard Arabic (MSA) reasonably well but break down on dialects and social media text. ASMAT, led by Fatiha Sadat (UQAM) with the industrial partner NLP Technologies (Atefeh Farzindar), is a project to build language resources and tools for dialectal Arabic in that setting. It focuses on the Maghrebi dialects, where resources are very scarce, and has three goals: identifying the language and dialect of a text, mapping between dialects and MSA in both directions, and translating dialectal Arabic into English and French. By December 2013, with NSERC Engage funding, two pieces were in place. The first is a dialect identifier covering 18 Arabic varieties, which reached 98% overall accuracy on social media text. The second is a rule-based converter from Tunisian dialect to MSA. The planned next step is a hybrid rule-based and statistical MT system that translates Tunisian into French or English using MSA as a pivot language. The project matters because it lets standard-language tools reach informal, under-resourced varieties. ## Key points - The project has three tasks: (1) language and dialect identification, (2) dialect↔MSA mapping, and (3) MT from Arabic dialects into English and French, with an emphasis on Maghrebi dialects. - The dialect identifier is a Naive Bayes classifier over character bi-grams. It was trained on forum and blog data covering 18 Arabic dialects and reached about 98% overall accuracy on social media text. - A rule-based system already converts Tunisian dialect social media text into MSA. - The planned MT system is a hybrid of statistical and rule-based methods. It will translate Tunisian into French or English through MSA as a pivot. - The 2013 work was an industrial collaboration with NLP Technologies under an NSERC Engage grant, and further grants were expected from late 2014. ## Interesting ideas - A Naive Bayes model over character bi-grams is a cheap and transparent classifier that can still tell apart many closely related varieties with high accuracy, so it is a strong baseline before trying heavier models. - Identifying the variety sentence by sentence makes it possible to send each dialectal portion of a mixed text to its own pre-processing tools, instead of forcing one pipeline onto everything. - The project maps a non-standard variety onto the standard one with explicit rules, so that existing standard-language tools can be reused rather than rebuilt for every dialect. - Translating through a well-resourced pivot (MSA) avoids needing parallel data for every dialect–target pair, which is important when resources are scarce. - The project combines hand-written rules with statistical models when data is very limited, letting rules cover what the small corpora cannot. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (tokens): A small char-bigram classifier could label each input sentence's register (standard prose, chat-style, archaic, transcribed speech), and the tokenizer could then choose its settings based on that label. - **I2** (tree): Sentences flagged as non-standard could be sent to a dedicated normalization pass or parser configuration, and the routing decision would be logged as part of the explanation. - **I3** (tokens): A rule-based normalizer for informal English (spelling variants, abbreviations, missing apostrophes) placed before the tagger would let the existing en crate handle such text, and each rewrite would remain traceable. - **I4** (back): This supports translating through a single canonical meaning representation; a variety only needs to be mapped into the meta-world, not paired with every output language. - **I5** (learn): Self-induced or hand-written repair rules could seed silver data for low-resource text types, and statistical training would then generalize beyond the rules. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works 🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 1,484 works ## Cite Cite as: 2014. The ASMAT project – Arabic social media analysis tools. Proceedings of the 17th Annual Conference of the European Association for Machine Translation. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{anon2014asmat, title = {{The ASMAT project – Arabic social media analysis tools}}, booktitle = {Proceedings of the 17th Annual Conference of the European Association for Machine Translation}, year = {2014}, url = {https://aclanthology.org/2014.eamt-1.27/} } ``` - [← previous Resche Catherine, Economic Terms and Beyond: Capitalising on the Wealth of Notions. How Researchers in Specialised Varieties of English Can Benefit from Focusing on Terms](https://md.research.elamur.ai/free/en/papers/hardy-2014-resche-catherine-economic-terms-capitalising/) - [next → Validation Issues induced by an Automatic Pre-Annotation Mechanism in the Building of Non-projective Dependency Treebanks](https://md.research.elamur.ai/free/en/papers/lacroix-2014-validation-issues-induced-automatic-pre/) - [↑ back to papers of 2014](https://md.research.elamur.ai/free/en/papers/2014/page/3/) *Updated 24 October 2026*