---
title: "Event-Based Knowledge MLM for Arabic Event Detection (2022)"
description: "Event detection for Arabic news has mostly relied on hand-built features such as TF-IDF, n-grams, POS and NER, or on rule… 21 references."
authors: ["Asma Yamani", "Amjad Alsulami", "Rabeah Al-Zaidy"]
year: 2022
venue: "Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP)"
source: "https://aclanthology.org/2022.wanlp-1.25/"
doi: "10.18653/v1/2022.wanlp-1.25"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yamani-2022-event-based-knowledge-mlm-arabic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yamani-2022-event-based-knowledge-mlm-arabic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/8/) / Event-Based Knowledge MLM for Arabic Event Detection

# Event-Based Knowledge MLM for Arabic Event Detection

**Authors:** [Asma Yamani](https://md.research.elamur.ai/free/en/authors/asma-yamani/), Amjad Alsulami and Rabeah Al-Zaidy · **Year:** 2022 · **Venue:** [Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP)](https://md.research.elamur.ai/free/en/venues/wanlp/) ([WANLP 2022](https://md.research.elamur.ai/free/en/venues/wanlp/2022/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.wanlp-1.25/) · [PDF](https://aclanthology.org/2022.wanlp-1.25.pdf) · [DOI 10.18653/v1/2022.wanlp-1.25](https://doi.org/10.18653/v1/2022.wanlp-1.25) · [OpenAlex](https://openalex.org/W4385572785)

**Citations:** 1 citations (OpenAlex, 2026-10-14)

## Summary
Event detection for Arabic news has mostly relied on hand-built features such as TF-IDF, n-grams, POS and NER, or on rule systems. Contextual transformers had barely been tried for it. The authors therefore ask whether pre-training itself can be tilted toward the event task. They built AraEvent, a manually annotated tweet corpus from four Twitter news accounts, labelled with 9 event types adapted from ACE2005. It has two parts collected in different months: AraEvent(November) and AraEvent(July).

From the November part they used the odds ratio to rank words most indicative of each event type. During MLM pre-training, these words are masked preferentially instead of masking tokens at random. The resulting EBK-BERT is then fine-tuned per event type and compared with a randomly masked counterpart. EBK masking helped every event type on same-period test data. It gave an average gain of 3.67% when the test data came from another period and fine-tuning used only 100 samples. This suggests that task-aware masking is a cheap way to inject domain knowledge into language models for lower-resource languages.

## Key points
- AraEvent has two subsets, AraEvent(November) and AraEvent(July), sourced from 4 popular Twitter news accounts and annotated with 9 ACE2005-inspired event types (e.g., Conflict, Justice, Life, Movement, Business).
- The masking lexicon comes from odds-ratio scores computed on AraEvent(November); the baseline is the standard random 15% masking used by BERT.
- On same-period test data (80:20 split), EBK-BERT beat random masking on all 9 types, with gains of roughly 0.56–3.645%.
- On the July data (a different period):
  - with full training sets, the average gain was 1.74%;
  - with 100 balanced training samples, the average gain was 3.67%, up to 6.25% for some types;
  - Conflict and Contact gained little, which the authors attribute to biased significant-word selection.
- Both resources are released on Hugging Face as Asma/AraEvent and Asma/EBK-BERT.
- Stated limitations:
  - the short collection window biases the lexicon toward events of that period;
  - lemmatizer errors leak duplicate forms into the word list;
  - results cover MSA only, not dialects.

## Interesting ideas
- Bias the MLM objective toward words a downstream task cares about, rather than masking uniformly, so pre-training effort concentrates where the task needs it.
- Use a simple, inspectable statistic (the odds ratio per class) to derive the list of event-indicative words, keeping the "knowledge" human-readable instead of buried in weights.
- Test on data from a different time period, not just a random split, to reveal whether gains survive topical drift. Pair this with a deliberately tiny training regime (100 samples), where knowledge injection matters most.
- The same trigger and entities can mean a real event or a merely planned one depending on tense, so detecting a trigger is not enough to assert that something happened.
- Upstream lemmatization errors silently split one word into several lexicon entries, distorting frequency-based selection.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): when pre-training or denoising gparse encoders, mask predicates and frame-evoking verbs more often so representations sharpen exactly where frame extraction depends on them.
- **I2** (world): build per-frame trigger lexicons from annotated sentences with odds ratios, and keep them as auditable entries beside VerbNet/FrameNet classes.
- **I3** (learn): add temporal or genre-shifted held-out sets plus low-data runs to evaluation, so improvements that only fit one collection window are exposed.
- **I4** (frames): attach tense/modality status to each extracted frame, so the world of the text separates events that happened from intended or future ones.
- **I5** (tokens): validate lemma consistency before any statistic-driven lexicon is built from en output, merging variant forms that should share a lemma.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [King Fahd University of Petroleum and Minerals](https://md.research.elamur.ai/free/en/universities/king-fahd-university-of-petroleum-and-minerals/) — University, Dhahran, Saudi Arabia · 52 works · 4,666 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [🔎 Information extraction](https://md.research.elamur.ai/free/en/topics/information-extraction/) · 167 works

## Cite

Cite as: Asma Yamani, Amjad Alsulami and Rabeah Al-Zaidy. 2022. Event-Based Knowledge MLM for Arabic Event Detection. Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{yamani2022event,
  title = {{Event-Based Knowledge MLM for Arabic Event Detection}},
  author = {Asma Yamani, Amjad Alsulami and Rabeah Al-Zaidy},
  booktitle = {Proceedings of the Seventh Arabic Natural Language Processing Workshop (WANLP)},
  year = {2022},
  doi = {10.18653/v1/2022.wanlp-1.25},
  url = {https://doi.org/10.18653/v1/2022.wanlp-1.25}
}
```

- [← previous Establishing a Baseline for Arabic Patents Classification: A Comparison of Twelve Approaches](https://md.research.elamur.ai/free/en/papers/omar-2022-establishing-baseline-arabic-patents-classification/)
- [next → Frequency-centroid features for word recognition of non-native English speakers](https://md.research.elamur.ai/free/en/papers/berjon-2022-frequency-centroid-features-word-recognition/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/8/)

## References

21 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahmed Abdelali, Kareem Darwish, Nadir Durrani, and Hamdy Mubarak. 2016. [Farasa: A fast and furious segmenter for Arabic](https://md.research.elamur.ai/free/en/papers/abdelali-2016-farasa-fast-furious-segmenter-arabic/). In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Demonstrations, pages 11–16, San Diego, California. Association for Computational Linguistics.
2. Ahmed Abdelali, Sabit Hassan, Hamdy Mubarak, Kareem Darwish, and Younes Samih. 2021. [Pre-training BERT on arabic tweets: Practical considerations](https://md.research.elamur.ai/free/en/papers/abdelali-2021-pre-training-bert-arabic-tweets/). CoRR, abs/2102.10684. [↗](https://arxiv.org/abs/2102.10684)
3. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2021. [ARBERT & MARBERT: Deep bidirectional transformers for Arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 7088–7105, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.acl-long.551)
4. ACE. 2005. (automatic content extraction) arabic annotation guidelines for events. Linguistic Data Consortium. [find](https://scholar.google.com/scholar?q=ACE%2E+2005%2E+%28automatic+content+extraction%29+arabic+annotation+guidelines+for+events%2E+Linguistic+Data+Consortium%2E)
5. Alaa Alharbi and Mark Lee. 2021. [Kawarith: an arabic twitter corpus for crisis events](https://md.research.elamur.ai/free/en/papers/alharbi-2021-kawarith-arabic-twitter-corpus-crisis/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 42–52.
6. Hind Almerekhi, Maram Hasanain, and Tamer Elsayed. 2016. Evetar: A new test collection for event detection in arabic tweets. In Proceedings of the 39th International ACM SIGIR conference on Research and Development in Information Retrieval, pages 689–692. [find](https://scholar.google.com/scholar?q=Hind+Almerekhi%2C+Maram+Hasanain%2C+and+Tamer+Elsayed%2E+2016%2E+Evetar%3A+A+new+test+collection+for+event+detection+in+arabic+tweets%2E+In+Proceedings+of+the+39th+International+ACM+SIGIR+conference+on+Research+a)
7. Nasser Alsaedi and Pete Burnap. 2015. Arabic event detection in social media. In Computational Linguistics and Intelligent Text Processing, pages 384–401, Cham. Springer International Publishing. [find](https://scholar.google.com/scholar?q=Nasser+Alsaedi+and+Pete+Burnap%2E+2015%2E+Arabic+event+detection+in+social+media%2E+In+Computational+Linguistics+and+Intelligent+Text+Processing%2C+pages+384%E2%80%93401%2C+Cham%2E+Springer+International+Publishing%2E)
8. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [AraBERT: Transformer-based model for Arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. [↗](https://doi.org/10.48550/arxiv.2003.00104)
9. Yubo Chen, Liheng Xu, Kang Liu, Daojian Zeng, and Jun Zhao. 2015. [Event extraction via dynamic multi-pooling convolutional neural networks](https://md.research.elamur.ai/free/en/papers/chen-2015-event-extraction-dynamic-multi-pooling/). In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 167–176.
10. Amina Chouigui, Oussama Ben Khiroun, and Bilel Elayeb. 2018. A tf-idf and co-occurrence based approach for events extraction from arabic news corpus. In International Conference on Applications of Natural Language to Information Systems, pages 272–280. Springer. [find](https://scholar.google.com/scholar?q=Amina+Chouigui%2C+Oussama+Ben+Khiroun%2C+and+Bilel+Elayeb%2E+2018%2E+A+tf%2Didf+and+co%2Doccurrence+based+approach+for+events+extraction+from+arabic+news+corpus%2E+In+International+Conference+on+Applications+of+Nat)
11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). Cite arxiv:1810.04805Comment: 13 pages. [↗](https://arxiv.org/abs/1810.04805)
12. Ibrahim Abu El-Khair. 2016. [1.5 billion words arabic corpus](https://md.research.elamur.ai/free/en/papers/el-khair-2016-1-5-billion-words-arabic/). CoRR, abs/1611.04033. [↗](https://arxiv.org/abs/1611.04033)
13. Maram Hasanain, Reem Suwaileh, Tamer Elsayed, Mucahid Kutlu, and Hind Almerekhi. 2017. Evetar: Building a large-scale multi-task test collection over arabic tweets. [↗](https://doi.org/10.1007/s10791-017-9325-7)
14. Go Inoue, Bashar Alhafni, Nurpeiis Baimukan, Houda Bouamor, and Nizar Habash. 2021. [The interplay of variant, size, and task type in Arabic pre-trained language models](https://md.research.elamur.ai/free/en/papers/inoue-2021-interplay-variant-size-task-type/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, Kyiv, Ukraine (Online). Association for Computational Linguistics.
15. Mandar Joshi, Danqi Chen, Yinhan Liu, Daniel S. Weld, Luke Zettlemoyer, and Omer Levy. 2019. [Spanbert: Improving pre-training by representing and predicting spans](https://md.research.elamur.ai/free/en/papers/joshi-2019-spanbert-improving-pre-training-representing/). CoRR, abs/1907.10529. [↗](https://arxiv.org/abs/1907.10529)
16. Kornraphop Kawintiranon and Lisa Singh. 2021. [Knowledge enhanced masked language model for stance detection](https://md.research.elamur.ai/free/en/papers/kawintiranon-2021-knowledge-enhanced-masked-language-model/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4725–4735, Online. Association for Computational Linguistics. [↗](https://doi.org/10.48448/a5gk-tk54)
17. Tommaso Caselli, Osman Mutlu, Angelo Basile, and Ali Hürriyetoğlu. 2021. Protest-er: Retraining bert for protest event extraction. In Proceedings of the 4th Workshop on Challenges and Applications of Automated Extraction of Socio-political Events from Text (CASE 2021), pages 12–19. [find](https://scholar.google.com/scholar?q=Tommaso+Caselli%2C+Osman+Mutlu%2C+Angelo+Basile%2C+and+Ali+H%C3%BCrriyetog%CC%86lu%2E+2021%2E+Protest%2Der%3A+Retraining+bert+for+protest+event+extraction%2E+In+Proceedings+of+the+4th+Workshop+on+Challenges+and+Applications+of)
18. Chen Lin, Timothy Miller, Dmitriy Dligach, Steven Bethard, and Guergana Savova. 2021. EntityBERT: Entity-centric masking strategy for model pretraining for the clinical domain. In Proceedings of the 20th Workshop on Biomedical Language Processing, pages 191–201, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Chen+Lin%2C+Timothy+Miller%2C+Dmitriy+Dligach%2C+Steven+Bethard%2C+and+Guergana+Savova%2E+2021%2E+EntityBERT%3A+Entity%2Dcentric+masking+strategy+for+model+pretraining+for+the+clinical+domain%2E+In+Proceedings+of+the+2)
19. Yubo Chen, Shulin Liu, Xiang Zhang, Kang Liu, and Jun Zhao. 2017. [Automatically labeled data generation for large scale event extraction](https://md.research.elamur.ai/free/en/papers/chen-2017-automatically-labeled-data-generation-large/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 409–419.
20. Shulin Liu, Yang Li, Feng Zhang, Tao Yang, and Xinpeng Zhou. 2019. Event detection without triggers. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 735–744. [find](https://scholar.google.com/scholar?q=Shulin+Liu%2C+Yang+Li%2C+Feng+Zhang%2C+Tao+Yang%2C+and+Xinpeng+Zhou%2E+2019%2E+Event+detection+without+triggers%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+of+the+Association+for+Computat)
21. Shulin Liu, Kang Liu, Shizhu He, and Jun Zhao. 2016. A probabilistic soft logic based approach to exploiting latent and global information in event classification. In AAAI. [find](https://scholar.google.com/scholar?q=Shulin+Liu%2C+Kang+Liu%2C+Shizhu+He%2C+and+Jun+Zhao%2E+2016%2E+A+probabilistic+soft+logic+based+approach+to+exploiting+latent+and+global+information+in+event+classification%2E+In+AAAI%2E)

*Updated 1 November 2026*