--- title: "Towards a Corpus of Violence Acts in Arabic Social Media (2016)" description: "The authors build a resource to help human rights organisations find reports of abuse in… 0.8. agreement. 6 references, cited by 8 papers here." authors: ["Ayman Alhelbawy", "Udo Kruschwitz", "Massimo Poesio"] year: 2016 venue: "Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)" source: "https://aclanthology.org/L16-1257/" licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)" licence_url: "https://creativecommons.org/licenses/by-nc/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/alhelbawy-2016-corpus-violence-acts-arabic-social/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/alhelbawy-2016-corpus-violence-acts-arabic-social/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/4/) / Towards a Corpus of Violence Acts in Arabic Social Media # Towards a Corpus of Violence Acts in Arabic Social Media **Authors:** [Ayman Alhelbawy](https://md.research.elamur.ai/free/en/authors/ayman-alhelbawy/), [Udo Kruschwitz](https://md.research.elamur.ai/free/en/authors/udo-kruschwitz/) and [Massimo Poesio](https://md.research.elamur.ai/free/en/authors/massimo-poesio/) · **Year:** 2016 · **Venue:** [Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)](https://md.research.elamur.ai/free/en/venues/lrec/) ([LREC 2016](https://md.research.elamur.ai/free/en/venues/lrec/2016/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/) **Links:** [Publisher's page](https://aclanthology.org/L16-1257/) · [PDF](https://aclanthology.org/L16-1257.pdf) · [OpenAlex](https://openalex.org/W2577527184) **Citations:** 11 citations (OpenAlex, 2026-10-14) · [8](https://md.research.elamur.ai/free/en/papers/alhelbawy-2016-corpus-violence-acts-arabic-social/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary The authors build a resource to help human rights organisations find reports of abuse in Arab social media. Their sources are often in conflict zones, and most posts are written in dialects rather than MSA. Data collection had two stages. First, a keyword stream from the Twitter API used a hand-built list of 237 Arabic violence stems, expanded with orthographic variants and affixes; it gathered 557,576 tweets. Second, an advanced filter removed emoticon-bearing posts, very short posts, sexual adverts and near-duplicates, leaving 76,619 tweets. Of these, 20,151 were labelled on CrowdFlower. Each tweet received at least five judgements under a scheme of violence classes (HRA, crime, conflict, violence, accident, crisis, political opinion, plus "other"), whose definitions were refined in a pilot and reviewed by Minority Rights Group experts. Each label gets a confidence score weighted by annotator trust. Fleiss' kappa rises from 0.50 over all tweets to 0.85 for tweets with confidence ≥ 0.8. Agreement stays poor for Conflict and, below high confidence, for Crime. The corpus is freely released. ## Key points - Expanding the keyword list: - The 237 stems were multiplied by reversing Darwish's (2002) letter normalization, generating every spelling with the easily confused alef, final ya and final ta-marbuta/ha forms. - Common prefixes and suffixes were attached. - Malformed generated forms were then pruned by hand. - Advanced filter steps: - drop tweets containing emoticons; - drop tweets with fewer than 5 tokens; - remove sexual adverts using a 15-word list of advertiser terms and hashtags; - deduplicate by normalizing text (strip hashtags, line breaks and separators, apply letter normalization) and hashing it with SHA-1. - Annotation scheme development: - A pilot used 5 classes, 31 test questions, 1,025 tweets and 3 judgements per tweet. - The final job used the expanded class set, 206 test questions and at least 5 judgements per tweet, with more raters added when agreement was low. - Aggregate label distribution: - other 38%, opinion 21%, conflict 16%, HRA 12% (2,367 tweets), crime 7%; - accident, violence and crisis each 1–3%; - aggregate and individual-judgement distributions differ by only about ±2%. - Kappa by confidence threshold: - Overall kappa is 0.50 (all tweets), 0.78 (CS ≥ 0.7), 0.85 (≥ 0.8) and 0.99 (≥ 0.9). - Conflict never exceeds 0.37. - Crime jumps to 0.91 only at CS ≥ 0.9. - Two releases exist: the raw job report with duplicates and a cleaned set of unique tweets. ## Interesting ideas - Run a normalization rule backwards to enumerate all plausible surface spellings of a lexicon entry, so an exact-match filter can catch variants it would otherwise miss. - Score each candidate label as the trust-weighted share of the raters who chose it, then measure agreement separately in confidence buckets instead of reporting one global number. - Detect near-duplicates by stripping cosmetic material (hashtags, spacing, letter variants) before hashing, so trivially edited copies collapse to one key. - Run a small pilot with seeded gold questions, study which questions annotators missed and their objections, then revise the class inventory before the full job. - The HRA vs. Conflict confusion shows that the same verb ("killed", "martyr") gets different event labels depending on who the agent is (state forces vs. combatants) and whether the victim is a civilian. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (tokens): Inverting our spelling normalizers (British/American, hyphenation, apostrophe variants) could generate lookup keys for lexicon and gazetteer matching without a fuzzy matcher. - **I2** (learn): Weighting witness parsers by their measured reliability and reporting agreement per confidence band would show where silver data is safe to train on and where it is not. - **I3** (learn): A normalize-then-hash pass over training and test sentences would guard evaluation against leakage from lightly reformatted duplicates. - **I4** (learn): Seeding LLM-teacher batches with known-answer items and revising label schemes after a pilot would act as a negative control on teacher quality. - **I5** (frames): Event-type labels should be derived from role fillers (agent's authority status, victim type) in the frame rather than from the trigger word, which keeps the decision explainable. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of Sheffield](https://md.research.elamur.ai/free/en/universities/university-of-sheffield/) — University, Sheffield, United Kingdom · 113 works · 21,542 citations - [University of Essex](https://md.research.elamur.ai/free/en/universities/university-of-essex/) — University, Colchester, United Kingdom · 34 works · 3,537 citations Also on the byline: [Fayoum University](https://ror.org/023gzwx10). 📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works 🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works ## Cite Cite as: Ayman Alhelbawy, Udo Kruschwitz and Massimo Poesio. 2016. Towards a Corpus of Violence Acts in Arabic Social Media. Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{alhelbawy2016corpus, title = {{Towards a Corpus of Violence Acts in Arabic Social Media}}, author = {Ayman Alhelbawy, Udo Kruschwitz and Massimo Poesio}, booktitle = {Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)}, year = {2016}, doi = {10.63317/4rooy5xv6w96}, url = {https://doi.org/10.63317/4rooy5xv6w96} } ``` - [← previous Toward incremental dialogue act segmentation in fast-paced interactive dialogue systems](https://md.research.elamur.ai/free/en/papers/manuvinakurike-2016-incremental-dialogue-act-segmentation-fast/) - [next → A Dataset for Open Event Extraction in English](https://md.research.elamur.ai/free/en/papers/nguyen-2016-dataset-open-event-extraction-english/) - [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/4/) ## References 6 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Artstein, R. and Poesio, M. (2008). [Inter-coder agreement for computational linguistics](https://md.research.elamur.ai/free/en/papers/artstein-2008-survey-article-inter-coder-agreement/). Computational Linguistics, 34(4):555–596. [↗](https://doi.org/10.1162/coli.07-034-r2) 2. Darwish, K. (2002). Building a shallow arabic morphological analyzer in one day. In Proceedings of the ACL-02 Workshop on Computational Approaches to Semitic Languages, SEMITIC ’02, pages 1–8. ACL. [find](https://scholar.google.com/scholar?q=Darwish%2C+K%2E+%282002%29%2E+Building+a+shallow+arabic+morphological+analyzer+in+one+day%2E+In+Proceedings+of+the+ACL%2D02+Workshop+on+Computational+Approaches+to+Semitic+Languages%2C+SEMITIC+%E2%80%9902%2C+pages+1%E2%80%938%2E+ACL%2E) 3. Fleiss, J. L. (1971). Measuring nominal scale agreement among many raters. Psychological bulletin, 76(5):378. [↗](https://doi.org/10.1037/h0031619) 4. Landis, J. R. and Koch, G. G. (1977). The measurement of observer agreement for categorical data. biometrics, pages 159–174. [find](https://scholar.google.com/scholar?q=Landis%2C+J%2E+R%2E+and+Koch%2C+G%2E+G%2E+%281977%29%2E+The+measurement+of+observer+agreement+for+categorical+data%2E+biometrics%2C+pages+159%E2%80%93174%2E) 5. NIST. (April 1995). Secure hash standard. In Federal Information Processing Standard. FIBS Publication. [find](https://scholar.google.com/scholar?q=NIST%2E+%28April+1995%29%2E+Secure+hash+standard%2E+In+Federal+Information+Processing+Standard%2E+FIBS+Publication%2E) 6. Snow, R., O’Connor, B., Jurafsky, D., and Ng, A. Y. (2008). [Cheap and fast—but is it good?: Evaluating non-expert annotations for natural language tasks](https://md.research.elamur.ai/free/en/papers/snow-2008-cheap-fast-but-is-it/). In EMNLP, pages 254–263. ACL. Not for all individual classes though Note that it is impossible to have a truly objective judgement in many of these cases. *Updated 1 November 2026*