---
title: "ArCovidVac: Analyzing Arabic Tweets About COVID-19 Vaccination — Mubarak 2022"
description: "The authors built ArCovidVac, which they describe as the largest hand-annotated Arabic tweet collection… 63.1% F1. 47 references, cited by 3 papers here."
authors: ["Hamdy Mubarak", "Sabit Hassan", "Shammur Absar Chowdhury", "Firoj Alam"]
year: 2022
venue: "Proceedings of the Thirteenth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2022.lrec-1.344/"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mubarak-2022-arcovidvac-analyzing-arabic-tweets-about/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mubarak-2022-arcovidvac-analyzing-arabic-tweets-about/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/) / ArCovidVac: Analyzing Arabic Tweets About COVID-19 Vaccination

# ArCovidVac: Analyzing Arabic Tweets About COVID-19 Vaccination

**Authors:** [Hamdy Mubarak](https://md.research.elamur.ai/free/en/authors/hamdy-mubarak/), [Sabit Hassan](https://md.research.elamur.ai/free/en/authors/sabit-hassan/), [Shammur Absar Chowdhury](https://md.research.elamur.ai/free/en/authors/shammur-absar-chowdhury/) and [Firoj Alam](https://md.research.elamur.ai/free/en/authors/firoj-alam/) · **Year:** 2022 · **Venue:** Proceedings of the Thirteenth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.lrec-1.344/) · [PDF](https://aclanthology.org/2022.lrec-1.344.pdf) · [OpenAlex](https://openalex.org/W4221140942)

**Citations:** 10 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/mubarak-2022-arcovidvac-analyzing-arabic-tweets-about/cited-by/) in our library, of the 140,884 papers whose bibliographies we read

## Summary
Public discussion of COVID-19 vaccines on Arabic Twitter mixed useful guidance with rumours and conspiracy content. Manually labelled Arabic resources for this topic were scarce, which limited automatic monitoring for policymakers. The authors built ArCovidVac, which they describe as the largest hand-annotated Arabic tweet collection on vaccination campaigns, covering many countries in the Arab region. They gathered tweets from early 2021 using vaccine-related keywords and kept only those with notable engagement. They then labelled 10K tweets on two layers: ten fine-grained content types and a three-way stance toward vaccination. A binary informativeness label is derived automatically from the content types. Beyond the labels, they analysed the dataset itself: which vaccines were mentioned most, trending hashtags, topics, offensiveness, and how stance shifted over time. As benchmarks, they compared SVMs on word and character n-grams with fine-tuned AraBERT and QARiB. The transformer models clearly beat the SVMs on every task. Stance detection stayed the hardest task, with a best F1 of 63.1%. The resource can support multitask models and decision-making in public health.

## Key points
- Collection used the twarc search API between 5 Jan and 3 Feb 2021 and yielded 550K unique tweets. Keeping tweets with at least 10 likes or retweets left 14K, and 10K of those were randomly sampled for annotation.
- The ten content classes fall into two groups. Seven are treated as more informative: Advice, Plan, Restrictions, Info-news, Rumors, Requests and Celebrity. Three are treated as less informative: Personal, Unrelated and Others.
- Stance has three classes: pro-vaccination, neutral and anti-vaccination.
- Fine-tuning used the ktrain library with learning rate 8e-5, truncation at 54 tokens and 3 epochs.
  - AraBERT reached a top macro F1 of 80% in one benchmark.
  - On stance, QARiB was best at 63.1% F1, about 10 points above the SVMs.
- The authors manually reviewed 200 errors per task.
  - Plan was often confused with Info-news.
  - Neutral and negative stance tended to be predicted as the majority positive class.
  - Multi-label tweets accounted for about 10% of content-type errors.
  - Pro-vaccine stance rose when leaders such as kings were publicly vaccinated.

## Interesting ideas
- Annotators label only the fine-grained classes, and the coarse informativeness label is computed from them by a fixed mapping, so one pass yields a consistent label hierarchy for free.
- Engagement counts serve as a cheap importance filter before sampling for annotation, which concentrates human effort on content that actually spreads.
- Error review uses a fixed sample of 200 mistakes, sorted into causes: class confusion, annotation errors, multi-label cases and missing context. This separates gold-data problems from model problems.
- Stance errors came from question–answer pairs, negation, sarcasm and target-specific stance (rejecting one country's vaccine while wanting a safer one), showing that stance needs a target and the discourse structure.
- A stance timeline is aligned with real-world events, such as leaders getting vaccinated, to explain shifts in opinion.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): silver data could be tagged at the finest frame or role level, with coarser categories derived deterministically rather than labelled separately, which keeps the levels mutually consistent.
- **I2** (learn): a similar pre-filter, such as agreement or disagreement among witness parsers, could decide which sentences are sent to LLM teacher tiers or human review.
- **I3** (learn): evaluation reports could include a categorized error sample, so failures due to bad gold trees or frames are not mistaken for parser or reader weaknesses.
- **I4** (frames): stance could be modelled as a relation between a speaker and an explicit target frame, handling questions and their answers as speech acts rather than as polarity words.
- **I5** (world): attitudes of entities could be stored as time-stamped states linked to triggering events, so the system can explain why a character's view changed.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [🐪 Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/)

## Institutions

- [University of Pittsburgh](https://md.research.elamur.ai/free/en/universities/university-of-pittsburgh/) — University, Pittsburgh, United States · 32 works · 20,612 citations
- [Hamad bin Khalifa University](https://md.research.elamur.ai/free/en/universities/hamad-bin-khalifa-university/) — University, Doha, Qatar · 58 works · 11,454 citations

## Cite

Cite as: Hamdy Mubarak, Sabit Hassan, Shammur Absar Chowdhury and Firoj Alam. 2022. ArCovidVac: Analyzing Arabic Tweets About COVID-19 Vaccination. Proceedings of the Thirteenth Language Resources and Evaluation Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mubarak2022arcovidvac,
  title = {{ArCovidVac: Analyzing Arabic Tweets About COVID-19 Vaccination}},
  author = {Hamdy Mubarak, Sabit Hassan, Shammur Absar Chowdhury and Firoj Alam},
  booktitle = {Proceedings of the Thirteenth Language Resources and Evaluation Conference},
  year = {2022},
  doi = {10.63317/2zpc6rcxar5t},
  url = {https://doi.org/10.63317/2zpc6rcxar5t}
}
```

- [← previous The CODI-CRAC 2022 Shared Task on Anaphora, Bridging, and Discourse Deixis in Dialogue](https://md.research.elamur.ai/free/en/papers/yu-2022-codi-crac-2022-shared-task/)
- [next → Introducing BEREL: BERT Embeddings for Rabbinic-Encoded Language](https://md.research.elamur.ai/free/en/papers/shmidman-2022-introducing-berel-bert-embeddings-rabbinic/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/4/)

## References

47 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abdelali, A., Mubarak, H., Samih, Y., Hassan, S., and Darwish, K. (2020). Arabic dialect identification in the wild. arXiv, abs/2005.06557. [↗](https://arxiv.org/abs/2005.06557) [our card](https://md.research.elamur.ai/free/en/papers/abdelali-2020-arabic-dialect-identification-wild/)
2. Abdelali, A., Hassan, S., Mubarak, H., Darwish, K., and Samih, Y. (2021). Pre-training bert on arabic tweets: Practical considerations. arXiv: abs/2102.10684. [↗](https://arxiv.org/abs/2102.10684) [our card](https://md.research.elamur.ai/free/en/papers/abdelali-2021-pre-training-bert-arabic-tweets/)
3. Abdul-Mageed, M., Elmadany, A., Pabbi, D., Verma, K., and Lin, R. (2020). Mega-COV: A billion-scale dataset of 65 languages for COVID-19. arXiv:2005.06012. [↗](https://arxiv.org/abs/2005.06012)
4. Alam, F., Dalvi, F., Shaar, S., Durrani, N., Mubarak, H., Nikolov, A., Da San Martino, G., Abdelali, A., Sajjad, H., Darwish, K., and Nakov, P. (2021a). Fighting the COVID-19 infodemic in social media: A holistic perspective and a call to arms. In Proceedings of the International AAAI Conference on Web and Social Media, ICWSM ’21, pages 913– 922, Online. AAAI. [find](https://scholar.google.com/scholar?q=Alam%2C+F%2E%2C+Dalvi%2C+F%2E%2C+Shaar%2C+S%2E%2C+Durrani%2C+N%2E%2C+Mubarak%2C+H%2E%2C+Nikolov%2C+A%2E%2C+Da+San+Martino%2C+G%2E%2C+Abdelali%2C+A%2E%2C+Sajjad%2C+H%2E%2C+Darwish%2C+K%2E%2C+and+Nakov%2C+P%2E+%282021a%29%2E+Fighting+the+COVID%2D19+infodemic+in+social+media)
5. Alam, F., Shaar, S., Dalvi, F., Sajjad, H., Nikolov, A., Mubarak, H., Da San Martino, G., Abdelali, A., Durrani, N., Darwish, K., Al-Homaid, A., Zaghouani, W., Caselli, T., Danoe, G., Stolk, F., Bruntink, B., and Nakov, P. (2021b). Fighting the COVID-19 infodemic: Modeling the perspective of journalists, fact-checkers, social media platforms, policy makers, and the society. In Findings of the Association for Computational Linguistics, EMNLP (Findings) ’21, pages 611–649, Punta Cana, Dominican Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Alam%2C+F%2E%2C+Shaar%2C+S%2E%2C+Dalvi%2C+F%2E%2C+Sajjad%2C+H%2E%2C+Nikolov%2C+A%2E%2C+Mubarak%2C+H%2E%2C+Da+San+Martino%2C+G%2E%2C+Abdelali%2C+A%2E%2C+Durrani%2C+N%2E%2C+Darwish%2C+K%2E%2C+Al%2DHomaid%2C+A%2E%2C+Zaghouani%2C+W%2E%2C+Caselli%2C+T%2E%2C+Danoe%2C+G%2E%2C+Stolk%2C+F%2E%2C+Brunt)
6. Alqurashi, S., Alhindi, A., and Alanazi, E. (2020). Large arabic twitter dataset on COVID-19. arXiv preprint arXiv:2004.04315. [↗](https://arxiv.org/abs/2004.04315) [our card](https://md.research.elamur.ai/free/en/papers/alqurashi-2020-large-arabic-twitter-dataset-covid/)
7. Alshaabi, T., Dewhurst, D. R., Minot, J. R., Arnold, M. V., Adams, J. L., Danforth, C. M., and Dodds, P. S. (2021). The growing amplification of social media: Measuring temporal and social contagion dynamics for over 150 languages on twitter for 2009– 2020. EPJ data science, 10(1):15. [find](https://scholar.google.com/scholar?q=Alshaabi%2C+T%2E%2C+Dewhurst%2C+D%2E+R%2E%2C+Minot%2C+J%2E+R%2E%2C+Arnold%2C+M%2E+V%2E%2C+Adams%2C+J%2E+L%2E%2C+Danforth%2C+C%2E+M%2E%2C+and+Dodds%2C+P%2E+S%2E+%282021%29%2E+The+growing+amplification+of+social+media%3A+Measuring+temporal+and+social+contagion+d)
8. Alsudias, L. and Rayson, P. (2020). COVID-19 and Arabic Twitter: How can Arab world governments and public health organizations learn from social media? In Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Alsudias%2C+L%2E+and+Rayson%2C+P%2E+%282020%29%2E+COVID%2D19+and+Arabic+Twitter%3A+How+can+Arab+world+governments+and+public+health+organizations+learn+from+social+media%3F+In+Proceedings+of+the+1st+Workshop+on+NLP+for+C)
9. Antoun, W., Baly, F., and Hajj, H. (2020). AraBERT: Transformer-based model for Arabic language understanding. In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15, Marseille, France. European Language Resource Association. [our card](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/)
10. Baly, R., Mohtarami, M., Glass, J., Màrquez, L., Moschitti, A., and Nakov, P. (2018). Integrating stance detection and fact checking in a unified corpus. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 21–27, New Orleans, Louisiana. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/baly-2018-integrating-stance-detection-fact-checking/)
11. Banda, J. M., Tekumalla, R., Wang, G., Yu, J., Liu, T., Ding, Y., Artemova, E., Tutubalina, E., and Chowell, G. (2021). A large-scale COVID-19 twitter chatter dataset for open scientific research—an international collaboration. Epidemiologia, 2(3):315–324. [find](https://scholar.google.com/scholar?q=Banda%2C+J%2E+M%2E%2C+Tekumalla%2C+R%2E%2C+Wang%2C+G%2E%2C+Yu%2C+J%2E%2C+Liu%2C+T%2E%2C+Ding%2C+Y%2E%2C+Artemova%2C+E%2E%2C+Tutubalina%2C+E%2E%2C+and+Chowell%2C+G%2E+%282021%29%2E+A+large%2Dscale+COVID%2D19+twitter+chatter+dataset+for+open+scientific+research%E2%80%94an+i)
12. Chen, E., Lerman, K., and Ferrara, E. (2020). Tracking social media discourse about the COVID-19 pandemic: Development of a public coronavirus Twitter data set. JMIR Public Health Surveill, 6(2):e19273. [find](https://scholar.google.com/scholar?q=Chen%2C+E%2E%2C+Lerman%2C+K%2E%2C+and+Ferrara%2C+E%2E+%282020%29%2E+Tracking+social+media+discourse+about+the+COVID%2D19+pandemic%3A+Development+of+a+public+coronavirus+Twitter+data+set%2E+JMIR+Public+Health+Surveill%2C+6%282%29%3Ae1927)
13. Chowdhury, S. A., Ghosh, A., Stepanov, E. A., Bayer, A. O., Riccardi, G., and Klasinas, I. (2014). Cross-language transfer of semantic annotation via targeted crowdsourcing. In Proceedings of the 15th Annual Conference of the International Speech Communication Association, Singapore. ISCA. [find](https://scholar.google.com/scholar?q=Chowdhury%2C+S%2E+A%2E%2C+Ghosh%2C+A%2E%2C+Stepanov%2C+E%2E+A%2E%2C+Bayer%2C+A%2E+O%2E%2C+Riccardi%2C+G%2E%2C+and+Klasinas%2C+I%2E+%282014%29%2E+Cross%2Dlanguage+transfer+of+semantic+annotation+via+targeted+crowdsourcing%2E+In+Proceedings+of+the+15th)
14. Chowdhury, S. A., Calvo, M., Ghosh, A., Stepanov, E. A., Bayer, A. O., Riccardi, G., Garcı́a, F., and Sanchis, E. (2015). Selection and aggregation techniques for crowdsourced semantic annotation task. In Proceedings of the 16th Annual Conference of the International Speech Communication Association, Dresden, Germany. ISCA. [find](https://scholar.google.com/scholar?q=Chowdhury%2C+S%2E+A%2E%2C+Calvo%2C+M%2E%2C+Ghosh%2C+A%2E%2C+Stepanov%2C+E%2E+A%2E%2C+Bayer%2C+A%2E+O%2E%2C+Riccardi%2C+G%2E%2C+Garc%C4%B1%CC%81a%2C+F%2E%2C+and+Sanchis%2C+E%2E+%282015%29%2E+Selection+and+aggregation+techniques+for+crowdsourced+semantic+annotation+task)
15. Chowdhury, S. A., Abdelali, A., Darwish, K., Soon-Gyo, J., Salminen, J., and Jansen, B. J. (2020a). Improving arabic text categorization using transformer training diversification. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 226–236, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Chowdhury%2C+S%2E+A%2E%2C+Abdelali%2C+A%2E%2C+Darwish%2C+K%2E%2C+Soon%2DGyo%2C+J%2E%2C+Salminen%2C+J%2E%2C+and+Jansen%2C+B%2E+J%2E+%282020a%29%2E+Improving+arabic+text+categorization+using+transformer+training+diversification%2E+In+Proceedings+of+t)
16. Chowdhury, S. A., Mubarak, H., Abdelali, A., Jung, S.-g., Jansen, B. J., and Salminen, J. (2020b). A multi-platform Arabic news comment dataset for offensive language detection. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 6203–6212, Marseille, France. European Language Resources Association. [our card](https://md.research.elamur.ai/free/en/papers/chowdhury-2020-multi-platform-arabic-news-comment/)
17. Cinelli, M., Quattrociocchi, W., Galeazzi, A., Valensise, C. M., Brugnoli, E., Schmidt, A. L., Zola, P., Zollo, F., and Scala, A. (2020). The COVID-19 social media infodemic. Scientific reports, 10(1):1–10. [find](https://scholar.google.com/scholar?q=Cinelli%2C+M%2E%2C+Quattrociocchi%2C+W%2E%2C+Galeazzi%2C+A%2E%2C+Valensise%2C+C%2E+M%2E%2C+Brugnoli%2C+E%2E%2C+Schmidt%2C+A%2E+L%2E%2C+Zola%2C+P%2E%2C+Zollo%2C+F%2E%2C+and+Scala%2C+A%2E+%282020%29%2E+The+COVID%2D19+social+media+infodemic%2E+Scientific+reports%2C+10%281%29)
18. Devlin, J., Chang, M.-W., Lee, K., and Toutanova, K. (2019). BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota, June. Association for Computational Linguistics. [our card](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/)
19. Elhadad, M. K., Li, K. F., and Gebali, F. (2020). Covid-19-fakes: a twitter (arabic/english) dataset for detecting misleading information on covid-19. In International Conference on Intelligent Networking and Collaborative Systems, pages 256– 268. Springer. [find](https://scholar.google.com/scholar?q=Elhadad%2C+M%2E+K%2E%2C+Li%2C+K%2E+F%2E%2C+and+Gebali%2C+F%2E+%282020%29%2E+Covid%2D19%2Dfakes%3A+a+twitter+%28arabic%2Fenglish%29+dataset+for+detecting+misleading+information+on+covid%2D19%2E+In+International+Conference+on+Intelligent+Networ)
20. Haouari, F., Hasanain, M., Suwaileh, R., and Elsayed, T. (2021a). ArCOV-19: The first Arabic COVID-19 Twitter dataset with propagation networks. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 82–91, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Haouari%2C+F%2E%2C+Hasanain%2C+M%2E%2C+Suwaileh%2C+R%2E%2C+and+Elsayed%2C+T%2E+%282021a%29%2E+ArCOV%2D19%3A+The+first+Arabic+COVID%2D19+Twitter+dataset+with+propagation+networks%2E+In+Proceedings+of+the+Sixth+Arabic+Natural+Language+Pro)
21. Haouari, F., Hasanain, M., Suwaileh, R., and Elsayed, T. (2021b). ArCOV19-rumors: Arabic COVID-19 Twitter dataset for misinformation detection. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 72–81, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Haouari%2C+F%2E%2C+Hasanain%2C+M%2E%2C+Suwaileh%2C+R%2E%2C+and+Elsayed%2C+T%2E+%282021b%29%2E+ArCOV19%2Drumors%3A+Arabic+COVID%2D19+Twitter+dataset+for+misinformation+detection%2E+In+Proceedings+of+the+Sixth+Arabic+Natural+Language+Proc)
22. Hassan, S., Samih, Y., Mubarak, H., and Abdelali, A. (2020). ALT at SemEval-2020 task 12: Arabic and English offensive language identification in social media. In Proceedings of the Fourteenth Workshop on Semantic Evaluation, pages 1891– 1897, Barcelona (online). International Committee for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hassan%2C+S%2E%2C+Samih%2C+Y%2E%2C+Mubarak%2C+H%2E%2C+and+Abdelali%2C+A%2E+%282020%29%2E+ALT+at+SemEval%2D2020+task+12%3A+Arabic+and+English+offensive+language+identification+in+social+media%2E+In+Proceedings+of+the+Fourteenth+Worksho)
23. Hassan, S., Mubarak, H., Abdelali, A., and Darwish, K. (2021). ASAD: Arabic social media analytics and unDerstanding. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 113–118, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hassan%2C+S%2E%2C+Mubarak%2C+H%2E%2C+Abdelali%2C+A%2E%2C+and+Darwish%2C+K%2E+%282021%29%2E+ASAD%3A+Arabic+social+media+analytics+and+unDerstanding%2E+In+Proceedings+of+the+16th+Conference+of+the+European+Chapter+of+the+Association+f)
24. Hossain, T., Logan IV, R. L., Ugarte, A., Matsubara, Y., Young, S., and Singh, S. (2020). COVIDLies: Detecting COVID-19 misinformation on social media. In Proceedings of the 1st Workshop on NLP for COVID-19, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hossain%2C+T%2E%2C+Logan+IV%2C+R%2E+L%2E%2C+Ugarte%2C+A%2E%2C+Matsubara%2C+Y%2E%2C+Young%2C+S%2E%2C+and+Singh%2C+S%2E+%282020%29%2E+COVIDLies%3A+Detecting+COVID%2D19+misinformation+on+social+media%2E+In+Proceedings+of+the+1st+Workshop+on+NLP+for+CO)
25. Huang, B. and Carley, K. M. (2020). Disinformation and misinformation on twitter during the novel coronavirus outbreak. arXiv preprint arXiv:2006.04278. [↗](https://arxiv.org/abs/2006.04278)
26. Khouja, J. (2020). Stance prediction and claim verification: An Arabic perspective. In Proceedings of the Third Workshop on Fact Extraction and VERification (FEVER), pages 8–17, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Khouja%2C+J%2E+%282020%29%2E+Stance+prediction+and+claim+verification%3A+An+Arabic+perspective%2E+In+Proceedings+of+the+Third+Workshop+on+Fact+Extraction+and+VERification+%28FEVER%29%2C+pages+8%E2%80%9317%2C+Online%2E+Association+fo)
27. Li, L., Zhang, Q., Wang, X., Zhang, J., Wang, T., Gao, T., Duan, W., Tsoi, K. K., and Wang, F. (2020). Characterizing the propagation of situational information in social media during covid-19 epidemic: A case study on weibo. IEEE Transactions on Computational Social Systems, 7(2):556–562. [find](https://scholar.google.com/scholar?q=Li%2C+L%2E%2C+Zhang%2C+Q%2E%2C+Wang%2C+X%2E%2C+Zhang%2C+J%2E%2C+Wang%2C+T%2E%2C+Gao%2C+T%2E%2C+Duan%2C+W%2E%2C+Tsoi%2C+K%2E+K%2E%2C+and+Wang%2C+F%2E+%282020%29%2E+Characterizing+the+propagation+of+situational+information+in+social+media+during+covid%2D19+epidemi)
28. Maiya, A. S. (2020). ktrain: A low-code library for augmented machine learning. arXiv preprint arXiv:2004.10703. [↗](https://arxiv.org/abs/2004.10703)
29. Medford, R. J., Saleh, S. N., Sumarsono, A., Perl, T. M., and Lehmann, C. U. (2020). An ”infodemic”: Leveraging high-volume twitter data to understand public sentiment for the COVID-19 outbreak. medRxiv 2020.04.03.20052936. [find](https://scholar.google.com/scholar?q=Medford%2C+R%2E+J%2E%2C+Saleh%2C+S%2E+N%2E%2C+Sumarsono%2C+A%2E%2C+Perl%2C+T%2E+M%2E%2C+and+Lehmann%2C+C%2E+U%2E+%282020%29%2E+An+%E2%80%9Dinfodemic%E2%80%9D%3A+Leveraging+high%2Dvolume+twitter+data+to+understand+public+sentiment+for+the+COVID%2D19+outbreak%2E+medRx)
30. Mitchell, A. and Page, D. (2014). State of the news media 2014: Overview. Pew Research Center. [find](https://scholar.google.com/scholar?q=Mitchell%2C+A%2E+and+Page%2C+D%2E+%282014%29%2E+State+of+the+news+media+2014%3A+Overview%2E+Pew+Research+Center%2E)
31. Mubarak, H. and Darwish, K. (2016). Demographic surveys of arab annotators on crowdflower. In Proceedings of ACM WebSci16 Workshop “Weaving Relations of Trust in Crowd Work: Transparency and Reputation across Platforms, Hannover, Germany. [find](https://scholar.google.com/scholar?q=Mubarak%2C+H%2E+and+Darwish%2C+K%2E+%282016%29%2E+Demographic+surveys+of+arab+annotators+on+crowdflower%2E+In+Proceedings+of+ACM+WebSci16+Workshop+%E2%80%9CWeaving+Relations+of+Trust+in+Crowd+Work%3A+Transparency+and+Reputatio)
32. Mubarak, H. and Hassan, S. (2021a). ArCorona: Analyzing Arabic tweets in the early days of coronavirus (COVID-19) pandemic. In Proceedings of the 12th International Workshop on Health Text Mining and Information Analysis, pages 1–6, online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mubarak%2C+H%2E+and+Hassan%2C+S%2E+%282021a%29%2E+ArCorona%3A+Analyzing+Arabic+tweets+in+the+early+days+of+coronavirus+%28COVID%2D19%29+pandemic%2E+In+Proceedings+of+the+12th+International+Workshop+on+Health+Text+Mining+and+)
33. Mubarak, H. and Hassan, S. (2021b). UL2C: Mapping user locations to countries on Arabic Twitter. In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 145–153, Kyiv, Ukraine (Virtual). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mubarak%2C+H%2E+and+Hassan%2C+S%2E+%282021b%29%2E+UL2C%3A+Mapping+user+locations+to+countries+on+Arabic+Twitter%2E+In+Proceedings+of+the+Sixth+Arabic+Natural+Language+Processing+Workshop%2C+pages+145%E2%80%93153%2C+Kyiv%2C+Ukraine+%28)
34. Nagoudi, E. M. B., Elmadany, A., Abdul-Mageed, M., and Alhindi, T. (2020). Machine generation and detection of Arabic manipulated and fake news. In Proceedings of the Fifth Arabic Natural Language Processing Workshop, pages 69–84, Barcelona, Spain (Online). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Nagoudi%2C+E%2E+M%2E+B%2E%2C+Elmadany%2C+A%2E%2C+Abdul%2DMageed%2C+M%2E%2C+and+Alhindi%2C+T%2E+%282020%29%2E+Machine+generation+and+detection+of+Arabic+manipulated+and+fake+news%2E+In+Proceedings+of+the+Fifth+Arabic+Natural+Language+Pro)
35. Nakov, P., Da San Martino, G., Elsayed, T., Barrón-Cedeno, A., Mı́guez, R., Shaar, S., Alam, F., Haouari, F., Hasanain, M., Babulkov, N., et al. (2021a). The clef-2021 checkthat! lab on detecting check-worthy claims, previously fact-checked claims, and fake news. In Proceedings of the European Conference on Information Retrieval, ECIR ’21, pages 639–649, Online. Springer. [find](https://scholar.google.com/scholar?q=Nakov%2C+P%2E%2C+Da+San+Martino%2C+G%2E%2C+Elsayed%2C+T%2E%2C+Barro%CC%81n%2DCedeno%2C+A%2E%2C+M%C4%B1%CC%81guez%2C+R%2E%2C+Shaar%2C+S%2E%2C+Alam%2C+F%2E%2C+Haouari%2C+F%2E%2C+Hasanain%2C+M%2E%2C+Babulkov%2C+N%2E%2C+et+al%2E+%282021a%29%2E+The+clef%2D2021+checkthat%21+lab+on+detecting+che)
36. Nakov, P., Giovanni, D. S. M., Elsayed, T., Barrón-Cedeño, A., Mı́guez, R., Shaar, S., Alam, F., Haouari, F., Hasanain, M., Mansour, W., Hamdan, B., Ali, Z. S., Babulkov, N., Nikolov, A., Shahi, G. K., Struß, J. M., Mandl, T., Kutlu, M., and Kartal, Y. S. (2021b). Overview of the CLEF-2021 Check-That! lab on detecting check-worthy claims, previously fact-checked claims, and fake news. LNCS (12880). Springer. [find](https://scholar.google.com/scholar?q=Nakov%2C+P%2E%2C+Giovanni%2C+D%2E+S%2E+M%2E%2C+Elsayed%2C+T%2E%2C+Barro%CC%81n%2DCeden%CC%83o%2C+A%2E%2C+M%C4%B1%CC%81guez%2C+R%2E%2C+Shaar%2C+S%2E%2C+Alam%2C+F%2E%2C+Haouari%2C+F%2E%2C+Hasanain%2C+M%2E%2C+Mansour%2C+W%2E%2C+Hamdan%2C+B%2E%2C+Ali%2C+Z%2E+S%2E%2C+Babulkov%2C+N%2E%2C+Nikolov%2C+A%2E%2C+Shahi%2C+G%2E+)
37. Nakov, P., Barrón-Cedeño, A., Da San Martino, G., Alam, F., Struß, J. M., Mandl, T., Mı́guez, R., Caselli, T., Kutlu, M., Zaghouani, W., Li, C., Shaar, S., Shahi, G. K., Mubarak, H., Nikolov, A., Babulkov, N., Kartal, Y. S., and Beltrán, J. (2022). The CLEF-2022 CheckThat! Lab on fighting the covid-19 infodemic and fake news detection. In Matthias Hagen, et al., editors, Advances in Information Retrieval, pages 416–428, Cham. Springer International Publishing. [find](https://scholar.google.com/scholar?q=Nakov%2C+P%2E%2C+Barro%CC%81n%2DCeden%CC%83o%2C+A%2E%2C+Da+San+Martino%2C+G%2E%2C+Alam%2C+F%2E%2C+Stru%C3%9F%2C+J%2E+M%2E%2C+Mandl%2C+T%2E%2C+M%C4%B1%CC%81guez%2C+R%2E%2C+Caselli%2C+T%2E%2C+Kutlu%2C+M%2E%2C+Zaghouani%2C+W%2E%2C+Li%2C+C%2E%2C+Shaar%2C+S%2E%2C+Shahi%2C+G%2E+K%2E%2C+Mubarak%2C+H%2E%2C+Nikolov%2C+A%2E%2C+Ba)
38. Pulido, C. M., Villarejo-Carballido, B., Redondo-Sama, G., and Gómez, A. (2020). Covid-19 infodemic: More retweets for science-based information on coronavirus than for false information. International sociology, 35(4):377–392. [find](https://scholar.google.com/scholar?q=Pulido%2C+C%2E+M%2E%2C+Villarejo%2DCarballido%2C+B%2E%2C+Redondo%2DSama%2C+G%2E%2C+and+Go%CC%81mez%2C+A%2E+%282020%29%2E+Covid%2D19+infodemic%3A+More+retweets+for+science%2Dbased+information+on+coronavirus+than+for+false+information%2E+Internation)
39. Qazi, U., Imran, M., and Ofli, F. (2020a). Geocov19. SIGSPATIAL Special, 12(1):6–15. [find](https://scholar.google.com/scholar?q=Qazi%2C+U%2E%2C+Imran%2C+M%2E%2C+and+Ofli%2C+F%2E+%282020a%29%2E+Geocov19%2E+SIGSPATIAL+Special%2C+12%281%29%3A6%E2%80%9315%2E)
40. Qazi, U., Imran, M., and Ofli, F. (2020b). Geo-CoV19: A dataset of hundreds of millions of multilingual COVID-19 tweets with location information. SIGSPATIAL Special, 12(1):6–15. [find](https://scholar.google.com/scholar?q=Qazi%2C+U%2E%2C+Imran%2C+M%2E%2C+and+Ofli%2C+F%2E+%282020b%29%2E+Geo%2DCoV19%3A+A+dataset+of+hundreds+of+millions+of+multilingual+COVID%2D19+tweets+with+location+information%2E+SIGSPATIAL+Special%2C+12%281%29%3A6%E2%80%9315%2E)
41. Shaar, S., Alam, F., Da San Martino, G., Nikolov, A., Zaghouani, W., Nakov, P., and Feldman, A. (2021). Findings of the NLP4IF-2021 shared tasks on fighting the COVID-19 infodemic and censorship detection. In Proceedings of the Fourth Workshop on NLP for Internet Freedom: Censorship, Disinformation, and Propaganda, pages 82–92, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Shaar%2C+S%2E%2C+Alam%2C+F%2E%2C+Da+San+Martino%2C+G%2E%2C+Nikolov%2C+A%2E%2C+Zaghouani%2C+W%2E%2C+Nakov%2C+P%2E%2C+and+Feldman%2C+A%2E+%282021%29%2E+Findings+of+the+NLP4IF%2D2021+shared+tasks+on+fighting+the+COVID%2D19+infodemic+and+censorship+detec)
42. Shahi, G. K. and Nandini, D. (2020). FakeCovid – a multilingual cross-domain fact check news dataset for covid-19. In Workshop Proceedings of the 14th International AAAI Conference on Web and Social Media, Online. AAAI. [find](https://scholar.google.com/scholar?q=Shahi%2C+G%2E+K%2E+and+Nandini%2C+D%2E+%282020%29%2E+FakeCovid+%E2%80%93+a+multilingual+cross%2Ddomain+fact+check+news+dataset+for+covid%2D19%2E+In+Workshop+Proceedings+of+the+14th+International+AAAI+Conference+on+Web+and+Social+M)
43. Shahi, G. K., Dirkson, A., and Majchrzak, T. A. (2020). An exploratory study of covid-19 misinformation on twitter. arXiv preprint arXiv:2005.05710. [↗](https://arxiv.org/abs/2005.05710)
44. Song, X., Petrak, J., Jiang, Y., Singh, I., Maynard, D., and Bontcheva, K. (2020). Classification aware neural topic model and its application on a new COVID-19 disinformation corpus. arXiv:2006.03354. [↗](https://arxiv.org/abs/2006.03354)
45. Vidgen, B., Hale, S., Guest, E., Margetts, H., Broniatowski, D., Waseem, Z., Botelho, A., Hall, M., and Tromble, R. (2020). Detecting East Asian prejudice on social media. In Proceedings of the Fourth Workshop on Online Abuse and Harms, pages 162– 172, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Vidgen%2C+B%2E%2C+Hale%2C+S%2E%2C+Guest%2C+E%2E%2C+Margetts%2C+H%2E%2C+Broniatowski%2C+D%2E%2C+Waseem%2C+Z%2E%2C+Botelho%2C+A%2E%2C+Hall%2C+M%2E%2C+and+Tromble%2C+R%2E+%282020%29%2E+Detecting+East+Asian+prejudice+on+social+media%2E+In+Proceedings+of+the+Fourth)
46. Yang, Q., Alamro, H., Albaradei, S., Salhi, A., Lv, X., Ma, C., Alshehri, M., Jaber, I., Tifratene, F., Wang, W., et al. (2020). SenWave: Monitoring the global sentiments under the covid-19 pandemic. arXiv preprint arXiv:2006.10842. [↗](https://arxiv.org/abs/2006.10842)
47. Zhou, X., Mulay, A., Ferrara, E., and Zafarani, R. (2020). ReCOVery: A multimodal repository for COVID-19 news credibility research. In Mathieu d’Aquin, et al., editors, Proceedings of the 29th ACM International Conference on Information and Knowledge Management, pages 3205–3212, Online. ACM. [find](https://scholar.google.com/scholar?q=Zhou%2C+X%2E%2C+Mulay%2C+A%2E%2C+Ferrara%2C+E%2E%2C+and+Zafarani%2C+R%2E+%282020%29%2E+ReCOVery%3A+A+multimodal+repository+for+COVID%2D19+news+credibility+research%2E+In+Mathieu+d%E2%80%99Aquin%2C+et+al%2E%2C+editors%2C+Proceedings+of+the+29th+ACM+In)