---
title: "A Second Pandemic? Analysis of Fake News about COVID-19"
description: "The paper looks at how COVID-19 and vaccine discussion on Twitter in Qatar varies… 66.7% accuracy. 68 references, cited by 13 papers here."
authors: ["Preslav Nakov", "Firoj Alam", "Shaden Shaar", "Giovanni Da San Martino", "Yifan Zhang"]
year: 2021
venue: "Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)"
source: "https://aclanthology.org/2021.ranlp-1.114/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/nakov-2021-second-pandemic-analysis-fake-news/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nakov-2021-second-pandemic-analysis-fake-news/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/6/) / A Second Pandemic? Analysis of Fake News about COVID-19 Vaccines in Qatar

# A Second Pandemic? Analysis of Fake News about COVID-19 Vaccines in Qatar

**Authors:** [Preslav Nakov](https://md.research.elamur.ai/free/en/authors/preslav-nakov/), [Firoj Alam](https://md.research.elamur.ai/free/en/authors/firoj-alam/), [Shaden Shaar](https://md.research.elamur.ai/free/en/authors/shaden-shaar/), [Giovanni Da San Martino](https://md.research.elamur.ai/free/en/authors/giovanni-da-san-martino/) and [Yifan Zhang](https://md.research.elamur.ai/free/en/authors/yifan-zhang/) · **Year:** 2021 · **Venue:** [Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)](https://md.research.elamur.ai/free/en/venues/ranlp/) ([RANLP 2021](https://md.research.elamur.ai/free/en/venues/ranlp/2021/)) · **Publisher:** [INCOMA Ltd.](https://md.research.elamur.ai/free/en/publishers/incoma-ltd/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.ranlp-1.114/) · [PDF](https://aclanthology.org/2021.ranlp-1.114.pdf) · [OpenAlex](https://openalex.org/W3213176494)

**Citations:** 20 citations (OpenAlex, 2026-10-14) · [13](https://md.research.elamur.ai/free/en/papers/nakov-2021-second-pandemic-analysis-fake-news/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper looks at how COVID-19 and vaccine discussion on Twitter in Qatar varies by language, and whether it carries false claims, manipulative rhetoric or particular perspectives. The authors gathered original tweets in Arabic and English (no retweets or replies). They removed near-duplicates, dropped very short posts and kept the most-engaged tweets. The resulting corpus has 606 Arabic and 2,589 English items, with the English part split into a pre-vaccine period and a vaccine-era period. They then ran three groups of existing or lightly trained analyzers:
- an SVM over the seven-question disinformation schema of Alam et al.,
- the Proppy and Prta propaganda detectors,
- the Tanbih BERT-based framing model.

The Arabic posts carried more false claims and rumours. The English posts were mostly factual but far more propagandistic, with loaded language as the leading technique. Arabic tweets mostly took a health-and-safety angle, while English ones centred on the economy. The study matters as a template: off-the-shelf multi-aspect tools can profile a regional infodemic over time and across languages. The data is released publicly.

## Key points
- Corpus sizes: 606 Arabic tweets (February 2020 to March 2021) and 2,589 English tweets.
  - 644 English tweets come from February to August 2020.
  - 1,945 English tweets come from November 2020 to January 2021.
- Disinformation classifiers: SVMs were trained on 504 English and 218 Arabic tweets from the Alam et al. seven-question schema, with hyper-parameters tuned by 10-fold cross-validation.
  - Accuracy on Q1 (verifiable claim) was 64.5 for English and 72.5 for Arabic.
  - The harder multiclass questions scored roughly 37–70.
- Propaganda tools:
  - Proppy is a maximum-entropy model with style features, trained on 51k articles, reaching F1 82.89. Its score is mapped to five likelihood bins.
  - Prta covers 18 techniques; its sentence-level classifier reaches F1 60.71.
  - Arabic tweets were machine-translated into English with the Google translation API before running both tools.
- Propaganda results: about 33% of English tweets were propagandistic in the first period and about 25% in the vaccine period, while Arabic propagandistic tweets were extremely rare. Among Arabic techniques, doubt appeared in about 50% of tweets and loaded language in about 20%.
- Framing: the Tanbih model was fine-tuned on the Media Frames Corpus, predicts 15 frames, and has 66.7% accuracy.
  - Arabic tweets led with Health and safety, then Economy, then Cultural identity.
  - English tweets led with Economy in both periods.
  - The authors tentatively attribute this to locals versus expats.

## Interesting ideas
- Instead of one "fake or not" label, a post is judged through several narrower questions: does it contain a checkable claim, is it likely false, is it of public interest, is it harmful, should a fact-checker or policymaker look at it. This turns a vague verdict into separate, inspectable decisions.
- The propaganda score is a probability, but it is reported as five ordinal bins from "very unlikely" to "very likely". This gives users a coarse, readable confidence instead of a raw number.
- Prta works at two granularities: a whole-sentence propaganda flag and fragment-level spans tagged with one of 18 named techniques. Labels are anchored to the exact text that triggers them.
- To reuse English-only tools on Arabic, the authors simply machine-translate first and then analyze. This is a cheap bridge whose cost is the unmeasured noise translation adds.
- The same pipeline is applied to two separate time windows to see whether the distributions stay stable (they largely did for factuality). Changes in propaganda share then become visible as drift.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): A speech-act layer could attach such separate judgments to each asserted proposition (claim present, hedged, alarming, call to action) rather than one global label, keeping each decision explainable.
- **I2** (answer): Readers could report their confidence in an answer as a small ordinal scale tied to score ranges, which is easier to audit and to compare across questions.
- **I3** (frames): Rhetorical devices such as loaded language, doubt or exaggeration could be marked on specific subtrees of the UD parse, so irony or implicature decisions point to the words that caused them.
- **I4** (back): Translation through the meta-world could be checked the same way: run the English analyzers on translated input and compare their outputs with the source-side meaning graph to expose losses introduced by translation.
- **I5** (learn): Evaluation could rerun fixed analyzers on corpora from different periods or genres as a stability check, flagging components whose label distributions shift unexpectedly.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Hamad bin Khalifa University](https://md.research.elamur.ai/free/en/universities/hamad-bin-khalifa-university/) — University, Doha, Qatar · 144 works · 14,175 citations
- [University of Padua](https://md.research.elamur.ai/free/en/universities/university-of-padua/) — University, Padua, Italy · 18 works · 4,260 citations

📚 Shelves (4): [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 626 works · [🤝 Communication](https://md.research.elamur.ai/free/en/shelves/social-sciences/communication/) · 104 works · [🤝 Health](https://md.research.elamur.ai/free/en/shelves/social-sciences/health/) · 6 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works

## Cite

Cite as: Preslav Nakov, Firoj Alam, Shaden Shaar, Giovanni Da San Martino and Yifan Zhang. 2021. A Second Pandemic? Analysis of Fake News about COVID-19 Vaccines in Qatar. Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{nakov2021second,
  title = {{A Second Pandemic? Analysis of Fake News about COVID-19 Vaccines in Qatar}},
  author = {Preslav Nakov, Firoj Alam, Shaden Shaar, Giovanni {Da San Martino} and Yifan Zhang},
  booktitle = {Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021)},
  year = {2021},
  doi = {10.26615/978-954-452-072-4\_114},
  url = {https://doi.org/10.26615/978-954-452-072-4_114}
}
```

- [← previous Focus on what matters: Applying Discourse Coherence Theory to Cross Document Coreference](https://md.research.elamur.ai/free/en/papers/held-2021-focus-what-matters-applying-discourse/)
- [next → ERNIE-ViLG: Unified Generative Pre-training for Bidirectional Vision-Language Generation](https://md.research.elamur.ai/free/en/papers/zhang-2021-ernie-vilg-unified-generative/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/6/)

## References

68 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Firoj Alam, Fahim Dalvi, Shaden Shaar, Nadir Durrani, Hamdy Mubarak, Alex Nikolov, Giovanni Da San Martino, Ahmed Abdelali, Hassan Sajjad, Kareem Darwish, and Preslav Nakov. 2021a. Fighting the COVID-19 infodemic in social media: A holistic perspective and a call to arms. In Proceedings of the International AAAI Conference on Web and Social Media, ICWSM ’21, pages 913–922. [↗](https://doi.org/10.48550/arxiv.2007.07996)
2. Alberto Barrón-Cedeño, Giovanni Da San Martino, Israa Jaradat, and Preslav Nakov. 2019. Proppy: A system to unmask propaganda in online news. In Proceedings of the Thirty-Third AAAI Conference on Artificial Intelligence, AAAI’19, pages 9847–9848, Honolulu, HI, USA. [find](https://scholar.google.com/scholar?q=Alberto+Barro%CC%81n%2DCeden%CC%83o%2C+Giovanni+Da+San+Martino%2C+Israa+Jaradat%2C+and+Preslav+Nakov%2E+2019%2E+Proppy%3A+A+system+to+unmask+propaganda+in+online+news%2E+In+Proceedings+of+the+Thirty%2DThird+AAAI+Conference+on+Ar)
3. Firoj Alam, Hassan Sajjad, Muhammad Imran, and Ferda Ofli. 2021b. CrisisBench: Benchmarking crisis-related social media datasets for humanitarian information processing. In Proceedings of the International AAAI Conference on Web and Social Media, ICWSM ’21, pages 923–932. [↗](https://doi.org/10.48550/arxiv.2004.06774)
4. Alberto Barrón-Cedeño, Tamer Elsayed, Preslav Nakov, Giovanni Da San Martino, Maram Hasanain, Reem Suwaileh, and Fatima Haouari. 2020. Check-That! at CLEF 2020: Enabling the automatic identification and verification of claims in social media. In Proceedings of the 42nd European Conference on Information Retrieval, ECIR ’19, pages 499–507, Lisbon, Portugal. [find](https://scholar.google.com/scholar?q=Alberto+Barro%CC%81n%2DCeden%CC%83o%2C+Tamer+Elsayed%2C+Preslav+Nakov%2C+Giovanni+Da+San+Martino%2C+Maram+Hasanain%2C+Reem+Suwaileh%2C+and+Fatima+Haouari%2E+2020%2E+Check%2DThat%21+at+CLEF+2020%3A+Enabling+the+automatic+identification)
5. Firoj Alam, Shaden Shaar, Fahim Dalvi, Hassan Sajjad, Alex Nikolov, Hamdy Mubarak, Giovanni Da San Martino, Ahmed Abdelali, Nadir Durrani, Kareem Darwish, Abdulaziz Al-Homaid, Wajdi Zaghouani, Tommaso Caselli, Gijs Danoe, Friso Stolk, Britt Bruntink, and Preslav Nakov. 2021c. Fighting the COVID-19 infodemic: Modeling the perspective of journalists, fact-checkers, social media platforms, policy makers, and the society. In Findings of EMNLP 2021. [↗](https://doi.org/10.48550/arxiv.2005.00033)
6. Phoebe Arnold. 2020. The challenges of online fact checking. Technical report, Full Fact. [find](https://scholar.google.com/scholar?q=Phoebe+Arnold%2E+2020%2E+The+challenges+of+online+fact+checking%2E+Technical+report%2C+Full+Fact%2E)
7. Pepa Atanasova, Preslav Nakov, Lluı́s Màrquez, Alberto Barrón-Cedeño, Georgi Karadzhov, Tsvetomila Mihaylova, Mitra Mohtarami, and James Glass. 2019. Automatic fact-checking using context and discourse information. Journal of Data and Information Quality (JDIQ), 11(3):1–27. [↗](https://doi.org/10.1145/3297722)
8. Isabelle Augenstein, Christina Lioma, Dongsheng Wang, Lucas Chaves Lima, Casper Hansen, Christian Hansen, and Jakob Grue Simonsen. 2019. [MultiFC: A real-world multi-domain dataset for evidence-based fact checking of claims](https://md.research.elamur.ai/free/en/papers/augenstein-2019-multifc-real-world-multi-domain/). In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP ’19, pages 4685–4697, Hong Kong, China. [↗](https://doi.org/10.48550/arxiv.1909.03242)
9. Ramy Baly, Georgi Karadzhov, Jisun An, Haewoon Kwak, Yoan Dinkov, Ahmed Ali, James Glass, and Preslav Nakov. 2020. What was written vs. who read it: News media profiling using text analysis and social media context. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL ’20, pages 3364–3374. [find](https://scholar.google.com/scholar?q=Ramy+Baly%2C+Georgi+Karadzhov%2C+Jisun+An%2C+Haewoon+Kwak%2C+Yoan+Dinkov%2C+Ahmed+Ali%2C+James+Glass%2C+and+Preslav+Nakov%2E+2020%2E+What+was+written+vs%2E+who+read+it%3A+News+media+profiling+using+text+analysis+and+social)
10. Ramy Baly, Georgi Karadzhov, Abdelrhman Saleh, James Glass, and Preslav Nakov. 2019. Multi-task ordinal regression for jointly predicting the trustworthiness and the leading political ideology of news media. In Proceedings of the 17th Annual Conference of the North American Chapter of the Association for Computational Linguistics, Alberto Barrón-Cedeno, Israa Jaradat, Giovanni Da San Martino, and Preslav Nakov. 2019. Proppy: Organizing the news based on their propagandistic content. Information Processing & Management, 56(5):1849–1864. [↗](https://doi.org/10.48550/arxiv.1904.00542)
11. Amber E Boydstun, Justin H Gross, Philip Resnik, and Noah A Smith. 2013. Identifying media frames and frame dynamics within and across policy issues. In New Directions in Analyzing Text as Data Workshop. [find](https://scholar.google.com/scholar?q=Amber+E+Boydstun%2C+Justin+H+Gross%2C+Philip+Resnik%2C+and+Noah+A+Smith%2E+2013%2E+Identifying+media+frames+and+frame+dynamics+within+and+across+policy+issues%2E+In+New+Directions+in+Analyzing+Text+as+Data+Worksh)
12. Dallas Card, Amber E. Boydstun, Justin H. Gross, Philip Resnik, and Noah A. Smith. 2015. [The Media Frames Corpus: Annotations of frames across issues](https://md.research.elamur.ai/free/en/papers/card-2015-media-frames-corpus-annotations-frames/). In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, ACL-IJCNLP ’15, pages 438–444, Beijing, China.
13. Loretta H Cheeks, Tracy L Stepien, Dara M Wald, and Ashraf Gaffar. 2020. Discovering news frames: An approach for exploring text, content, and concepts in online news sources. In Cognitive Analytics: Concepts, Methodologies, Tools, and Applications, pages 702–721. IGI Global. [find](https://scholar.google.com/scholar?q=Loretta+H+Cheeks%2C+Tracy+L+Stepien%2C+Dara+M+Wald%2C+and+Ashraf+Gaffar%2E+2020%2E+Discovering+news+frames%3A+An+approach+for+exploring+text%2C+content%2C+and+concepts+in+online+news+sources%2E+In+Cognitive+Analytics%3A+)
14. Anton Chernyavskiy, Dmitry Ilvovsky, and Preslav Nakov. 2021. Transformers: “The end of history” for NLP? In Proceedings of the European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases, ECML-PKDD’21, Bilbao, Spain (virtual). [find](https://scholar.google.com/scholar?q=Anton+Chernyavskiy%2C+Dmitry+Ilvovsky%2C+and+Preslav+Nakov%2E+2021%2E+Transformers%3A+%E2%80%9CThe+end+of+history%E2%80%9D+for+NLP%3F+In+Proceedings+of+the+European+Conference+on+Machine+Learning+and+Principles+and+Practice+of+K)
15. Matteo Cinelli, Walter Quattrociocchi, Alessandro Galeazzi, Carlo Michele Valensise, Emanuele Brugnoli, Ana Lucia Schmidt, Paola Zola, Fabiana Zollo, and Antonio Scala. 2020. The COVID-19 social media infodemic. Sci Rep., 10:16598. [↗](https://doi.org/10.1038/s41598-020-73510-5)
16. Giovanni Da San Martino, Alberto Barron-Cedeno, and Preslav Nakov. 2019. Findings of the NLP4IF-2019 shared task on fine-grained propaganda detection. In Proceedings of the 2nd Workshop on NLP for Internet Freedom (NLP4IF): Censorship, Disinformation, and Propaganda, NLP4IF ’19, pages 162–170, Hong Kong, China. [↗](https://doi.org/10.48550/arxiv.1910.09982)
17. Giovanni Da San Martino, Stefano Cresci, Alberto Barrón-Cedeño, Seunghak Yu, Roberto Di Pietro, and Preslav Nakov. 2020a. A survey on computational propaganda detection. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI ’20, pages 4826–4832. [find](https://scholar.google.com/scholar?q=Giovanni+Da+San+Martino%2C+Stefano+Cresci%2C+Alberto+Barro%CC%81n%2DCeden%CC%83o%2C+Seunghak+Yu%2C+Roberto+Di+Pietro%2C+and+Preslav+Nakov%2E+2020a%2E+A+survey+on+computational+propaganda+detection%2E+In+Proceedings+of+the+Twenty)
18. Giovanni Da San Martino, Shaden Shaar, Yifan Zhang, Seunghak Yu, Alberto Barrón-Cedeno, and Preslav Nakov. 2020b. Prta: A system to support the analysis of propaganda techniques in the news. In Proceedings of the Annual Meeting of Association for Computational Linguistics, ACL ’20, pages 287– 293. [↗](https://doi.org/10.48550/arxiv.2005.05854)
19. Giovanni Da San Martino, Seunghak Yu, Alberto Barrón-Cedeño, Rostislav Petrov, and Preslav Nakov. 2019. [Fine-grained analysis of propaganda in news articles](https://md.research.elamur.ai/free/en/papers/martino-2019-fine-grained-analysis-propaganda-news/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP ’19, pages 5636–5646, Hong Kong, China. [↗](https://doi.org/10.48550/arxiv.1910.02517)
20. Leon Derczynski, Kalina Bontcheva, Maria Liakata, Rob Procter, Geraldine Wong Sak Hoi, and Arkaitz Zubiaga. 2017. [SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours](https://md.research.elamur.ai/free/en/papers/derczynski-2017-semeval-2017-task-8-rumoureval/). In Proceedings of the 11th International Workshop on Semantic Evaluation, SemEval ’17, pages 60–67, Vancouver, Canada.
21. Dimitar Dimitrov, Bishr Bin Ali, Shaden Shaar, Firoj Alam, Fabrizio Silvestri, Hamed Firooz, Preslav Nakov, and Giovanni Da San Martino. 2021a. Detecting propaganda techniques in memes. In Proceedings of the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL-IJCNLP ’21, pages 6603–6617. [↗](https://doi.org/10.48550/arxiv.2109.08013)
22. Dimiter Dimitrov, Bishr Bin Ali, Shaden Shaar, Firoj Alam, Fabrizio Silvestri, Hamed Firooz, Preslav Nakov, and Giovanni Da San Martino. 2021b. Task 6 at SemEval-2021: Detection of persuasion techniques in texts and images. In Proceedings of the 15th International Workshop on Semantic Evaluation, SemEval ’21, pages 70–98. [find](https://scholar.google.com/scholar?q=Dimiter+Dimitrov%2C+Bishr+Bin+Ali%2C+Shaden+Shaar%2C+Firoj+Alam%2C+Fabrizio+Silvestri%2C+Hamed+Firooz%2C+Preslav+Nakov%2C+and+Giovanni+Da+San+Martino%2E+2021b%2E+Task+6+at+SemEval%2D2021%3A+Detection+of+persuasion+techniqu)
23. Tamer Elsayed, Preslav Nakov, Alberto Barrón-Cedeño, Maram Hasanain, Reem Suwaileh, Pepa Atanasova, and Giovanni Da San Martino. 2019. CheckThat! at CLEF 2019: Automatic identification and verification of claims. In Proceedings of the 41st European Conference on Information Retrieval, ECIR ’19, pages 309–315, Cologne, Germany. [find](https://scholar.google.com/scholar?q=Tamer+Elsayed%2C+Preslav+Nakov%2C+Alberto+Barro%CC%81n%2DCeden%CC%83o%2C+Maram+Hasanain%2C+Reem+Suwaileh%2C+Pepa+Atanasova%2C+and+Giovanni+Da+San+Martino%2E+2019%2E+CheckThat%21+at+CLEF+2019%3A+Automatic+identification+and+verificat)
24. Robert M Entman. 1993. Framing: Towards clarification of a fractured paradigm. McQuail’s reader in mass communication theory, pages 390–397. [find](https://scholar.google.com/scholar?q=Robert+M+Entman%2E+1993%2E+Framing%3A+Towards+clarification+of+a+fractured+paradigm%2E+McQuail%E2%80%99s+reader+in+mass+communication+theory%2C+pages+390%E2%80%93397%2E)
25. Pepa Gencheva, Preslav Nakov, Lluı́s Màrquez, Alberto Barrón-Cedeño, and Ivan Koychev. 2017. A context-aware approach for detecting worth-checking claims in political debates. In Proceedings of the International Conference on Recent Advances in Natural Language Processing, RANLP ’17, pages 267–276, Varna, Bulgaria. [find](https://scholar.google.com/scholar?q=Pepa+Gencheva%2C+Preslav+Nakov%2C+Llu%C4%B1%CC%81s+Ma%CC%80rquez%2C+Alberto+Barro%CC%81n%2DCeden%CC%83o%2C+and+Ivan+Koychev%2E+2017%2E+A+context%2Daware+approach+for+detecting+worth%2Dchecking+claims+in+political+debates%2E+In+Proceedings+of+the)
26. Genevieve Gorrell, Elena Kochkina, Maria Liakata, Ahmet Aker, Arkaitz Zubiaga, Kalina Bontcheva, and Leon Derczynski. 2019. [SemEval-2019 task 7: RumourEval, determining rumour veracity and support for rumours](https://md.research.elamur.ai/free/en/papers/gorrell-2019-semeval-2019-task-7-rumoureval/). In Proceedings of the 13th International Workshop on Semantic Evaluation, SemEval ’19, pages 845–854, Minneapolis, MN, USA.
27. Ivan Habernal, Raffael Hannemann, Christian Pollak, Christopher Klamm, Patrick Pauli, and Iryna Gurevych. 2017. Argotario: Computational argumentation meets serious games. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, EMNLP ’17, pages 7–12, Copenhagen, Denmark. [find](https://scholar.google.com/scholar?q=Ivan+Habernal%2C+Raffael+Hannemann%2C+Christian+Pollak%2C+Christopher+Klamm%2C+Patrick+Pauli%2C+and+Iryna+Gurevych%2E+2017%2E+Argotario%3A+Computational+argumentation+meets+serious+games%2E+In+Proceedings+of+the+2017+C)
28. Ivan Habernal, Patrick Pauli, and Iryna Gurevych. 2018. Adapting serious game for fallacious argumentation to German: Pitfalls, insights, and best practices. In Proceedings of the 11th International Conference on Language Resources and Evaluation, LREC ’18, pages 3329–3335, Miyazaki, Japan. [find](https://scholar.google.com/scholar?q=Ivan+Habernal%2C+Patrick+Pauli%2C+and+Iryna+Gurevych%2E+2018%2E+Adapting+serious+game+for+fallacious+argumentation+to+German%3A+Pitfalls%2C+insights%2C+and+best+practices%2E+In+Proceedings+of+the+11th+International+C)
29. Momchil Hardalov, Ivan Koychev, and Preslav Nakov. 2016. In search of credible news. In Proceedings of the 17th International Conference on Artificial Intelligence: Methodology, Systems, and Applications, AIMSA ’16, pages 172–180, Varna, Bulgaria. [find](https://scholar.google.com/scholar?q=Momchil+Hardalov%2C+Ivan+Koychev%2C+and+Preslav+Nakov%2E+2016%2E+In+search+of+credible+news%2E+In+Proceedings+of+the+17th+International+Conference+on+Artificial+Intelligence%3A+Methodology%2C+Systems%2C+and+Applicati)
30. Maram Hasanain, Fatima Haouari, Reem Suwaileh, Zien Sheikh Ali, Bayan Hamdan, Tamer Elsayed, Alberto Barrón-Cedeño, Giovanni Da San Martino, and Preslav Nakov. 2020. Overview of CheckThat! 2020 Arabic: Automatic identification and verification of claims in social media. In Working Notes of Conference of the Cross-Language Evaluation Forum for European Languages, CLEF ’2020. [find](https://scholar.google.com/scholar?q=Maram+Hasanain%2C+Fatima+Haouari%2C+Reem+Suwaileh%2C+Zien+Sheikh+Ali%2C+Bayan+Hamdan%2C+Tamer+Elsayed%2C+Alberto+Barro%CC%81n%2DCeden%CC%83o%2C+Giovanni+Da+San+Martino%2C+and+Preslav+Nakov%2E+2020%2E+Overview+of+CheckThat%21+2020+Arab)
31. Naeemul Hassan, Chengkai Li, and Mark Tremayne. 2015. Detecting check-worthy factual claims in presidential debates. In Proceedings of the 24th ACM International Conference on Information and Knowledge Management, CIKM ’15, pages 1835– 1838, Melbourne, Australia. [find](https://scholar.google.com/scholar?q=Naeemul+Hassan%2C+Chengkai+Li%2C+and+Mark+Tremayne%2E+2015%2E+Detecting+check%2Dworthy+factual+claims+in+presidential+debates%2E+In+Proceedings+of+the+24th+ACM+International+Conference+on+Information+and+Knowledg)
32. Tamanna Hossain, Robert L. Logan IV, Arjuna Ugarte, Yoshitomo Matsubara, Sean Young, and Sameer Singh. 2020. [COVIDLies: Detecting COVID-19 misinformation on social media](https://md.research.elamur.ai/free/en/papers/hossain-2020-covidlies-detecting-covid-19-misinformation/). In Proceedings of the 1st Workshop on NLP for COVID-19 (Part 2) at EMNLP.
33. Israa Jaradat, Pepa Gencheva, Alberto Barrón-Cedeño, Lluı́s Màrquez, and Preslav Nakov. 2018. Claim-Rank: Detecting check-worthy claims in Arabic and English. In Proceedings of the 16th Annual Conference of the North American Chapter of the Association for Computational Linguistics, NAACL-HLT ’18, pages 26–30, New Orleans, LA, USA. [find](https://scholar.google.com/scholar?q=Israa+Jaradat%2C+Pepa+Gencheva%2C+Alberto+Barro%CC%81n%2DCeden%CC%83o%2C+Llu%C4%B1%CC%81s+Ma%CC%80rquez%2C+and+Preslav+Nakov%2E+2018%2E+Claim%2DRank%3A+Detecting+check%2Dworthy+claims+in+Arabic+and+English%2E+In+Proceedings+of+the+16th+Annual+Conf)
34. Georgi Karadzhov, Preslav Nakov, Lluı́s Màrquez, Alberto Barrón-Cedeño, and Ivan Koychev. 2017. Fully automated fact checking using external sources. In Proceedings of the Conference on Recent Advances in Natural Language Processing, RANLP ’17, pages 344–353, Varna, Bulgaria. [↗](https://doi.org/10.48550/arxiv.1710.00341)
35. Preslav Nakov, Firoj Alam, Shaden Shaar, Giovanni Da San Martino, and Yifan Zhang. 2021a. COVID-19 in Bulgarian social media: Factuality, harmfulness, propaganda, and framing. In Proceedings of the International Conference on Recent Advances in Natural Language Processing, RANLP ’21. [find](https://scholar.google.com/scholar?q=Preslav+Nakov%2C+Firoj+Alam%2C+Shaden+Shaar%2C+Giovanni+Da+San+Martino%2C+and+Yifan+Zhang%2E+2021a%2E+COVID%2D19+in+Bulgarian+social+media%3A+Factuality%2C+harmfulness%2C+propaganda%2C+and+framing%2E+In+Proceedings+of+the+In)
36. Haewoon Kwak, Jisun An, and Yong-Yeol Ahn. 2020. A systematic media frame analysis of 1.5 million New York Times articles from 2000 to 2017. In Proceedings of the 12th ACM Conference on Web Science, WebSci ’20, pages 305–314, Southampton, United Kingdom. [find](https://scholar.google.com/scholar?q=Haewoon+Kwak%2C+Jisun+An%2C+and+Yong%2DYeol+Ahn%2E+2020%2E+A+systematic+media+frame+analysis+of+1%2E5+million+New+York+Times+articles+from+2000+to+2017%2E+In+Proceedings+of+the+12th+ACM+Conference+on+Web+Science%2C+W)
37. Preslav Nakov, Alberto Barrón-Cedeño, Tamer Elsayed, Reem Suwaileh, Lluı́s Màrquez, Wajdi Zaghouani, Pepa Atanasova, Spas Kyuchukov, and Giovanni Da San Martino. 2018. Overview of the CLEF-2018 CheckThat! Lab on automatic identification and verification of political claims. In Proceedings of the Ninth International Conference of the CLEF Association: Experimental IR Meets Multilinguality, Multimodality, and Interaction, Lecture Notes in Computer Science, pages 372–387, Avignon, France. [find](https://scholar.google.com/scholar?q=Preslav+Nakov%2C+Alberto+Barro%CC%81n%2DCeden%CC%83o%2C+Tamer+Elsayed%2C+Reem+Suwaileh%2C+Llu%C4%B1%CC%81s+Ma%CC%80rquez%2C+Wajdi+Zaghouani%2C+Pepa+Atanasova%2C+Spas+Kyuchukov%2C+and+Giovanni+Da+San+Martino%2E+2018%2E+Overview+of+the+CLEF%2D2018+Che)
38. David M.J. Lazer, Matthew A. Baum, Yochai Benkler, Adam J. Berinsky, Kelly M. Greenhill, Filippo Menczer, Miriam J. Metzger, Brendan Nyhan, Gordon Pennycook, David Rothschild, Michael Schudson, Steven A. Sloman, Cass R. Sunstein, Emily A. Thorson, Duncan J. Watts, and Jonathan L. Zittrain. 2018. The science of fake news. Science, 359(6380):1094–1096. [↗](https://doi.org/10.1126/science.aao2998)
39. Lifang Li, Qingpeng Zhang, Xiao Wang, Jun Zhang, Tao Wang, Tian-Lu Gao, Wei Duan, Kelvin Kam-fai Tsoi, and Fei-Yue Wang. 2020. Characterizing the propagation of situational information in social media during COVID-19 epidemic: A case study on Weibo. IEEE Transactions on Computational Social Systems, 7(2):556–562. [↗](https://doi.org/10.1109/tcss.2020.2980007)
40. Yaliang Li, Jing Gao, Chuishi Meng, Qi Li, Lu Su, Bo Zhao, Wei Fan, and Jiawei Han. 2016. A survey on truth discovery. SIGKDD Explor. Newsl., 17(2):1–16. [↗](https://doi.org/10.1145/2897350.2897352)
41. Siyi Liu, Lei Guo, Kate Mays, Margrit Betke, and Derry Tanti Wijaya. 2019. Detecting frames in news headlines and its application to analyzing news framing trends surrounding US gun violence. In Proceedings of the 23rd Conference on Computational Natural Language Learning, CoNLL ’19, pages 504–514, Hong Kong, China. [find](https://scholar.google.com/scholar?q=Siyi+Liu%2C+Lei+Guo%2C+Kate+Mays%2C+Margrit+Betke%2C+and+Derry+Tanti+Wijaya%2E+2019%2E+Detecting+frames+in+news+headlines+and+its+application+to+analyzing+news+framing+trends+surrounding+US+gun+violence%2E+In+Proce)
42. Richard J Medford, Sameh N Saleh, Andrew Sumar-sono, Trish M Perl, and Christoph U Lehmann. 2020. An “Infodemic”: Leveraging high-volume Twitter data to understand early public sentiment for the coronavirus disease 2019 outbreak. OFID, 7(7). [↗](https://doi.org/10.1093/ofid/ofaa258)
43. Tsvetomila Mihaylova, Georgi Karadzhov, Pepa Atanasova, Ramy Baly, Mitra Mohtarami, and Preslav Nakov. 2019. SemEval-2019 task 8: Fact checking in community question answering forums. In Proceedings of the 13th International Workshop on Semantic Evaluation, SemEval ’19, pages 860– 869, Minneapolis, MN, USA. [find](https://scholar.google.com/scholar?q=Tsvetomila+Mihaylova%2C+Georgi+Karadzhov%2C+Pepa+Atanasova%2C+Ramy+Baly%2C+Mitra+Mohtarami%2C+and+Preslav+Nakov%2E+2019%2E+SemEval%2D2019+task+8%3A+Fact+checking+in+community+question+answering+forums%2E+In+Proceedings+o)
44. Tsvetomila Mihaylova, Preslav Nakov, Lluı́s Màrquez, Alberto Barrón-Cedeño, Mitra Mohtarami, Georgi Karadjov, and James Glass. 2018. Fact checking in community forums. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, AAAI ’18, pages 879–886, New Orleans, LA, USA. [↗](https://doi.org/10.1609/aaai.v32i1.11983)
45. Preslav Nakov, David Corney, Maram Hasanain, Firoj Alam, Tamer Elsayed, Alberto Barrón-Cedeño, Paolo Papotti, Shaden Shaar, and Giovanni Da San Martino. 2021b. Automated fact-checking for assisting human fact-checkers. In Proceedings of the 30th International Joint Conference on Artificial Intelligence, IJCAI ’21, pages 4551–4558. [find](https://scholar.google.com/scholar?q=Preslav+Nakov%2C+David+Corney%2C+Maram+Hasanain%2C+Firoj+Alam%2C+Tamer+Elsayed%2C+Alberto+Barro%CC%81n%2DCeden%CC%83o%2C+Paolo+Papotti%2C+Shaden+Shaar%2C+and+Giovanni+Da+San+Martino%2E+2021b%2E+Automated+fact%2Dchecking+for+assisting+)
46. Preslav Nakov, Giovanni Da San Martino, Tamer Elsayed, Alberto Barrón-Cedeño, Rubén Mı́guez, Shaden Shaar, Firoj Alam, Fatima Haouari, Maram Hasanain, Nikolay Babulkov, Alex Nikolov, Gautam Kishore Shahi, Julia Maria Struß, and Thomas Mandl. 2021c. The CLEF-2021 CheckThat! Lab on detecting check-worthy claims, previously fact-checked claims, and fake news. In Proceedings of the 43rd European Conference on Information Retrieval, ECIR ’21, pages 639–649. [find](https://scholar.google.com/scholar?q=Preslav+Nakov%2C+Giovanni+Da+San+Martino%2C+Tamer+Elsayed%2C+Alberto+Barro%CC%81n%2DCeden%CC%83o%2C+Rube%CC%81n+M%C4%B1%CC%81guez%2C+Shaden+Shaar%2C+Firoj+Alam%2C+Fatima+Haouari%2C+Maram+Hasanain%2C+Nikolay+Babulkov%2C+Alex+Nikolov%2C+Gautam+Kishore)
47. Preslav Nakov, Giovanni Da San Martino, Tamer Elsayed, Alberto Barrón-Cedeño, Rubén Mı́guez, Shaden Shaar, Firoj Alam, Fatima Haouari, Maram Hasanain, Nikolay Babulkov, Alex Nikolov, Gautam Kishore Shahi, Julia Maria Struß, and Thomas Mandl. 2021d. The CLEF-2021 CheckThat! lab on detecting check-worthy claims, previously fact-checked claims, and fake news. In Proceedings of the 43rd European Conference on Information Retrieval, ECIR ’21, pages 639–649. [find](https://scholar.google.com/scholar?q=Preslav+Nakov%2C+Giovanni+Da+San+Martino%2C+Tamer+Elsayed%2C+Alberto+Barro%CC%81n%2DCeden%CC%83o%2C+Rube%CC%81n+M%C4%B1%CC%81guez%2C+Shaden+Shaar%2C+Firoj+Alam%2C+Fatima+Haouari%2C+Maram+Hasanain%2C+Nikolay+Babulkov%2C+Alex+Nikolov%2C+Gautam+Kishore)
48. Van-Hoang Nguyen, Kazunari Sugiyama, Preslav Nakov, and Min-Yen Kan. 2020. FANG: Leveraging social context for fake news detection using graph representation. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management, CIKM ’20, pages 1165–1174. [find](https://scholar.google.com/scholar?q=Van%2DHoang+Nguyen%2C+Kazunari+Sugiyama%2C+Preslav+Nakov%2C+and+Min%2DYen+Kan%2E+2020%2E+FANG%3A+Leveraging+social+context+for+fake+news+detection+using+graph+representation%2E+In+Proceedings+of+the+29th+ACM+Internatio)
49. Ayush Patwari, Dan Goldwasser, and Saurabh Bagchi. 2017. TATHYA: a multi-classifier system for detecting check-worthy statements in political debates. In Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, CIKM ’17, pages 2259–2262, Singapore. [find](https://scholar.google.com/scholar?q=Ayush+Patwari%2C+Dan+Goldwasser%2C+and+Saurabh+Bagchi%2E+2017%2E+TATHYA%3A+a+multi%2Dclassifier+system+for+detecting+check%2Dworthy+statements+in+political+debates%2E+In+Proceedings+of+the+2017+ACM+on+Conference+on+I)
50. Andrew Perrin. 2015. Social media usage. Pew research center, pages 52–68. [find](https://scholar.google.com/scholar?q=Andrew+Perrin%2E+2015%2E+Social+media+usage%2E+Pew+research+center%2C+pages+52%E2%80%9368%2E)
51. Cristina M Pulido, Beatriz Villarejo-Carballido, Gisela Redondo-Sama, and Aitor Gómez. 2020. COVID-19 infodemic: More retweets for science-based information on coronavirus than for false information. International Sociology, 35(4):377–392. [↗](https://doi.org/10.1177/0268580920914755)
52. Hannah Rashkin, Eunsol Choi, Jin Yea Jang, Svitlana Volkova, and Yejin Choi. 2017. [Truth of varying shades: Analyzing language in fake news and political fact-checking](https://md.research.elamur.ai/free/en/papers/rashkin-2017-truth-varying-shades-analyzing-language/). In Proceedings of the Conference on Empirical Methods in Natural Language Processing, EMNLP ’17, pages 2931–2937, Copenhagen, Denmark.
53. Shaden Shaar, Firoj Alam, Giovanni Da San Martino, and Preslav Nakov. 2021a. The role of context in detecting previously fact-checked claims. Arxiv/2104.07423. [find](https://scholar.google.com/scholar?q=Shaden+Shaar%2C+Firoj+Alam%2C+Giovanni+Da+San+Martino%2C+and+Preslav+Nakov%2E+2021a%2E+The+role+of+context+in+detecting+previously+fact%2Dchecked+claims%2E+Arxiv%2F2104%2E07423%2E)
54. Shaden Shaar, Firoj Alam, Giovanni Da San Martino, Alex Nikolov, Wajdi Zaghouani, Preslav Nakov, and Anna Feldman. 2021b. Findings of the NLP4IF-2021 shared tasks on fighting the COVID-19 infodemic and censorship detection. In Proceedings of the Fourth Workshop on NLP for Internet Freedom: Censorship, Disinformation, and Propaganda, pages 82–92. [↗](https://doi.org/10.48550/arxiv.2109.12986)
55. Shaden Shaar, Nikolay Babulkov, Giovanni Da San Martino, and Preslav Nakov. 2020a. [That is a known lie: Detecting previously fact-checked claims](https://md.research.elamur.ai/free/en/papers/shaar-2020-that-is-known-lie-detecting/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, ACL ’20, pages 3607–3618.
56. Shaden Shaar, Maram Hasanain, Bayan Hamdan, Zien Sheikh Ali, Fatima Haouari, Alex Nikolov, Mucahid Kutlu, Yavuz Selim Kartal, Firoj Alam, Giovanni Da San Martino, Alberto Barrón-Cedeño, Rubén Mı́guez, Javier Beltrán, Tamer Elsayed, and Preslav Nakov. 2021c. Overview of the CLEF-2021 CheckThat! lab task 1 on check-worthiness estimation in tweets and political debates. In Working Notes of CLEF 2021—Conference and Labs of the Evaluation Forum, CLEF ’2021. CEUR-WS.org. [find](https://scholar.google.com/scholar?q=Shaden+Shaar%2C+Maram+Hasanain%2C+Bayan+Hamdan%2C+Zien+Sheikh+Ali%2C+Fatima+Haouari%2C+Alex+Nikolov%2C+Mucahid+Kutlu%2C+Yavuz+Selim+Kartal%2C+Firoj+Alam%2C+Giovanni+Da+San+Martino%2C+Alberto+Barro%CC%81n%2DCeden%CC%83o%2C+Rube%CC%81n+M%C4%B1%CC%81gu)
57. Shaden Shaar, Alex Nikolov, Nikolay Babulkov, Firoj Alam, Alberto Barrón-Cedeño, Tamer Elsayed, Maram Hasanain, Reem Suwaileh, Fatima Haouari, Giovanni Da San Martino, and Preslav Nakov. 2020b. Overview of CheckThat! 2020 English: Automatic identification and verification of claims in social media. In Working Notes of CLEF 2021— Conference and Labs of the Evaluation Forum, CEUR Workshop Proceedings. CEUR-WS.org. [find](https://scholar.google.com/scholar?q=Shaden+Shaar%2C+Alex+Nikolov%2C+Nikolay+Babulkov%2C+Firoj+Alam%2C+Alberto+Barro%CC%81n%2DCeden%CC%83o%2C+Tamer+Elsayed%2C+Maram+Hasanain%2C+Reem+Suwaileh%2C+Fatima+Haouari%2C+Giovanni+Da+San+Martino%2C+and+Preslav+Nakov%2E+2020b%2E+Over)
58. Kai Shu, Amy Sliva, Suhang Wang, Jiliang Tang, and Huan Liu. 2017. Fake news detection on social media: A data mining perspective. SIGKDD Explor. Newsl., 19(1):22–36. [↗](https://doi.org/10.48550/arxiv.1708.01967)
59. Slavena Vasileva, Pepa Atanasova, Lluı́s Màrquez, Alberto Barrón-Cedeño, and Preslav Nakov. 2019. It takes nine to smell a rat: Neural multi-task learning for check-worthiness prediction. In Proceedings of the International Conference on Recent Advances in Natural Language Processing, RANLP ’19, pages 1229–1239, Varna, Bulgaria. [↗](https://doi.org/10.48550/arxiv.1908.07912)
60. Bertie Vidgen, Scott Hale, Ella Guest, Helen Margetts, David Broniatowski, Zeerak Waseem, Austin Botelho, Matthew Hall, and Rebekah Tromble. 2020. Detecting East Asian prejudice on social media. In Proceedings of the Workshop on Online Abuse and Harms, WOAH ’20, pages 162–172. [↗](https://doi.org/10.48550/arxiv.2005.03909)
61. Nguyen Vo and Kyumin Lee. 2018. The rise of guardians: Fact-checking URL recommendation to combat fake news. In Proceedings of the 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR ’18, pages 275–284, Ann Arbor, MI, USA. [↗](https://doi.org/10.48550/arxiv.1806.07516)
62. Soroush Vosoughi, Deb Roy, and Sinan Aral. 2018. The spread of true and false news online. Science, 359(6380):1146–1151. [↗](https://doi.org/10.1126/science.aap9559)
63. William Yang Wang. 2017. [“Liar, liar pants on fire”: A new benchmark dataset for fake news detection](https://md.research.elamur.ai/free/en/papers/wang-2017-liar-liar-pants-fire-new/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL ’17, pages 422–426, Vancouver, Canada.
64. Samuel C Woolley and Philip N Howard. 2018. Computational propaganda: political parties, politicians, and political manipulation on social media. Oxford University Press. [find](https://scholar.google.com/scholar?q=Samuel+C+Woolley+and+Philip+N+Howard%2E+2018%2E+Computational+propaganda%3A+political+parties%2C+politicians%2C+and+political+manipulation+on+social+media%2E+Oxford+University+Press%2E)
65. Seunghak Yu, Giovanni Da San Martino, Mitra Mohtarami, James Glass, and Preslav Nakov. 2021. Interpretable propaganda detection in news articles. In Proceedings of the International Conference on Recent Advances in Natural Language Processing, RANLP ’21. [↗](https://doi.org/10.48550/arxiv.2108.12802)
66. Yifan Zhang, Giovanni Da San Martino, Alberto Barrón-Cedeño, Salvatore Romeo, Jisun An, Haewoon Kwak, Todor Staykovski, Israa Jaradat, Georgi Karadzhov, Ramy Baly, Kareem Darwish, James Glass, and Preslav Nakov. 2019. Tanbih: Get to know what you are reading. In Proceedings of the Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing: System Demonstrations, EMNLP-IJCNLP ’19, pages 223–228, Hong Kong, China. [↗](https://doi.org/10.48550/arxiv.1910.02028)
67. Xinyi Zhou, Apurva Mulay, Emilio Ferrara, and Reza Zafarani. 2020. ReCOVery: A multimodal repository for COVID-19 news credibility research. In Proceedings of the 29th ACM International Conference on Information and Knowledge Management, CIKM ’20, pages 3205–3212. [find](https://scholar.google.com/scholar?q=Xinyi+Zhou%2C+Apurva+Mulay%2C+Emilio+Ferrara%2C+and+Reza+Zafarani%2E+2020%2E+ReCOVery%3A+A+multimodal+repository+for+COVID%2D19+news+credibility+research%2E+In+Proceedings+of+the+29th+ACM+International+Conference+on+)
68. Dimitrina Zlatkova, Preslav Nakov, and Ivan Koychev. 2019. Fact-checking meets fauxtography: Verifying claims about images. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, EMNLP ’19, pages 2099–2108, Hong Kong, China. [find](https://scholar.google.com/scholar?q=Dimitrina+Zlatkova%2C+Preslav+Nakov%2C+and+Ivan+Koychev%2E+2019%2E+Fact%2Dchecking+meets+fauxtography%3A+Verifying+claims+about+images%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Langua)

*Updated 1 November 2026*