---
title: "Rumor has it — Qazvinian 2011"
description: "Qazvinian and colleagues study rumors on Twitter, meaning statements that are unverifiable or deliberately false."
authors: ["Vahed Qazvinian", "Emily Rosengren", "Dragomir R. Radev", "Qiaozhu Mei"]
year: 2011
venue: "Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D11-1147/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/qazvinian-2011-rumor-has-it-identifying-misinformation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/qazvinian-2011-rumor-has-it-identifying-misinformation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2011](https://md.research.elamur.ai/free/en/papers/2011/) / Rumor has it: Identifying Misinformation in Microblogs

# Rumor has it: Identifying Misinformation in Microblogs

**Authors:** Vahed Qazvinian, Emily Rosengren, Dragomir R. Radev, [Qiaozhu Mei](https://md.research.elamur.ai/free/en/authors/qiaozhu-mei/) · **Year:** 2011 · **Venue:** Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/D11-1147/) · [PDF](https://aclanthology.org/D11-1147.pdf) · [OpenAlex](https://openalex.org/W2159981908)

**Citations:** 686 citations (openalex-snapshot · 2026-10-14) · [78](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Qazvinian and colleagues study rumors on Twitter, meaning statements that are unverifiable or deliberately false. They split the problem into two tasks. The first is rumor retrieval: from tweets matched by a broad keyword query, keep only those that actually discuss a given rumor. The second is belief classification: decide whether the poster endorses the rumor or instead denies, questions, or neutrally reports it.

The authors built a corpus around five rumors that About.com's Urban Legends site lists as false or only partly true, collected with hand-written regular expressions through the Twitter search API. Two annotators then labeled more than 10,000 tweets.

Their model combines statistical models over three feature families: content (words, part-of-speech), network (user history), and Twitter-specific signals (hashtags, URLs). It scores tweets by maximizing a linear combination of log-likelihood ratios. For retrieval, using all nine features gives a MAP of about 0.965, well above the regex baseline and KL-divergence ranking. Content features alone handle belief classification well.

The work matters because it presents what the authors describe as the first large-scale rumor dataset. It is also an early step toward automatically finding people who spread misinformation.

## Key points
- The dataset covers five rumors: obama, airfrance, cellphone, michelle, and palin. The obama story (4,975 tweets) and the palin story (4,423) are the largest, and the total is more than 10,400 annotated tweets.
- About 35% of tweets matched by the regex queries are false positives. Among the 6,774 tweets that really concern a rumor, 2,971 (around 43%) endorse it.
- On rumor retrieval, MAP rises from 0.587 for the plain regexp, to 0.803 for the best KL model, to 0.965 for the log-likelihood model with all 9 features (F of 0.897).
- On belief classification (5-fold CV), content features (TXT+POS) reach F 0.932. All features together reach 0.935 accuracy. Hashtags alone give fairly high precision but recall of only 0.099.
- In a learning-curve experiment on the obama rumor, Average Precision and accuracy both pass 80% after roughly 2,000 in-domain labeled tweets.

## Interesting ideas
- **Two-stage collection: query broadly, then filter.** A deliberately loose regex maximizes recall, and a learned filter then removes the false positives. This suits any topical corpus building, for example gathering Ukrainian-language posts on a specific claim, when keyword search alone would be noisy.
- **Belief as stance toward one proposition, not general sentiment.** The label asks whether the author accepts a particular claim, which is more specific than whether the tweet sounds positive. This framing carries over directly to fact-checking and claim-stance datasets.
- **Combining separate feature models through log-likelihood ratios.** Each feature family gets its own simple statistical model, and their likelihood ratios are summed linearly. This modular design makes it easy to add new signal types and to measure how much each one contributes.
- **Reporting precision and recall per feature family.** The breakdown shows, for instance, that hashtags are precise but cover few tweets. This tells practitioners which cheap signals are useful as high-confidence rules and which need a model to back them up.
- **Measuring how much labeled data a new topic needs.** The learning curve on a held-out topic estimates the annotation budget for each new rumor. This is a practical way to plan labeling effort in any multi-topic project.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

78 papers of our library cite it (of the 140884 whose bibliographies we read); 14 of them have a page here, 64 not yet.

- [Topic-Guided Sampling For Data-Efficient Multi-Domain Stance Detection](https://md.research.elamur.ai/free/en/papers/arakelyan-2023-topic-guided-sampling-data-efficient/)
- [Stance Detection with Bidirectional Conditional Encoding](https://md.research.elamur.ai/free/en/papers/augenstein-2016-stance-detection-bidirectional-conditional-encoding/)
- [SemEval-2017 Task 8: RumourEval: Determining rumour veracity and support for rumours](https://md.research.elamur.ai/free/en/papers/derczynski-2017-semeval-2017-task-8-rumoureval/)
- [Emergent: a novel data-set for stance classification](https://md.research.elamur.ai/free/en/papers/ferreira-2016-emergent-novel-data-set-stance/)
- [SemEval-2019 Task 7: RumourEval, Determining Rumour Veracity and Support for Rumours](https://md.research.elamur.ai/free/en/papers/gorrell-2019-semeval-2019-task-7-rumoureval/)
- [Cross-Domain Label-Adaptive Stance Detection](https://md.research.elamur.ai/free/en/papers/hardalov-2021-cross-domain-label-adaptive-stance/)
- [A Survey on Stance Detection for Mis- and Disinformation Identification](https://md.research.elamur.ai/free/en/papers/hardalov-2022-survey-stance-detection-mis-disinformation/)
- [Agentic Detection of Online Conspiracies](https://md.research.elamur.ai/free/en/papers/langstream-2026-biton-agentic-detection-online-conspiracies/)
- [Improving Stance Detection with Multi-Dataset Learning and Knowledge Distillation](https://md.research.elamur.ai/free/en/papers/li-2021-improving-stance-detection-multi-dataset/)
- [Rumor Detection on Twitter with Tree-structured Recursive Neural Networks](https://md.research.elamur.ai/free/en/papers/ma-2018-rumor-detection-twitter-tree-structured/)
- [DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning](https://md.research.elamur.ai/free/en/papers/popat-2018-declare-debunking-fake-news-false/)
- [Automated Fact Checking: Task Formulations, Methods and Future Directions](https://md.research.elamur.ai/free/en/papers/thorne-2018-automated-fact-checking-task-formulations/)
- [C-STANCE: A Large Dataset for Chinese Zero-Shot Stance Detection](https://md.research.elamur.ai/free/en/papers/zhao-2023-c-stance-large-dataset-chinese/)
- [Stanceosaurus: Classifying Stance Towards Multilingual Misinformation](https://md.research.elamur.ai/free/en/papers/zheng-2022-stanceosaurus-classifying-stance-towards-multilingual/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{qazvinian2011rumor,
  title = {{Rumor has it: Identifying Misinformation in Microblogs}},
  author = {Qazvinian, Vahed and Rosengren, Emily and Radev, Dragomir R. and Mei, Qiaozhu},
  booktitle = {Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing},
  year = {2011},
  url = {https://aclanthology.org/D11-1147/}
}
```

- [← previous Rule-Based Normalization of Historical Texts](https://md.research.elamur.ai/free/en/papers/bollmann-2011-rule-based-normalization-historical-texts/)
- [next → Selected Operations, Algorithms, and Applications of n-Tape Weighted Finite-State Machines](https://md.research.elamur.ai/free/en/papers/langstream-2011-kempe-selected-operations-algorithms-applications-n/)
- [↑ back to papers of 2011](https://md.research.elamur.ai/free/en/papers/2011/)

## References

25 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Floyd H. Allport and Milton Lepkin. 1945. Wartime rumors of waste and special privilege: why some people believe them. Journal of Abnormal and Social Psychology, 40(1):3 – 36. [find](https://scholar.google.com/scholar?q=Floyd+H%2E+Allport+and+Milton+Lepkin%2E+1945%2E+Wartime+rumors+of+waste+and+special+privilege%3A+why+some+people+believe+them%2E+Journal+of+Abnormal+and+Social+Psychology%2C+40%281%29%3A3+%E2%80%93+36%2E)
2. Gordon Allport and Leo Postman. 1947. The psychology of rumor. Holt, Rinehart, and Winston, New York. [find](https://scholar.google.com/scholar?q=Gordon+Allport+and+Leo+Postman%2E+1947%2E+The+psychology+of+rumor%2E+Holt%2C+Rinehart%2C+and+Winston%2C+New+York%2E)
3. Galen Andrew and Jianfeng Gao. 2007. Scalable training of l1-regularized log-linear models. In ICML ’07, pages 33–40. [find](https://scholar.google.com/scholar?q=Galen+Andrew+and+Jianfeng+Gao%2E+2007%2E+Scalable+training+of+l1%2Dregularized+log%2Dlinear+models%2E+In+ICML+%E2%80%9907%2C+pages+33%E2%80%9340%2E)
4. James Berger. 1985. Statistical decision theory and Bayesian Analysis (2nd ed.). New York: Springer-Verlag. [find](https://scholar.google.com/scholar?q=James+Berger%2E+1985%2E+Statistical+decision+theory+and+Bayesian+Analysis+%282nd+ed%2E%29%2E+New+York%3A+Springer%2DVerlag%2E)
5. Albert Bifet and Eibe Frank. 2010. Sentiment knowledge discovery in twitter streaming data. In Bernhard Pfahringer, Geoff Holmes, and Achim Hoffmann, editors, Discovery Science, volume 6332 of Lecture Notes in Computer Science, pages 1–15. Springer Berlin / Heidelberg. [find](https://scholar.google.com/scholar?q=Albert+Bifet+and+Eibe+Frank%2E+2010%2E+Sentiment+knowledge+discovery+in+twitter+streaming+data%2E+In+Bernhard+Pfahringer%2C+Geoff+Holmes%2C+and+Achim+Hoffmann%2C+editors%2C+Discovery+Science%2C+volume+6332+of+Lecture)
6. Jean Carletta. 1996. Assessing agreement on classification tasks: the kappa statistic. Comput. Linguist., 22(2):249–254. [find](https://scholar.google.com/scholar?q=Jean+Carletta%2E+1996%2E+Assessing+agreement+on+classification+tasks%3A+the+kappa+statistic%2E+Comput%2E+Linguist%2E%2C+22%282%29%3A249%E2%80%93254%2E) [our card](https://md.research.elamur.ai/free/en/papers/carletta-1996-assessing-agreement-classification-tasks-kappa/)
7. Philip Clarkson and Roni Rosenfeld. 1997. Statistical language modeling using the cmu-cambridge toolkit. Proceedings ESCA Eurospeech, 47:45–148. [find](https://scholar.google.com/scholar?q=Philip+Clarkson+and+Roni+Rosenfeld%2E+1997%2E+Statistical+language+modeling+using+the+cmu%2Dcambridge+toolkit%2E+Proceedings+ESCA+Eurospeech%2C+47%3A45%E2%80%93148%2E)
8. Nicholas DiFonzo and Prashant Bordia. 2007. Rumor, gossip, and urban legend. Diogenes, 54:19–35, February. [find](https://scholar.google.com/scholar?q=Nicholas+DiFonzo+and+Prashant+Bordia%2E+2007%2E+Rumor%2C+gossip%2C+and+urban+legend%2E+Diogenes%2C+54%3A19%E2%80%9335%2C+February%2E)
9. Nicholas DiFonzo, P. Prashant Bordia, and Ralph L. Rosnow. 1994. Reining in rumors. Organizational Dynamics, 23(1):47–62. [find](https://scholar.google.com/scholar?q=Nicholas+DiFonzo%2C+P%2E+Prashant+Bordia%2C+and+Ralph+L%2E+Rosnow%2E+1994%2E+Reining+in+rumors%2E+Organizational+Dynamics%2C+23%281%29%3A47%E2%80%9362%2E)
10. Rob Ennals, Dan Byler, John Mark Agosta, and Barbara Rosario. 2010. What is disputed on the web? In Proceedings of the 4th workshop on Information Credibility, WICOW ’10, pages 67–74. [find](https://scholar.google.com/scholar?q=Rob+Ennals%2C+Dan+Byler%2C+John+Mark+Agosta%2C+and+Barbara+Rosario%2E+2010%2E+What+is+disputed+on+the+web%3F+In+Proceedings+of+the+4th+workshop+on+Information+Credibility%2C+WICOW+%E2%80%9910%2C+pages+67%E2%80%9374%2E)
11. Jianfeng Gao, Galen Andrew, Mark Johnson, and Kristina Toutanova. 2007. A comparative study of parameter estimation methods for statistical natural language processing. In ACL ’07. [find](https://scholar.google.com/scholar?q=Jianfeng+Gao%2C+Galen+Andrew%2C+Mark+Johnson%2C+and+Kristina+Toutanova%2E+2007%2E+A+comparative+study+of+parameter+estimation+methods+for+statistical+natural+language+processing%2E+In+ACL+%E2%80%9907%2E)
12. Namrata Godbole, Manjunath Srinivasaiah, and Steven Skiena. 2007. Large-scale sentiment analysis for news and blogs. In Proceedings of the International Conference on Weblogs and Social Media (ICWSM), Boulder, CO, USA. [find](https://scholar.google.com/scholar?q=Namrata+Godbole%2C+Manjunath+Srinivasaiah%2C+and+Steven+Skiena%2E+2007%2E+Large%2Dscale+sentiment+analysis+for+news+and+blogs%2E+In+Proceedings+of+the+International+Conference+on+Weblogs+and+Social+Media+%28ICWSM%29%2C)
13. Ahmed Hassan, Vahed Qazvinian, and Dragomir Radev. 2010. What’s with the attitude? identifying sentences with attitude in online discussions. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, pages 1245–1255, Cambridge, MA, October. Association for Computatio… [find](https://scholar.google.com/scholar?q=Ahmed+Hassan%2C+Vahed+Qazvinian%2C+and+Dragomir+Radev%2E+2010%2E+What%E2%80%99s+with+the+attitude%3F+identifying+sentences+with+attitude+in+online+discussions%2E+In+Proceedings+of+the+2010+Conference+on+Empirical+Methods)
14. Edward S Herman and Noam Chomsky. 2002. Manufacturing Consent: The Political Economy of the Mass Media. Pantheon. [find](https://scholar.google.com/scholar?q=Edward+S+Herman+and+Noam+Chomsky%2E+2002%2E+Manufacturing+Consent%3A+The+Political+Economy+of+the+Mass+Media%2E+Pantheon%2E)
15. Courtenay Honeycutt and Susan C. Herring. 2009. Beyond microblogging: Conversation and collaboration via twitter. Hawaii International Conference on System Sciences, 0:1–10. [find](https://scholar.google.com/scholar?q=Courtenay+Honeycutt+and+Susan+C%2E+Herring%2E+2009%2E+Beyond+microblogging%3A+Conversation+and+collaboration+via+twitter%2E+Hawaii+International+Conference+on+System+Sciences%2C+0%3A1%E2%80%9310%2E)
16. Jeff Huang, Katherine M. Thornton, and Efthimis N. Efthimiadis. 2010. Conversational tagging in twitter. In Proceedings of the 21st ACM conference on Hypertext and hypermedia, HT ’10, pages 173–178. [find](https://scholar.google.com/scholar?q=Jeff+Huang%2C+Katherine+M%2E+Thornton%2C+and+Efthimis+N%2E+Efthimiadis%2E+2010%2E+Conversational+tagging+in+twitter%2E+In+Proceedings+of+the+21st+ACM+conference+on+Hypertext+and+hypermedia%2C+HT+%E2%80%9910%2C+pages+173%E2%80%93178%2E)
17. Klaus Krippendorff. 1980. Content Analysis: An Introduction to its Methodology. Beverly Hills: Sage Publications. [find](https://scholar.google.com/scholar?q=Klaus+Krippendorff%2E+1980%2E+Content+Analysis%3A+An+Introduction+to+its+Methodology%2E+Beverly+Hills%3A+Sage+Publications%2E)
18. Jure Leskovec, Lars Backstrom, and Jon Kleinberg. 2009. Meme-tracking and the dynamics of the news cycle. In KDD ’09: Proceedings of the 15th ACM SIGKDD international conference on Knowledge discovery and data mining, pages 497–506. [find](https://scholar.google.com/scholar?q=Jure+Leskovec%2C+Lars+Backstrom%2C+and+Jon+Kleinberg%2E+2009%2E+Meme%2Dtracking+and+the+dynamics+of+the+news+cycle%2E+In+KDD+%E2%80%9909%3A+Proceedings+of+the+15th+ACM+SIGKDD+international+conference+on+Knowledge+discovery)
19. Christopher D. Manning, Prabhakar Raghavan, and Hinrich Schütze. 2008. Introduction to Information Retrieval. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Christopher+D%2E+Manning%2C+Prabhakar+Raghavan%2C+and+Hinrich+Schu%CC%88tze%2E+2008%2E+Introduction+to+Information+Retrieval%2E+Cambridge+University+Press%2E)
20. Marcelo Mendoza, Barbara Poblete, and Carlos Castillo. Twitter under crisis: Can we trust what we rt? [find](https://scholar.google.com/scholar?q=Marcelo+Mendoza%2C+Barbara+Poblete%2C+and+Carlos+Castillo%2E+Twitter+under+crisis%3A+Can+we+trust+what+we+rt%3F)
21. Alexander Pak and Patrick Paroubek. 2010. Twitter as a corpus for sentiment analysis and opinion mining. In Nicoletta Calzolari (Conference Chair), Khalid Choukri, Bente Maegaard, Joseph Mariani, Jan Odijk, Stelios Piperidis, Mike Rosner, and Daniel Tapias, editors, Proceedings of the Seventh confer… [find](https://scholar.google.com/scholar?q=Alexander+Pak+and+Patrick+Paroubek%2E+2010%2E+Twitter+as+a+corpus+for+sentiment+analysis+and+opinion+mining%2E+In+Nicoletta+Calzolari+%28Conference+Chair%29%2C+Khalid+Choukri%2C+Bente+Maegaard%2C+Joseph+Mariani%2C+Jan+) [our card](https://md.research.elamur.ai/free/en/papers/pak-2010-twitter-as-corpus-sentiment-analysis/)
22. Bo Pang and Lillian Lee. 2004. A sentimental education: sentiment analysis using subjectivity summarization based on minimum cuts. In ACL’04, Morristown, NJ, USA. [find](https://scholar.google.com/scholar?q=Bo+Pang+and+Lillian+Lee%2E+2004%2E+A+sentimental+education%3A+sentiment+analysis+using+subjectivity+summarization+based+on+minimum+cuts%2E+In+ACL%E2%80%9904%2C+Morristown%2C+NJ%2C+USA%2E) [our card](https://md.research.elamur.ai/free/en/papers/pang-2004-sentimental-education-sentiment-analysis-subjectivity/)
23. Bo Pang and Lillian Lee. 2008. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrieval, 2:1–135. [find](https://scholar.google.com/scholar?q=Bo+Pang+and+Lillian+Lee%2E+2008%2E+Opinion+mining+and+sentiment+analysis%2E+Foundations+and+Trends+in+Information+Retrieval%2C+2%3A1%E2%80%93135%2E)
24. Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up?: sentiment classification using machine learning techniques. In Proceedings of conference on Empirical methods in natural language processing, EMNLP’02, pages 79–86. [find](https://scholar.google.com/scholar?q=Bo+Pang%2C+Lillian+Lee%2C+and+Shivakumar+Vaithyanathan%2E+2002%2E+Thumbs+up%3F%3A+sentiment+classification+using+machine+learning+techniques%2E+In+Proceedings+of+conference+on+Empirical+methods+in+natural+language+) [our card](https://md.research.elamur.ai/free/en/papers/pang-2002-thumbs-up-sentiment-classification-machine/)
25. Jacob Ratkiewicz, Michael Conover, Mark Meiss, Bruno Gonçalves, Snehal Patil, Alessandro Flammini, and Filippo Menczer. 2010. Detecting and tracking the spread of astroturf memes in microblog streams. CoRR, abs/1011.3768. — [arXiv:1011.3768](https://arxiv.org/abs/1011.3768) [find](https://scholar.google.com/scholar?q=Jacob+Ratkiewicz%2C+Michael+Conover%2C+Mark+Meiss%2C+Bruno+Gonc%CC%A7alves%2C+Snehal+Patil%2C+Alessandro+Flammini%2C+and+Filippo+Menczer%2E+2010%2E+Detecting+and+tracking+the+spread+of+astroturf+memes+in+microblog+streams)