---
title: "COVID-Fact: Fact Extraction and Verification — Saakyan 2021"
description: "Automated fact-checking for COVID-19 lacks datasets that cover the whole pipeline: real-world claims, open-web evidence documents, sentence-level…"
authors: ["Arkadiy Saakyan", "Tuhin Chakrabarty", "Smaranda Muresan"]
year: 2021
venue: "Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)"
source: "https://aclanthology.org/2021.acl-long.165/"
doi: "10.18653/v1/2021.acl-long.165"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/saakyan-2021-covid-fact-fact-extraction-verification/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/saakyan-2021-covid-fact-fact-extraction-verification/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/3/) / COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic

# COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic

**Authors:** [Arkadiy Saakyan](https://md.research.elamur.ai/free/en/authors/arkadiy-saakyan/), [Tuhin Chakrabarty](https://md.research.elamur.ai/free/en/authors/tuhin-chakrabarty/), [Smaranda Muresan](https://md.research.elamur.ai/free/en/authors/smaranda-muresan/) · **Year:** 2021 · **Venue:** Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.acl-long.165/) · [PDF](https://aclanthology.org/2021.acl-long.165.pdf) · [DOI 10.18653/v1/2021.acl-long.165](https://doi.org/10.18653/v1/2021.acl-long.165) · [OpenAlex](https://openalex.org/W3168245858)

**Citations:** 90 citations (OpenAlex, 2026-10-14) · [78](https://md.research.elamur.ai/free/en/papers/saakyan-2021-covid-fact-fact-extraction-verification/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary
Automated fact-checking for COVID-19 lacks datasets that cover the whole pipeline: real-world claims, open-web evidence documents, sentence-level evidence and veracity labels. Earlier resources such as FEVER and SciFact draw evidence from fixed collections (Wikipedia, CORD-19) and pay people to write false claims, which is expensive. The authors build COVID-Fact semi-automatically.

True claims are post titles from the strictly moderated r/COVID19 subreddit, where every post must link to a credible source. Syntactic, popularity, site-rank and search-based filters, followed by manual checks, keep only well-formed and trustworthy claims. False claims are created automatically. The attention of a SciFact-tuned BERT picks the salient words, masked-language-model infilling swaps them for contrasting ones, and an entailment check controls quality. SBERT ranks candidate evidence sentences, and crowd workers confirm them.

The resulting data mixes technical and lay-language claims. Systems trained on FEVER or SciFact perform poorly on it, which shows a domain gap. Models trained on COVID-Fact transfer zero-shot to SciFact. The approach offers a cheap route to domain-specific misinformation datasets.

## Key points
- COVID-Fact contains 4,086 claims: 1,296 supported and 2,790 automatically generated refuted counter-claims. Each claim comes with evidence documents and sentences. Data and code are at github.com/asaakyan/covidfact.
- The claim filtering pipeline ran as follows:
  - It started from 22,646 Reddit posts collected via the Pushshift API.
  - Syntactic filtering left 6,154 claims.
  - Trustworthiness filtering left 1,526. This step required an upvote ratio of at least 0.7, an Alexa rank within the top 50,000, and the source appearing in the top 5 Google results.
  - Manual validation left 1,407, and 111 more were dropped during evidence selection.
- Counter-claims are generated in three steps: salient words are chosen from attention scores of a BERT model fine-tuned on SciFact, replaced through MLM infilling with a model trained on CORD-19, and filtered with entailment-based quality control.
- Evidence comes from the linked source plus four extra documents from the top 5 Google results. SBERT retrieves the top five sentences per claim, and crowdsourcing validates them.
- Models trained only on FEVER or SciFact do poorly on COVID-Fact. Training on COVID-Fact yields useful zero-shot performance on SciFact claim verification.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Institutions

- [Columbia University](https://md.research.elamur.ai/free/en/universities/columbia-university/) — University, New York, United States · 118 works · 44 744 citations

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{saakyan2021covid,
  title = {{COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic}},
  author = {Saakyan, Arkadiy and Chakrabarty, Tuhin and Muresan, Smaranda},
  booktitle = {Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)},
  year = {2021},
  doi = {10.18653/v1/2021.acl-long.165},
  url = {https://doi.org/10.18653/v1/2021.acl-long.165}
}
```

- [← previous Trankit: A Light-Weight Transformer-based Toolkit for Multilingual Natural Language Processing](https://md.research.elamur.ai/free/en/papers/nguyen-2021-trankit-light-weight-transformer-based/)
- [next → On Releasing Annotator-Level Labels and Information in Datasets](https://md.research.elamur.ai/free/en/papers/prabhakaran-2021-releasing-annotator-level-labels-information/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/3/)

## References

36 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Tariq Alhindi, Savvas Petridis, and Smaranda Muresan. 2018. Where is your evidence: Improving fact-checking by justification modeling. In Proceedings of the First Workshop on Fact Extraction and VERification (FEVER), pages 85–90, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Tariq+Alhindi%2C+Savvas+Petridis%2C+and+Smaranda+Muresan%2E+2018%2E+Where+is+your+evidence%3A+Improving+fact%2Dchecking+by+justification+modeling%2E+In+Proceedings+of+the+First+Workshop+on+Fact+Extraction+and+VERif)
2. Ron Artstein and Massimo Poesio. 2008. Inter-coder agreement for computational linguistics. Comput. Linguist., 34(4):555–596. [find](https://scholar.google.com/scholar?q=Ron+Artstein+and+Massimo+Poesio%2E+2008%2E+Inter%2Dcoder+agreement+for+computational+linguistics%2E+Comput%2E+Linguist%2E%2C+34%284%29%3A555%E2%80%93596%2E)
3. Isabelle Augenstein, Christina Lioma, Dongsheng Wang, Lucas Chaves Lima, Casper Hansen, Christian Hansen, and Jakob Grue Simonsen. 2019. MultiFC: A real-world multi-domain dataset for evidence-based fact checking of claims. In Proceedings of the 2019 Conference on Empirical Methods in Natural Langua… [find](https://scholar.google.com/scholar?q=Isabelle+Augenstein%2C+Christina+Lioma%2C+Dongsheng+Wang%2C+Lucas+Chaves+Lima%2C+Casper+Hansen%2C+Christian+Hansen%2C+and+Jakob+Grue+Simonsen%2E+2019%2E+MultiFC%3A+A+real%2Dworld+multi%2Ddomain+dataset+for+evidence%2Dbased+f) [our card](https://md.research.elamur.ai/free/en/papers/augenstein-2019-multifc-real-world-multi-domain/)
4. Ricardo Campos, Vı́tor Mangaravite, Arian Pasquali, Alı́pio Mário Jorge, Célia Nunes, and Adam Jatowt. 2018. Yake! collection-independent automatic keyword extractor. In European Conference on Information Retrieval, pages 806–810. Springer. [find](https://scholar.google.com/scholar?q=Ricardo+Campos%2C+V%C4%B1%CC%81tor+Mangaravite%2C+Arian+Pasquali%2C+Al%C4%B1%CC%81pio+Ma%CC%81rio+Jorge%2C+Ce%CC%81lia+Nunes%2C+and+Adam+Jatowt%2E+2018%2E+Yake%21+collection%2Dindependent+automatic+keyword+extractor%2E+In+European+Conference+on+Infor)
5. Ricardo Campos, Vı́tor Mangaravite, Arian Pasquali, Alı́pio Jorge, Célia Nunes, and Adam Jatowt. 2020. Yake! keyword extraction from single documents using multiple local features. Information Sciences, 509:257 – 289. [find](https://scholar.google.com/scholar?q=Ricardo+Campos%2C+V%C4%B1%CC%81tor+Mangaravite%2C+Arian+Pasquali%2C+Al%C4%B1%CC%81pio+Jorge%2C+Ce%CC%81lia+Nunes%2C+and+Adam+Jatowt%2E+2020%2E+Yake%21+keyword+extraction+from+single+documents+using+multiple+local+features%2E+Information+Scienc)
6. Jacob Cohen. 1968. Weighted kappa: nominal scale agreement with provision for scaled disagreement or partial credit. Psychological bulletin, pages 213– 220. [find](https://scholar.google.com/scholar?q=Jacob+Cohen%2E+1968%2E+Weighted+kappa%3A+nominal+scale+agreement+with+provision+for+scaled+disagreement+or+partial+credit%2E+Psychological+bulletin%2C+pages+213%E2%80%93+220%2E)
7. Johannes Daxenberger, Steffen Eger, Ivan Habernal, Christian Stab, and Iryna Gurevych. 2017. What is the essence of a claim? cross-domain claim identification. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2055–2066, Copenhagen, Denmark. Association… [find](https://scholar.google.com/scholar?q=Johannes+Daxenberger%2C+Steffen+Eger%2C+Ivan+Habernal%2C+Christian+Stab%2C+and+Iryna+Gurevych%2E+2017%2E+What+is+the+essence+of+a+claim%3F+cross%2Ddomain+claim+identification%2E+In+Proceedings+of+the+2017+Conference+on)
8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proceedings+of+the+2019+Conference+of+the+N) [our card](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/)
9. Thomas Diggelmann, Jordan Boyd-Graber, Jannis Bulian, Massimiliano Ciaramita, and Markus Leippold. 2021. Climate-fever: A dataset for verification of real-world climate claims. [find](https://scholar.google.com/scholar?q=Thomas+Diggelmann%2C+Jordan+Boyd%2DGraber%2C+Jannis+Bulian%2C+Massimiliano+Ciaramita%2C+and+Markus+Leippold%2E+2021%2E+Climate%2Dfever%3A+A+dataset+for+verification+of+real%2Dworld+climate+claims%2E) [our card](https://md.research.elamur.ai/free/en/papers/diggelmann-2020-climate-fever-dataset-verification-real/)
10. Chris Donahue, Mina Lee, and Percy Liang. 2020. Enabling language models to fill in the blanks. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2492– 2501, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Chris+Donahue%2C+Mina+Lee%2C+and+Percy+Liang%2E+2020%2E+Enabling+language+models+to+fill+in+the+blanks%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+2492%E2%80%93+) [our card](https://md.research.elamur.ai/free/en/papers/donahue-2020-enabling-language-models-fill-blanks/)
11. Andreas Hanselowski, Christian Stab, Claudia Schulz, Zile Li, and Iryna Gurevych. 2019. A richly annotated corpus for different tasks in automated fact-checking. In Proceedings of the 23rd Conference on Computational Natural Language Learning (CoNLL), pages 493–503, Hong Kong, China. Association for… [find](https://scholar.google.com/scholar?q=Andreas+Hanselowski%2C+Christian+Stab%2C+Claudia+Schulz%2C+Zile+Li%2C+and+Iryna+Gurevych%2E+2019%2E+A+richly+annotated+corpus+for+different+tasks+in+automated+fact%2Dchecking%2E+In+Proceedings+of+the+23rd+Conference+)
12. Christopher Hidey, Tuhin Chakrabarty, Tariq Alhindi, Siddharth Varia, Kriste Krstovski, Mona Diab, and Smaranda Muresan. 2020. DeSePtion: Dual sequence prediction and adversarial examples for improved fact-checking. In Proceedings of the 58th Annual Meeting of the Association for Computational Lingu… [find](https://scholar.google.com/scholar?q=Christopher+Hidey%2C+Tuhin+Chakrabarty%2C+Tariq+Alhindi%2C+Siddharth+Varia%2C+Kriste+Krstovski%2C+Mona+Diab%2C+and+Smaranda+Muresan%2E+2020%2E+DeSePtion%3A+Dual+sequence+prediction+and+adversarial+examples+for+improved)
13. Tamanna Hossain, Robert L. Logan IV, Arjuna Ugarte, Yoshitomo Matsubara, Sean Young, and Sameer Singh. 2020. COVIDLies: Detecting COVID-19 misinformation on social media. In Proceedings of the 1st Workshop on NLP for COVID-19 (Part 2) at EMNLP 2020, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Tamanna+Hossain%2C+Robert+L%2E+Logan+IV%2C+Arjuna+Ugarte%2C+Yoshitomo+Matsubara%2C+Sean+Young%2C+and+Sameer+Singh%2E+2020%2E+COVIDLies%3A+Detecting+COVID%2D19+misinformation+on+social+media%2E+In+Proceedings+of+the+1st+Wor)
14. Yichen Jiang, Shikha Bordia, Zheng Zhong, Charles Dognin, Maneesh Singh, and Mohit Bansal. 2020. HoVer: A dataset for many-hop fact extraction and claim verification. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3441–3460, Online. Association for Computational Ling… [find](https://scholar.google.com/scholar?q=Yichen+Jiang%2C+Shikha+Bordia%2C+Zheng+Zhong%2C+Charles+Dognin%2C+Maneesh+Singh%2C+and+Mohit+Bansal%2E+2020%2E+HoVer%3A+A+dataset+for+many%2Dhop+fact+extraction+and+claim+verification%2E+In+Findings+of+the+Association+fo) [our card](https://md.research.elamur.ai/free/en/papers/jiang-2020-hover-dataset-many-hop-fact/)
15. Nayeon Lee, Yejin Bang, Andrea Madotto, and Pascale Fung. 2020. Misinformation has high perplexity. [find](https://scholar.google.com/scholar?q=Nayeon+Lee%2C+Yejin+Bang%2C+Andrea+Madotto%2C+and+Pascale+Fung%2E+2020%2E+Misinformation+has+high+perplexity%2E)
16. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the… [find](https://scholar.google.com/scholar?q=Mike+Lewis%2C+Yinhan+Liu%2C+Naman+Goyal%2C+Marjan+Ghazvininejad%2C+Abdelrahman+Mohamed%2C+Omer+Levy%2C+Veselin+Stoyanov%2C+and+Luke+Zettlemoyer%2E+2020%2E+BART%3A+Denoising+sequence%2Dto%2Dsequence+pre%2Dtraining+for+natural+l) [our card](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/)
17. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Roberta%3A+A+robustly+optimized+BERT+pretraining+approach%2E+C)
18. Tsvetomila Mihaylova, Georgi Karadzhov, Pepa Atanasova, Ramy Baly, Mitra Mohtarami, and Preslav Nakov. 2019. Semeval-2019 task 8: Fact checking in community question answering forums. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 860–869. [find](https://scholar.google.com/scholar?q=Tsvetomila+Mihaylova%2C+Georgi+Karadzhov%2C+Pepa+Atanasova%2C+Ramy+Baly%2C+Mitra+Mohtarami%2C+and+Preslav+Nakov%2E+2019%2E+Semeval%2D2019+task+8%3A+Fact+checking+in+community+question+answering+forums%2E+In+Proceedings+o)
19. Tsvetomila Mihaylova, Preslav Nakov, Lluı́s Màrquez, Alberto Barrón-Cedeño, Mitra Mohtarami, Georgi Karadzhov, and James R. Glass. 2018. Fact checking in community forums. CoRR, abs/1803.03178. — [arXiv:1803.03178](https://arxiv.org/abs/1803.03178) [find](https://scholar.google.com/scholar?q=Tsvetomila+Mihaylova%2C+Preslav+Nakov%2C+Llu%C4%B1%CC%81s+Ma%CC%80rquez%2C+Alberto+Barro%CC%81n%2DCeden%CC%83o%2C+Mitra+Mohtarami%2C+Georgi+Karadzhov%2C+and+James+R%2E+Glass%2E+2018%2E+Fact+checking+in+community+forums%2E+CoRR%2C+abs%2F1803%2E03178%2E+%E2%80%94+%5B)
20. George A Miller. 1995. Wordnet: a lexical database for english. Communications of the ACM, 38(11):39– 41. [find](https://scholar.google.com/scholar?q=George+A+Miller%2E+1995%2E+Wordnet%3A+a+lexical+database+for+english%2E+Communications+of+the+ACM%2C+38%2811%29%3A39%E2%80%93+41%2E) [our card](https://md.research.elamur.ai/free/en/papers/miller-1994-wordnet/)
21. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019a. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonst… [find](https://scholar.google.com/scholar?q=Myle+Ott%2C+Sergey+Edunov%2C+Alexei+Baevski%2C+Angela+Fan%2C+Sam+Gross%2C+Nathan+Ng%2C+David+Grangier%2C+and+Michael+Auli%2E+2019a%2E+fairseq%3A+A+fast%2C+extensible+toolkit+for+sequence+modeling%2E+In+Proceedings+of+the+201) [our card](https://md.research.elamur.ai/free/en/papers/ott-2019-fairseq-fast-extensible-toolkit-sequence/)
22. Myle Ott, Sergey Edunov, Alexei Baevski, Angela Fan, Sam Gross, Nathan Ng, David Grangier, and Michael Auli. 2019b. fairseq: A fast, extensible toolkit for sequence modeling. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics (Demonst… [find](https://scholar.google.com/scholar?q=Myle+Ott%2C+Sergey+Edunov%2C+Alexei+Baevski%2C+Angela+Fan%2C+Sam+Gross%2C+Nathan+Ng%2C+David+Grangier%2C+and+Michael+Auli%2E+2019b%2E+fairseq%3A+A+fast%2C+extensible+toolkit+for+sequence+modeling%2E+In+Proceedings+of+the+201) [our card](https://md.research.elamur.ai/free/en/papers/ott-2019-fairseq-fast-extensible-toolkit-sequence/)
23. Adam Poliak, Max Fleming, Cash Costello, Kenton W Murray, Mahsa Yarmohammadi, Shivani Pandya, Darius Irani, Milind Agarwal, Udit Sharma, Shuo Sun, Nicola Ivanov, Lingxi Shang, Kaushik Srinivasan, Seolhwa Lee, Xu Han, Smisha Agarwal, and João Sedoc. 2020. Collecting verified COVID-19 question answer… [find](https://scholar.google.com/scholar?q=Adam+Poliak%2C+Max+Fleming%2C+Cash+Costello%2C+Kenton+W+Murray%2C+Mahsa+Yarmohammadi%2C+Shivani+Pandya%2C+Darius+Irani%2C+Milind+Agarwal%2C+Udit+Sharma%2C+Shuo+Sun%2C+Nicola+Ivanov%2C+Lingxi+Shang%2C+Kaushik+Srinivasan%2C+Seol)
24. Dean Pomerleau and Delip Rao. 2017. Fake news challenge. [find](https://scholar.google.com/scholar?q=Dean+Pomerleau+and+Delip+Rao%2E+2017%2E+Fake+news+challenge%2E)
25. Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence embeddings using Siamese BERT-networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,… [find](https://scholar.google.com/scholar?q=Nils+Reimers+and+Iryna+Gurevych%2E+2019%2E+Sentence%2DBERT%3A+Sentence+embeddings+using+Siamese+BERT%2Dnetworks%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the) [our card](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/)
26. Akhilesh Sudhakar, Bhargav Upadhyay, and Arjun Maheswaran. 2019. “transforming” delete, retrieve, generate approach for controlled text style transfer. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Lang… [find](https://scholar.google.com/scholar?q=Akhilesh+Sudhakar%2C+Bhargav+Upadhyay%2C+and+Arjun+Maheswaran%2E+2019%2E+%E2%80%9Ctransforming%E2%80%9D+delete%2C+retrieve%2C+generate+approach+for+controlled+text+style+transfer%2E+In+Proceedings+of+the+2019+Conference+on+Empiric) [our card](https://md.research.elamur.ai/free/en/papers/sudhakar-2019-transforming-delete-retrieve-generate-approach/)
27. James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2018. Fever: a large-scale dataset for fact extraction and verification. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Vo… [find](https://scholar.google.com/scholar?q=James+Thorne%2C+Andreas+Vlachos%2C+Christos+Christodoulopoulos%2C+and+Arpit+Mittal%2E+2018%2E+Fever%3A+a+large%2Dscale+dataset+for+fact+extraction+and+verification%2E+In+Proceedings+of+the+2018+Conference+of+the+Nort) [our card](https://md.research.elamur.ai/free/en/papers/thorne-2018-fever-large-scale-dataset-fact/)
28. James Thorne, Andreas Vlachos, Oana Cocarascu, Christos Christodoulopoulos, and Arpit Mittal. 2019. The FEVER2.0 shared task. In Proceedings of the Second Workshop on Fact Extraction and VERification (FEVER), pages 1–6, Hong Kong, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=James+Thorne%2C+Andreas+Vlachos%2C+Oana+Cocarascu%2C+Christos+Christodoulopoulos%2C+and+Arpit+Mittal%2E+2019%2E+The+FEVER2%2E0+shared+task%2E+In+Proceedings+of+the+Second+Workshop+on+Fact+Extraction+and+VERification+)
29. Andreas Vlachos and Sebastian Riedel. 2014. Fact checking: Task definition and dataset construction. In Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science, pages 18–22. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Andreas+Vlachos+and+Sebastian+Riedel%2E+2014%2E+Fact+checking%3A+Task+definition+and+dataset+construction%2E+In+Proceedings+of+the+ACL+2014+Workshop+on+Language+Technologies+and+Computational+Social+Science%2C+) [our card](https://md.research.elamur.ai/free/en/papers/vlachos-2014-fact-checking-task-definition-dataset/)
30. Soroush Vosoughi, Deb Roy, and Sinan Aral. 2018. The spread of true and false news online. Science, 359(6380):1146–1151. [find](https://scholar.google.com/scholar?q=Soroush+Vosoughi%2C+Deb+Roy%2C+and+Sinan+Aral%2E+2018%2E+The+spread+of+true+and+false+news+online%2E+Science%2C+359%286380%29%3A1146%E2%80%931151%2E)
31. David Wadden, Shanchuan Lin, Kyle Lo, Lucy Lu Wang, Madeleine van Zuylen, Arman Cohan, and Hannaneh Hajishirzi. 2020. Fact or fiction: Verifying scientific claims. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7534–7550, Online. Association… [find](https://scholar.google.com/scholar?q=David+Wadden%2C+Shanchuan+Lin%2C+Kyle+Lo%2C+Lucy+Lu+Wang%2C+Madeleine+van+Zuylen%2C+Arman+Cohan%2C+and+Hannaneh+Hajishirzi%2E+2020%2E+Fact+or+fiction%3A+Verifying+scientific+claims%2E+In+Proceedings+of+the+2020+Conferenc) [our card](https://md.research.elamur.ai/free/en/papers/wadden-2020-fact-or-fiction-verifying-scientific/)
32. Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural language understanding. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 353–355,… [find](https://scholar.google.com/scholar?q=Alex+Wang%2C+Amanpreet+Singh%2C+Julian+Michael%2C+Felix+Hill%2C+Omer+Levy%2C+and+Samuel+Bowman%2E+2018%2E+GLUE%3A+A+multi%2Dtask+benchmark+and+analysis+platform+for+natural+language+understanding%2E+In+Proceedings+of+the) [our card](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/)
33. Lucy Lu Wang, Kyle Lo, Yoganand Chandrasekhar, Russell Reas, Jiangjiang Yang, Doug Burdick, Darrin Eide, Kathryn Funk, Yannis Katsis, Rodney Michael Kinney, Yunyao Li, Ziyang Liu, William Merrill, Paul Mooney, Dewey A. Murdick, Devvret Rishi, Jerry Sheehan, Zhihong Shen, Brandon Stilson, Alex D. Wad… [find](https://scholar.google.com/scholar?q=Lucy+Lu+Wang%2C+Kyle+Lo%2C+Yoganand+Chandrasekhar%2C+Russell+Reas%2C+Jiangjiang+Yang%2C+Doug+Burdick%2C+Darrin+Eide%2C+Kathryn+Funk%2C+Yannis+Katsis%2C+Rodney+Michael+Kinney%2C+Yunyao+Li%2C+Ziyang+Liu%2C+William+Merrill%2C+Pau) [our card](https://md.research.elamur.ai/free/en/papers/wang-2020-cord-19-covid-19-open/)
34. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. A broad-coverage challenge corpus for sentence understanding through inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Pa… [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Nikita+Nangia%2C+and+Samuel+Bowman%2E+2018%2E+A+broad%2Dcoverage+challenge+corpus+for+sentence+understanding+through+inference%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chap)
35. Edwin Zhang, Nikhil Gupta, Rodrigo Nogueira, Kyunghyun Cho, and Jimmy Lin. 2020a. Rapidly deploying a neural search engine for the COVID-19 Open Research Dataset. In Proceedings of the 1st Workshop on NLP for COVID-19 at ACL 2020, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Edwin+Zhang%2C+Nikhil+Gupta%2C+Rodrigo+Nogueira%2C+Kyunghyun+Cho%2C+and+Jimmy+Lin%2E+2020a%2E+Rapidly+deploying+a+neural+search+engine+for+the+COVID%2D19+Open+Research+Dataset%2E+In+Proceedings+of+the+1st+Workshop+on)
36. Yizhe Zhang, Guoyin Wang, Chunyuan Li, Zhe Gan, Chris Brockett, and Bill Dolan. 2020b. POINTER: Constrained progressive text generation via insertion-based generative pre-training. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 8649–8670, Onl… [find](https://scholar.google.com/scholar?q=Yizhe+Zhang%2C+Guoyin+Wang%2C+Chunyuan+Li%2C+Zhe+Gan%2C+Chris+Brockett%2C+and+Bill+Dolan%2E+2020b%2E+POINTER%3A+Constrained+progressive+text+generation+via+insertion%2Dbased+generative+pre%2Dtraining%2E+In+Proceedings+of+t)