---
title: "FEVER: a Large-scale Dataset for Fact — Thorne 2018"
description: "Before FEVER, datasets for automated fact checking held only a few hundred claims."
authors: ["James Thorne", "Andreas Vlachos", "Christos Christodoulopoulos", "Arpit Mittal"]
year: 2018
venue: "Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)"
source: "https://aclanthology.org/N18-1074/"
doi: "10.18653/v1/n18-1074"
arxiv: "1803.05355"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/thorne-2018-fever-large-scale-dataset-fact/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/thorne-2018-fever-large-scale-dataset-fact/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/) / FEVER: a Large-scale Dataset for Fact Extraction and VERification

# FEVER: a Large-scale Dataset for Fact Extraction and VERification

**Authors:** [James Thorne](https://md.research.elamur.ai/free/en/authors/james-thorne/), [Andreas Vlachos](https://md.research.elamur.ai/free/en/authors/andreas-vlachos/), [Christos Christodoulopoulos](https://md.research.elamur.ai/free/en/authors/christos-christodoulopoulos/), [Arpit Mittal](https://md.research.elamur.ai/free/en/authors/arpit-mittal/) · **Year:** 2018 · **Venue:** Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/N18-1074/) · [PDF](https://aclanthology.org/N18-1074.pdf) · [DOI 10.18653/v1/n18-1074](https://doi.org/10.18653/v1/n18-1074) · [arXiv 1803.05355](https://arxiv.org/abs/1803.05355) · [OpenAlex](https://openalex.org/W2789566302)

**Citations:** 1287 citations (openalex-snapshot · 2026-10-14) · [918](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Before FEVER, datasets for automated fact checking held only a few hundred claims. Many of them also handed systems the source text instead of making them find it. FEVER provides 185,445 claims built from the introductory sections of about 50,000 popular Wikipedia pages (June 2017 dump). Annotators wrote short single-fact claims from sampled sentences. They then altered those claims with six mutation types inspired by natural logic, such as negation, entity substitution and generalization. In a separate pass, other annotators labelled each claim SUPPORTED, REFUTED or NOT ENOUGH INFO. For the first two labels, they also marked the sentences that justify the verdict, sometimes taken from several pages. To measure difficulty, the authors built a baseline pipeline with three steps: DrQA-based document retrieval, sentence selection, and a textual entailment classifier (decomposable attention). On the test set, it scores 31.87% accuracy when the correct evidence is required and 50.91% when evidence is ignored. Oracle runs show that selecting the evidence sentences is the hardest step. FEVER gives verification research a large testbed with evidence annotations.

## Key points
- How much evidence claims need: 31.75% of claims have more than one valid evidence set, 16.82% need several sentences combined, and 12.15% need evidence from more than one Wikipedia page.
- Annotation quality: the labels reach a Fleiss κ of 0.6841. Evidence selection has 95.42% precision and 72.36% recall. Claims average 9.4 tokens, close to the hypothesis length in SNLI.
- Baseline pipeline on the test set: top-5 DrQA documents, then 5 retrieved sentences, then a Decomposable Attention model trained with the NEARESTP sampling strategy for NOT ENOUGH INFO examples. It reaches 31.87% when the correct evidence is required and 50.91% when it is not.
- Learning curves: Decomposable Attention is unstable with fewer than about 6,000 training examples, then overtakes an MLP baseline as data grows. This suggests the dataset is large enough to tell models apart.
- Released resources: the data at fever.ai, the annotation interfaces (github.com/awslabs/fever) and the baseline system (github.com/sheffieldnlp/fever-baselines).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

918 papers of our library cite it (of the 140884 whose bibliographies we read); 96 of them have a page here, 822 not yet.

- [Zero-Shot Stance Detection: A Dataset and Model using Generalized Topic Representations](https://md.research.elamur.ai/free/en/papers/allaway-2020-zero-shot-stance-detection-dataset/)
- [Evidentiality-guided Generation for Knowledge-Intensive NLP Tasks](https://md.research.elamur.ai/free/en/papers/asai-2022-evidentiality-guided-generation-knowledge-intensive/)
- [Self-RAG: Learning to Retrieve, Generate, and Critique through Self-Reflection](https://md.research.elamur.ai/free/en/papers/asai-2023-self-rag-learning-retrieve-generate/)
- [Task-aware Retrieval with Instructions](https://md.research.elamur.ai/free/en/papers/asai-2023-task-aware-retrieval-instructions/)
- [Generating Fact Checking Explanations](https://md.research.elamur.ai/free/en/papers/atanasova-2020-generating-fact-checking-explanations/)
- [MultiFC: A Real-World Multi-Domain Dataset for Evidence-Based Fact Checking of Claims](https://md.research.elamur.ai/free/en/papers/augenstein-2019-multifc-real-world-multi-domain/)
- [The Internal State of an LLM Knows When It's Lying](https://md.research.elamur.ai/free/en/papers/azaria-2023-internal-state-llm-knows-when/)
- [Integrating Stance Detection and Fact Checking in a Unified Corpus](https://md.research.elamur.ai/free/en/papers/baly-2018-integrating-stance-detection-fact-checking/)
- [Predicting Factuality of Reporting and Bias of News Media Sources](https://md.research.elamur.ai/free/en/papers/baly-2018-predicting-factuality-reporting-bias-news/)
- [LLM2Vec: Large Language Models Are Secretly Powerful Text Encoders](https://md.research.elamur.ai/free/en/papers/behnamghader-2024-llm2vec-large-language-models-secretly/)
- [Attributed Question Answering: Evaluation and Modeling for Attributed Large Language Models](https://md.research.elamur.ai/free/en/papers/bohnet-2022-attributed-question-answering-evaluation-modeling/)
- [TabFact: A Large-scale Dataset for Table-based Fact Verification](https://md.research.elamur.ai/free/en/papers/chen-2019-tabfact-large-scale-dataset-table/)
- [FacTool: Factuality Detection in Generative AI -- A Tool Augmented Framework for Multi-Task and Multi-Domain Scenarios](https://md.research.elamur.ai/free/en/papers/chern-2023-factool-factuality-detection-generative-ai/)
- [STANDER: An Expert-Annotated Dataset for News Stance Detection and Evidence Retrieval](https://md.research.elamur.ai/free/en/papers/conforti-2020-stander-expert-annotated-dataset-news/)
- [Promptagator: Few-shot Dense Retrieval From 8 Examples](https://md.research.elamur.ai/free/en/papers/dai-2022-promptagator-few-shot-dense-retrieval/)
- [Explaining Answers with Entailment Trees](https://md.research.elamur.ai/free/en/papers/dalvi-2021-explaining-answers-entailment-trees/)
- [ERASER: A Benchmark to Evaluate Rationalized NLP Models](https://md.research.elamur.ai/free/en/papers/deyoung-2020-eraser-benchmark-evaluate-rationalized-nlp/)
- [CLIMATE-FEVER: A Dataset for Verification of Real-World Climate Claims](https://md.research.elamur.ai/free/en/papers/diggelmann-2020-climate-fever-dataset-verification-real/)
- [Overview of the CLEF-2019 CheckThat!: Automatic Identification and Verification of Claims](https://md.research.elamur.ai/free/en/papers/elsayed-2021-overview-clef-2019-checkthat-automatic/)
- [Enabling Large Language Models to Generate Text with Citations](https://md.research.elamur.ai/free/en/papers/gao-2023-enabling-large-language-models-generate/)
- [RARR: Researching and Revising What Language Models Say, Using Language Models](https://md.research.elamur.ai/free/en/papers/gao-2023-rarr-researching-revising-what-language/)
- [Retrieval-Augmented Generation for Large Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/gao-2023-retrieval-augmented-generation-large-language/)
- [Re2G: Retrieve, Rerank, Generate](https://md.research.elamur.ai/free/en/papers/glass-2022-re2g-retrieve-rerank-generate/)
- [CRITIC: Large Language Models Can Self-Correct with Tool-Interactive Critiquing](https://md.research.elamur.ai/free/en/papers/gou-2023-critic-large-language-models-self/)
- [Annotating and Modeling Fine-grained Factuality in Summarization](https://md.research.elamur.ai/free/en/papers/goyal-2021-annotating-modeling-fine-grained-factuality/)
- [A Survey on Automated Fact-Checking](https://md.research.elamur.ai/free/en/papers/guo-2022-survey-automated-fact-checking/)
- [UKP-Athene: Multi-Sentence Textual Entailment for Claim Verification](https://md.research.elamur.ai/free/en/papers/hanselowski-2018-ukp-athene-multi-sentence-textual/)
- [A Survey on Stance Detection for Mis- and Disinformation Identification](https://md.research.elamur.ai/free/en/papers/hardalov-2022-survey-stance-detection-mis-disinformation/)
- [Transformer-Patcher: One Mistake worth One Neuron](https://md.research.elamur.ai/free/en/papers/huang-2023-transformer-patcher-one-mistake-worth/)
- [Unsupervised Dense Information Retrieval with Contrastive Learning](https://md.research.elamur.ai/free/en/papers/izacard-2021-unsupervised-dense-information-retrieval-contrastive/)
- [Survey of Hallucination in Natural Language Generation](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/)
- [HoVer: A Dataset for Many-Hop Fact Extraction And Claim Verification](https://md.research.elamur.ai/free/en/papers/jiang-2020-hover-dataset-many-hop-fact/)
- [SMART: Robust and Efficient Fine-Tuning for Pre-trained Natural Language Models through Principled Regularized Optimization](https://md.research.elamur.ai/free/en/papers/jiang-2020-smart-robust-efficient-fine-tuning/)
- [BIDER: Bridging Knowledge Inconsistency for Efficient Retrieval-Augmented LLMs via Key Supporting Evidence](https://md.research.elamur.ai/free/en/papers/jin-2024-bider-bridging-knowledge-inconsistency-efficient/)
- [ARAFA: An LLM-Generated Arabic Fact-Checking Dataset](https://md.research.elamur.ai/free/en/papers/khalil-2026-arafa-llm-generated-arabic-fact/)
- [Demonstrate-Search-Predict: Composing retrieval and language models for knowledge-intensive NLP](https://md.research.elamur.ai/free/en/papers/khattab-2022-demonstrate-search-predict-composing-retrieval/)
- [ContractNLI: A Dataset for Document-level Natural Language Inference for Contracts](https://md.research.elamur.ai/free/en/papers/koreeda-2021-contractnli-dataset-document-level-natural/)
- [Explainable Automated Fact-Checking for Public Health Claims](https://md.research.elamur.ai/free/en/papers/kotonya-2020-explainable-automated-fact-checking-public/)
- [LongEval: Guidelines for Human Evaluation of Faithfulness in Long-form Summarization](https://md.research.elamur.ai/free/en/papers/krishna-2023-longeval-guidelines-human-evaluation-faithfulness/)
- [Internet-augmented language models through few-shot prompting for open-domain question answering](https://md.research.elamur.ai/free/en/papers/lazaridou-2022-internet-augmented-language-models-through/)
- [NV-Embed: Improved Techniques for Training LLMs as Generalist Embedding Models](https://md.research.elamur.ai/free/en/papers/lee-2024-nv-embed-improved-techniques-training/)
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/)
- [RA-DIT: Retrieval-Augmented Dual Instruction Tuning](https://md.research.elamur.ai/free/en/papers/lin-2023-ra-dit-retrieval-augmented-dual/)
- [Fine-grained Fact Verification with Kernel Graph Attention Network](https://md.research.elamur.ai/free/en/papers/liu-2020-fine-grained-fact-verification-kernel/)
- [A Token-level Reference-free Hallucination Detection Benchmark for Free-form Text Generation](https://md.research.elamur.ai/free/en/papers/liu-2022-token-level-reference-free-hallucination/)
- [WANLI: Worker and AI Collaboration for Natural Language Inference Dataset Creation](https://md.research.elamur.ai/free/en/papers/liu-2022-wanli-worker-ai-collaboration-natural/)
- [Fine-Tuning LLaMA for Multi-Stage Text Retrieval](https://md.research.elamur.ai/free/en/papers/ma-2023-fine-tuning-llama-multi-stage/)
- [SemEval-2020 Task 11: Detection of Propaganda Techniques in News Articles](https://md.research.elamur.ai/free/en/papers/martino-2020-semeval-2020-task-11-detection/)
- [Constructing a Japanese Claim Decomposition Dataset for Fact-Checking of LLM-Generated Texts](https://md.research.elamur.ai/free/en/papers/masano-2026-constructing-japanese-claim-decomposition-dataset/)
- [Teaching language models to support answers with verified quotes](https://md.research.elamur.ai/free/en/papers/menick-2022-teaching-language-models-support-answers/)
- [Recent Advances in Natural Language Processing via Large Pre-Trained Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/min-2021-recent-advances-natural-language-processing/)
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://md.research.elamur.ai/free/en/papers/min-2023-factscore-fine-grained-atomic-evaluation/)
- [Automatic Stance Detection Using End-to-End Memory Networks](https://md.research.elamur.ai/free/en/papers/mohtarami-2018-automatic-stance-detection-end-end/)
- [Generative Representational Instruction Tuning](https://md.research.elamur.ai/free/en/papers/muennighoff-2024-generative-representational-instruction-tuning/)
- [A Comprehensive Overview of Large Language Models](https://md.research.elamur.ai/free/en/papers/naveed-2023-comprehensive-overview-large-language-models/)
- [Nomic Embed: Training a Reproducible Long Context Text Embedder](https://md.research.elamur.ai/free/en/papers/nussbaum-2024-nomic-embed-training-reproducible-long/)
- [Fact-Checking Complex Claims with Program-Guided Reasoning](https://md.research.elamur.ai/free/en/papers/pan-2023-fact-checking-complex-claims-program/)
- [Better Scores, Worse Grounding: Hidden Regressions after Fine-Tuning in Dialogue Fact Verification](https://md.research.elamur.ai/free/en/papers/park-2026-better-scores-worse-grounding-hidden/)
- [Unsupervised Question Decomposition for Question Answering](https://md.research.elamur.ai/free/en/papers/perez-2020-unsupervised-question-decomposition-question-answering/)
- [KILT: a Benchmark for Knowledge Intensive Language Tasks](https://md.research.elamur.ai/free/en/papers/petroni-2020-kilt-benchmark-knowledge-intensive-language/)
- [Automatic Detection of Fake News](https://md.research.elamur.ai/free/en/papers/rosas-2018-automatic-detection-fake-news/)
- [COVID-Fact: Fact Extraction and Verification of Real-World Claims on COVID-19 Pandemic](https://md.research.elamur.ai/free/en/papers/saakyan-2021-covid-fact-fact-extraction-verification/)
- [From Articles to Premises: Building PrimeFacts, an Extraction Methodology and Resource for Fact-Checking Evidence](https://md.research.elamur.ai/free/en/papers/sahitaj-2026-articles-premises-building-primefacts-extraction/)
- [ColBERTv2: Effective and Efficient Retrieval via Lightweight Late Interaction](https://md.research.elamur.ai/free/en/papers/santhanam-2022-colbertv2-effective-efficient-retrieval-lightweight/)
- [Towards Debiasing Fact Verification Models](https://md.research.elamur.ai/free/en/papers/schuster-2019-debiasing-fact-verification-models/)
- [Get Your Vitamin C! Robust Fact Verification with Contrastive Evidence](https://md.research.elamur.ai/free/en/papers/schuster-2021-get-your-vitamin-c-robust/)
- [Enhancing Retrieval-Augmented Large Language Models with Iterative Retrieval-Generation Synergy](https://md.research.elamur.ai/free/en/papers/shao-2023-enhancing-retrieval-augmented-large-language/)
- [Prompting GPT-3 To Be Reliable](https://md.research.elamur.ai/free/en/papers/si-2022-prompting-gpt-3-reliable/)
- [How Often Are Errors in Natural Language Reasoning Due to Paraphrastic Variability?](https://md.research.elamur.ai/free/en/papers/srikanth-2024-how-often-are-errors-natural/)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/)
- [One Embedder, Any Task: Instruction-Finetuned Text Embeddings](https://md.research.elamur.ai/free/en/papers/su-2022-one-embedder-any-task-instruction/)
- [MultiHop-RAG: Benchmarking Retrieval-Augmented Generation for Multi-Hop Queries](https://md.research.elamur.ai/free/en/papers/tang-2024-multihop-rag-benchmarking-retrieval-augmented/)
- [BEIR: A Heterogenous Benchmark for Zero-shot Evaluation of Information Retrieval Models](https://md.research.elamur.ai/free/en/papers/thakur-2021-beir-heterogenous-benchmark-zero/)
- [Automated Fact Checking: Task Formulations, Methods and Future Directions](https://md.research.elamur.ai/free/en/papers/thorne-2018-automated-fact-checking-task-formulations/)
- [The Fact Extraction and VERification (FEVER) Shared Task](https://md.research.elamur.ai/free/en/papers/thorne-2018-fact-extraction-verification-fever-shared/)
- [Fact or Fiction: Verifying Scientific Claims](https://md.research.elamur.ai/free/en/papers/wadden-2020-fact-or-fiction-verifying-scientific/)
- [Text Embeddings by Weakly-Supervised Contrastive Pre-training](https://md.research.elamur.ai/free/en/papers/wang-2022-text-embeddings-weakly-supervised-contrastive/)
- [Survey on Factuality in Large Language Models: Knowledge, Retrieval and Domain-Specificity](https://md.research.elamur.ai/free/en/papers/wang-2023-survey-factuality-large-language-models/)
- [Improving Text Embeddings with Large Language Models](https://md.research.elamur.ai/free/en/papers/wang-2024-improving-text-embeddings-large-language/)
- [Multilingual E5 Text Embeddings: A Technical Report](https://md.research.elamur.ai/free/en/papers/wang-2024-multilingual-e5-text-embeddings-technical/)
- [Measuring Association Between Labels and Free-Text Rationales](https://md.research.elamur.ai/free/en/papers/wiegreffe-2021-measuring-association-between-labels-free/)
- [ReAct: Synergizing Reasoning and Acting in Language Models](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/)
- [TwoWingOS: A Two-Wing Optimization Strategy for Evidential Claim Verification](https://md.research.elamur.ai/free/en/papers/yin-2018-twowingos-two-wing-optimization-strategy/)
- [Benchmarking Zero-shot Text Classification: Datasets, Evaluation and Entailment Approach](https://md.research.elamur.ai/free/en/papers/yin-2019-benchmarking-zero-shot-text-classification/)
- [DocNLI: A Large-scale Dataset for Document-level Natural Language Inference](https://md.research.elamur.ai/free/en/papers/yin-2021-docnli-large-scale-dataset-document/)
- [Making Retrieval-Augmented Language Models Robust to Irrelevant Context](https://md.research.elamur.ai/free/en/papers/yoran-2023-making-retrieval-augmented-language-models/)
- [Generate rather than Retrieve: Large Language Models are Strong Context Generators](https://md.research.elamur.ai/free/en/papers/yu-2022-generate-rather-than-retrieve-large/)
- [Defending Against Neural Fake News](https://md.research.elamur.ai/free/en/papers/zellers-2019-defending-against-neural-fake-news/)
- [AlignScore: Evaluating Factual Consistency with A Unified Alignment Function](https://md.research.elamur.ai/free/en/papers/zha-2023-alignscore-evaluating-factual-consistency-unified/)
- [mGTE: Generalized Long-Context Text Representation and Reranking Models for Multilingual Text Retrieval](https://md.research.elamur.ai/free/en/papers/zhang-2024-mgte-generalized-long-context-text/)
- [Verify-and-Edit: A Knowledge-Enhanced Chain-of-Thought Framework](https://md.research.elamur.ai/free/en/papers/zhao-2023-verify-edit-knowledge-enhanced-chain/)
- [Stanceosaurus: Classifying Stance Towards Multilingual Misinformation](https://md.research.elamur.ai/free/en/papers/zheng-2022-stanceosaurus-classifying-stance-towards-multilingual/)
- [Reasoning Over Semantic-Level Graph for Fact Checking](https://md.research.elamur.ai/free/en/papers/zhong-2020-reasoning-over-semantic-level-graph/)
- [AGIEval: A Human-Centric Benchmark for Evaluating Foundation Models](https://md.research.elamur.ai/free/en/papers/zhong-2024-agieval-human-centric-benchmark-evaluating/)
- [GEAR: Graph-based Evidence Aggregating and Reasoning for Fact Verification](https://md.research.elamur.ai/free/en/papers/zhou-2019-gear-graph-based-evidence-aggregating/)
- [ToolQA: A Dataset for LLM Question Answering with External Tools](https://md.research.elamur.ai/free/en/papers/zhuang-2023-toolqa-dataset-llm-question-answering/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{thorne2018fever,
  title = {{FEVER: a Large-scale Dataset for Fact Extraction and VERification}},
  author = {Thorne, James and Vlachos, Andreas and Christodoulopoulos, Christos and Mittal, Arpit},
  booktitle = {Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers)},
  year = {2018},
  doi = {10.18653/v1/n18-1074},
  eprint = {1803.05355},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/n18-1074}
}
```

- [← previous Fast Lexically Constrained Decoding with Dynamic Beam Allocation for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/post-2018-fast-lexically-constrained-decoding-dynamic/)
- [next → Few-Shot Charge Prediction with Discriminative Legal Attributes](https://md.research.elamur.ai/free/en/papers/hu-2018-few-shot-charge-prediction-discriminative/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/2/)

## References

24 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Edward Loper and Steven Bird. 2002. Nltk: The natural language toolkit. In Proceedings of the ACL-02 Workshop on Effective Tools and Methodologies for Teaching Natural Language Processing and Computational Linguistics - Volume 1. Association for Computational Linguistics, Stroudsburg, PA, USA, ETMTN… — [doi:10.3115/1118108.1118117](https://doi.org/10.3115/1118108.1118117) [find](https://scholar.google.com/scholar?q=Edward+Loper+and+Steven+Bird%2E+2002%2E+Nltk%3A+The+natural+language+toolkit%2E+In+Proceedings+of+the+ACL%2D02+Workshop+on+Effective+Tools+and+Methodologies+for+Teaching+Natural+Language+Processing+and+Computat)
2. Gabor Angeli and Christopher D. Manning. 2014. Nat-uralLI: Natural logic inference for common sense reasoning. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. pages 534–545. [find](https://scholar.google.com/scholar?q=Gabor+Angeli+and+Christopher+D%2E+Manning%2E+2014%2E+Nat%2DuralLI%3A+Natural+logic+inference+for+common+sense+reasoning%2E+In+Proceedings+of+the+2014+Conference+on+Empirical+Methods+in+Natural+Language+Processing)
3. Samuel R. Bowman, Gabor Angeli, Christopher Potts, and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. [find](https://scholar.google.com/scholar?q=Samuel+R%2E+Bowman%2C+Gabor+Angeli%2C+Christopher+Potts%2C+and+Christopher+D%2E+Manning%2E+2015%2E+A+large+annotated+corpus+for+learning+natural+language+inference%2E+In+Proceedings+of+the+2015+Conference+on+Empirica)
4. Christopher D Manning, Mihai Surdeanu, John Bauer, Jenny Rose Finkel, Steven Bethard, and David Mc-Closky. 2014. The Stanford CoreNLP natural language processing toolkit. In ACL (System Demonstrations). pages 55–60. [find](https://scholar.google.com/scholar?q=Christopher+D+Manning%2C+Mihai+Surdeanu%2C+John+Bauer%2C+Jenny+Rose+Finkel%2C+Steven+Bethard%2C+and+David+Mc%2DClosky%2E+2014%2E+The+Stanford+CoreNLP+natural+language+processing+toolkit%2E+In+ACL+%28System+Demonstrations) [our card](https://md.research.elamur.ai/free/en/papers/manning-2014-stanford-corenlp-natural-language-processing/)
5. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. Reading wikipedia to answer open-domain questions. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, pages 1870–1879. https://d… [find](https://scholar.google.com/scholar?q=Danqi+Chen%2C+Adam+Fisch%2C+Jason+Weston%2C+and+Antoine+Bordes%2E+2017%2E+Reading+wikipedia+to+answer+open%2Ddomain+questions%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Computational+Lingui) [our card](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/)
6. Mausam, Michael Schmitz, Robert Bart, Stephen Soderland, and Oren Etzioni. 2012. Open language learning for information extraction. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning. pages 523–534. [find](https://scholar.google.com/scholar?q=Mausam%2C+Michael+Schmitz%2C+Robert+Bart%2C+Stephen+Soderland%2C+and+Oren+Etzioni%2E+2012%2E+Open+language+learning+for+information+extraction%2E+In+Proceedings+of+the+2012+Joint+Conference+on+Empirical+Methods+in+)
7. Ankur Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 2249–225… [find](https://scholar.google.com/scholar?q=Ankur+Parikh%2C+Oscar+Ta%CC%88ckstro%CC%88m%2C+Dipanjan+Das%2C+and+Jakob+Uszkoreit%2E+2016%2E+A+decomposable+attention+model+for+natural+language+inference%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+N) [our card](https://md.research.elamur.ai/free/en/papers/parikh-2016-decomposable-attention-model-natural-language/)
8. Ido Dagan, Bill Dolan, Bernardo Magnini, and Dan Roth. 2009. Recognizing textual entailment: Rational, evaluation and approaches. Natural Language Engineering 15(4):i–xvii. https://doi.org/10.1017/S1351324909990209. — [doi:10.1017/s1351324909990209](https://doi.org/10.1017/s1351324909990209) [↗](https://doi.org/10.1017/S1351324909990209)
9. Joe Ellis, Jeremy Getman, Dana Fore, Neil Kuster, Zhiyi Song, Ann Bies, and Stephanie Strassel. 2016. Overview of Linguistic Resources for the TAC KBP 2016 Evaluations : Methodologies and Results. Proceedings of TAC KBP 2016 Workshop, National Institute of Standards and Technology, Maryland, USA (Ld… [find](https://scholar.google.com/scholar?q=Joe+Ellis%2C+Jeremy+Getman%2C+Dana+Fore%2C+Neil+Kuster%2C+Zhiyi+Song%2C+Ann+Bies%2C+and+Stephanie+Strassel%2E+2016%2E+Overview+of+Linguistic+Resources+for+the+TAC+KBP+2016+Evaluations+%3A+Methodologies+and+Results%2E+Pro)
10. Dean Pomerleau and Delip Rao. 2017. Fake news challenge. [http://fakenewschallenge.org/](http://fakenewschallenge.org/). [↗](http://fakenewschallenge.org/)
11. P. Rajpurkar, J. Zhang, K. Lopyrev, and P. Liang. 2016. SQuAD: 100,000+ questions for machine comprehension of text. In Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=P%2E+Rajpurkar%2C+J%2E+Zhang%2C+K%2E+Lopyrev%2C+and+P%2E+Liang%2E+2016%2E+SQuAD%3A+100%2C000%2B+questions+for+machine+comprehension+of+text%2E+In+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/)
12. William Ferreira and Andreas Vlachos. 2016. Emergent: a novel data-set for stance classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. San Diego, California, pages 1163–1168. [find](https://scholar.google.com/scholar?q=William+Ferreira+and+Andreas+Vlachos%2E+2016%2E+Emergent%3A+a+novel+data%2Dset+for+stance+classification%2E+In+Proceedings+of+the+2016+Conference+of+the+North+American+Chapter+of+the+Association+for+Computation) [our card](https://md.research.elamur.ai/free/en/papers/ferreira-2016-emergent-novel-data-set-stance/)
13. Benjamin Riedel, Isabelle Augenstein, George Spithourakis, and Sebastian Riedel. 2017. A simple but tough-to-beat baseline for the Fake News Challenge stance detection task. CoRR abs/1707.03264. http://arxiv.org/abs/1707.03264. [↗](http://arxiv.org/abs/1707.03264)
14. Joseph L Fleiss. 1971. Measuring nominal scale agreement among many raters. Psychological bulletin 76(5):378. [find](https://scholar.google.com/scholar?q=Joseph+L+Fleiss%2E+1971%2E+Measuring+nominal+scale+agreement+among+many+raters%2E+Psychological+bulletin+76%285%29%3A378%2E)
15. Tim Rocktäschel and Sebastian Riedel. 2017. End-to-end differentiable proving. In Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, California, United States. volume abs/1705.11040. [http://arxiv.org…](http://arxiv.org…) [↗](http://arxiv.org…)
16. Matt Gardner, Joel Grus, Mark Neumann, Oyvind Tafjord, Pradeep Dasigi, Nelson Liu, Matthew Peters, Michael Schmitz, and Luke Zettlemoyer. 2017. AllenNLP: A Deep Semantic Natural Language Processing Platform . [find](https://scholar.google.com/scholar?q=Matt+Gardner%2C+Joel+Grus%2C+Mark+Neumann%2C+Oyvind+Tafjord%2C+Pradeep+Dasigi%2C+Nelson+Liu%2C+Matthew+Peters%2C+Michael+Schmitz%2C+and+Luke+Zettlemoyer%2E+2017%2E+AllenNLP%3A+A+Deep+Semantic+Natural+Language+Processing+Pl) [our card](https://md.research.elamur.ai/free/en/papers/gardner-2018-allennlp-deep-semantic-natural-language/)
17. Álvaro Rodrigo, Anselmo Peñas, and Felisa Verdejo. 2009. Overview of the answer validation exercise 2008. In Carol Peters, Thomas Deselaers, Nicola Ferro, Julio Gonzalo, Gareth J. F. Jones, Mikko Kurimo, Thomas Mandl, Anselmo Peñas, and Vivien Petras, editors, Evaluating Systems for Multilingual… [find](https://scholar.google.com/scholar?q=A%CC%81lvaro+Rodrigo%2C+Anselmo+Pen%CC%83as%2C+and+Felisa+Verdejo%2E+2009%2E+Overview+of+the+answer+validation+exercise+2008%2E+In+Carol+Peters%2C+Thomas+Deselaers%2C+Nicola+Ferro%2C+Julio+Gonzalo%2C+Gareth+J%2E+F%2E+Jones%2C+Mikko+Ku)
18. Michael Heilman and Noah A. Smith. 2010. Good Question! statistical ranking for question generation. In Proceedings of the 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics. pages 609–617. [find](https://scholar.google.com/scholar?q=Michael+Heilman+and+Noah+A%2E+Smith%2E+2010%2E+Good+Question%21+statistical+ranking+for+question+generation%2E+In+Proceedings+of+the+2010+Annual+Conference+of+the+North+American+Chapter+of+the+Association+for+C)
19. Mio Kobayashi, Ai Ishii, Chikara Hoshino, Hiroshi Miyashita, and Takuya Matsuzaki. 2017. Automated historical fact-checking by passage retrieval, word statistics, and virtual question-answering. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Lon… [find](https://scholar.google.com/scholar?q=Mio+Kobayashi%2C+Ai+Ishii%2C+Chikara+Hoshino%2C+Hiroshi+Miyashita%2C+and+Takuya+Matsuzaki%2E+2017%2E+Automated+historical+fact%2Dchecking+by+passage+retrieval%2C+word+statistics%2C+and+virtual+question%2Danswering%2E+In+Pr)
20. Min Joon Seo, Aniruddha Kembhavi, Ali Farhadi, and Hannaneh Hajishirzi. 2016. Bidirectional attention flow for machine comprehension. CoRR abs/1611.01603. http://arxiv.org/abs/1611.01603. [↗](http://arxiv.org/abs/1611.01603) [our card](https://md.research.elamur.ai/free/en/papers/seo-2016-bidirectional-attention-flow-machine-comprehension/)
21. Craig Silverman. 2015. Lies, Damn Lies and Viral Content. [http://towcenter.org/research/](http://towcenter.org/research/) lies-damn-lies-and-viral-content/. [↗](http://towcenter.org/research/)
22. Niket Tandon, Gerard de Melo, and Gerhard Weikum. 2011. Deriving a Web-scale common sense fact database. In Proceedings of the 25th AAAI Conference on Artificial Intelligence (AAAI 2011). AAAI Press, Palo Alto, CA, USA, pages 152–157. [find](https://scholar.google.com/scholar?q=Niket+Tandon%2C+Gerard+de+Melo%2C+and+Gerhard+Weikum%2E+2011%2E+Deriving+a+Web%2Dscale+common+sense+fact+database%2E+In+Proceedings+of+the+25th+AAAI+Conference+on+Artificial+Intelligence+%28AAAI+2011%29%2E+AAAI+Press%2C+)
23. Andreas Vlachos and Sebastian Riedel. 2014. Fact checking: Task definition and dataset construction. In Proceedings of the ACL 2014 Workshop on Language Technologies and Computational Social Science. ACL. [http://www.aclweb.org/anthology/W14-2508](http://www.aclweb.org/anthology/W14-2508). [↗](http://www.aclweb.org/anthology/W14-2508) [our card](https://md.research.elamur.ai/free/en/papers/vlachos-2014-fact-checking-task-definition-dataset/)
24. William Yang Wang. 2017. “Liar, Liar Pants on Fire”: A new benchmark dataset for fake news detection. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). [http://aclweb.org/anthology/P17-2067](http://aclweb.org/anthology/P17-2067). [↗](http://aclweb.org/anthology/P17-2067) [our card](https://md.research.elamur.ai/free/en/papers/wang-2017-liar-liar-pants-fire-new/)