---
title: "A simple but tough-to-beat baseline for the Fake News Challenge"
description: "The authors release their code and offer the system as a strong, transparent reference baseline for… 16 references, cited by 46 papers here."
authors: ["Benjamin Riedel", "Isabelle Augenstein", "Georgios P. Spithourakis", "Sebastian Riedel"]
year: 2017
venue: "arXiv:1707.03264"
source: "https://arxiv.org/abs/1707.03264"
doi: "10.48550/arxiv.1707.03264"
arxiv: "1707.03264"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/riedel-2017-simple-but-tough-beat-baseline/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/riedel-2017-simple-but-tough-beat-baseline/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/2/) / A simple but tough-to-beat baseline for the Fake News Challenge stance detection task

# A simple but tough-to-beat baseline for the Fake News Challenge stance detection task

**Authors:** Benjamin Riedel, [Isabelle Augenstein](https://md.research.elamur.ai/free/en/authors/isabelle-augenstein/), [Georgios P. Spithourakis](https://md.research.elamur.ai/free/en/authors/georgios-spithourakis/) and [Sebastian Riedel](https://md.research.elamur.ai/free/en/authors/sebastian-riedel/) · **Year:** 2017 · **Venue:** arXiv:1707.03264 · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1707.03264) · [PDF](https://arxiv.org/pdf/1707.03264) · [DOI 10.48550/arxiv.1707.03264](https://doi.org/10.48550/arxiv.1707.03264) · [arXiv 1707.03264](https://arxiv.org/abs/1707.03264) · [OpenAlex](https://openalex.org/W2735017898)

**Citations:** 196 citations (OpenAlex, 2026-10-14) · [46](https://md.research.elamur.ai/free/en/papers/riedel-2017-simple-but-tough-beat-baseline/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses stance detection for Stage 1 of the Fake News Challenge (FNC-1). Given a headline and an article body, a system must decide whether the body agrees with, disagrees with, discusses, or is unrelated to the headline. Such systems are meant to support human fact checkers. The UCL Machine Reading (UCLMR) team built a deliberately minimal, single end-to-end model. It feeds bag-of-words term-frequency vectors of the headline and body, plus a TF-IDF cosine similarity score, into a multi-layer perceptron with one hidden layer, implemented in TensorFlow and scikit-learn. This model placed third among 50 teams. Its score was within a fraction of a point of the two winners, both of which used complex ensembles. However, the error analysis shows that most of its success comes from separating related from unrelated pairs. It handles the "agree" and especially "disagree" classes poorly, and these are the classes most useful for fact checking. The authors release their code and offer the system as a strong, transparent reference baseline for FNC-1.

## Key points
- **Features:** TF vectors of the headline and body use a 5,000-word vocabulary with stop words removed but negation terms kept. Together with the TF-IDF cosine similarity, they form a 10,001-dimensional input.
- **Model and training:** The MLP has one hidden layer of 100 ReLU units and a softmax output. Training uses cross-entropy loss, L2 regularisation of 0.0001, dropout with keep probability 0.6, the Adam optimiser with a learning rate of 0.01, gradient clipping at 5, batches of 500, and 90 epochs.
- **Leaderboard:** The FNC-1 score was 81.72%, placing third. The winners were "SOLAT in the SWEN" (Talos Intelligence) at 82.02% and "Athene" (TU Darmstadt) at 81.97%. The official baseline scored 75.20%.
- **Per-class accuracy:** Related vs. unrelated accuracy was 96.55%. Class accuracy was 97.90% for "unrelated", 81.38% for "discuss", 44.04% for "agree", and only 6.60% for "disagree".
- **Code and origins:** The code is available at github.com/uclmr/fakenewschallenge. The setup builds on earlier FNC-1 work by Davis and Proctor at Stanford.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 626 works

🧭 Topics (1): [😊 Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) · 1,035 works

## Cite

Cite as: Benjamin Riedel, Isabelle Augenstein, Georgios P. Spithourakis and Sebastian Riedel. 2017. A simple but tough-to-beat baseline for the Fake News Challenge stance detection task. arXiv:1707.03264.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{riedel2017simple,
  title = {{A simple but tough-to-beat baseline for the Fake News Challenge stance detection task}},
  author = {Benjamin Riedel, Isabelle Augenstein, Georgios P. Spithourakis and Sebastian Riedel},
  year = {2017},
  doi = {10.48550/arxiv.1707.03264},
  eprint = {1707.03264},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1707.03264}
}
```

- [← previous Argumentation Mining in User-Generated Web Discourse](https://md.research.elamur.ai/free/en/papers/habernal-2017-argumentation-mining-user-generated-web/)
- [next → FigureQA: An Annotated Figure Dataset for Visual Reasoning](https://md.research.elamur.ai/free/en/papers/kahou-2017-figureqa-annotated-figure-dataset-visual/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/2/)

## References

16 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G. S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Mané, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B.… [find](https://scholar.google.com/scholar?q=M%2E+Abadi%2C+A%2E+Agarwal%2C+P%2E+Barham%2C+E%2E+Brevdo%2C+Z%2E+Chen%2C+C%2E+Citro%2C+G%2E+S%2E+Corrado%2C+A%2E+Davis%2C+J%2E+Dean%2C+M%2E+Devin%2C+S%2E+Ghemawat%2C+I%2E+Goodfellow%2C+A%2E+Harp%2C+G%2E+Irving%2C+M%2E+Isard%2C+Y%2E+Jia%2C+R%2E+Jozefowicz%2C+L%2E+Kaiser%2C+M)
2. A. Agrawal, D. Chin, and K. Chen. Cosine siamese models for stance detection, 2017. URL [http://web.stanford.edu/class/cs224n/reports/2759862.pdf](http://web.stanford.edu/class/cs224n/reports/2759862.pdf). [↗](http://web.stanford.edu/class/cs224n/reports/2759862.pdf)
3. S. Arora, Y. Liang, and T. Ma. A simple but tough-to-beat baseline for sentence embeddings. In International Conference for Learning Representations, 2017. [find](https://scholar.google.com/scholar?q=S%2E+Arora%2C+Y%2E+Liang%2C+and+T%2E+Ma%2E+A+simple+but+tough%2Dto%2Dbeat+baseline+for+sentence+embeddings%2E+In+International+Conference+for+Learning+Representations%2C+2017%2E)
4. S. Baird, D. Sibley, and Y. Pan. Talos targets disinformation with Fake News Challenge victory, 2017. URL [https://blog.talosintelligence.com/2017/06/talos-fake-news-challenge.html](https://blog.talosintelligence.com/2017/06/talos-fake-news-challenge.html). [↗](https://blog.talosintelligence.com/2017/06/talos-fake-news-challenge.html)
5. G. Cybenko. Approximation by superpositions of a sigmoidal function. Mathematics of Control, Signals and Systems, 1989. [find](https://scholar.google.com/scholar?q=G%2E+Cybenko%2E+Approximation+by+superpositions+of+a+sigmoidal+function%2E+Mathematics+of+Control%2C+Signals+and+Systems%2C+1989%2E)
6. R. Davis and C. Proctor. Fake news, real consequences: Recruiting neural networks for the fight against fake news, 2017. URL [https://web.stanford.edu/class/cs224n/reports/2761239.pdf](https://web.stanford.edu/class/cs224n/reports/2761239.pdf). [↗](https://web.stanford.edu/class/cs224n/reports/2761239.pdf)
7. B. Galbraith, H. Iqbal, H. van Veen, D. Rao, J. Thorne, and Y. Pan. A baseline implementation for FNC-1, 2017. URL [https://github.com/FakeNewsChallenge/fnc-1-baseline](https://github.com/FakeNewsChallenge/fnc-1-baseline). [↗](https://github.com/FakeNewsChallenge/fnc-1-baseline)
8. R. H. R. Hahnloser, R. Sarpeshkar, M. A. Mahowald, R. J. Douglas, and H. S. Seung. Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit. Nature, 2000. [find](https://scholar.google.com/scholar?q=R%2E+H%2E+R%2E+Hahnloser%2C+R%2E+Sarpeshkar%2C+M%2E+A%2E+Mahowald%2C+R%2E+J%2E+Douglas%2C+and+H%2E+S%2E+Seung%2E+Digital+selection+and+analogue+amplification+coexist+in+a+cortex%2Dinspired+silicon+circuit%2E+Nature%2C+2000%2E)
9. A. Hanselowski, A. PVS, B. Schiller, and F. Caspelherr. Description of the system developed by team Athene in the FNC-1, 2017. URL [https://github.com/hanselowski/athene_system/blob/master/system_description_athene.pdf](https://github.com/hanselowski/athene_system/blob/master/system_description_athene.pdf). [↗](https://github.com/hanselowski/athene_system/blob/master/system_description_athene.pdf)
10. K. S. Jones. A statistical interpretation of term specificity and its application in retrieval. Journal of Documentation, 1972. [find](https://scholar.google.com/scholar?q=K%2E+S%2E+Jones%2E+A+statistical+interpretation+of+term+specificity+and+its+application+in+retrieval%2E+Journal+of+Documentation%2C+1972%2E)
11. D. P. Kingma and J. Ba. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). In International Conference for Learning Representations, 2015.
12. H. P. Luhn. A statistical approach to mechanized encoding and searching of literary information. IBM Journal of Research and Development, 1957. [find](https://scholar.google.com/scholar?q=H%2E+P%2E+Luhn%2E+A+statistical+approach+to+mechanized+encoding+and+searching+of+literary+information%2E+IBM+Journal+of+Research+and+Development%2C+1957%2E)
13. R. Pascanu, T. Mikolov, and Y. Bengio. On the difficulty of training recurrent neural networks. In International Conference on Machine Learning, 2013. [find](https://scholar.google.com/scholar?q=R%2E+Pascanu%2C+T%2E+Mikolov%2C+and+Y%2E+Bengio%2E+On+the+difficulty+of+training+recurrent+neural+networks%2E+In+International+Conference+on+Machine+Learning%2C+2013%2E)
14. F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, and M. P. E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 2011. [find](https://scholar.google.com/scholar?q=F%2E+Pedregosa%2C+G%2E+Varoquaux%2C+A%2E+Gramfort%2C+V%2E+Michel%2C+B%2E+Thirion%2C+O%2E+Grisel%2C+M%2E+Blondel%2C+P%2E+Prettenhofer%2C+R%2E+Weiss%2C+V%2E+Dubourg%2C+J%2E+Vanderplas%2C+A%2E+Passos%2C+D%2E+Cournapeau%2C+M%2E+Brucher%2C+and+M%2E+P%2E+E%2E+Duchesna)
15. D. Pomerleau and D. Rao. fakenewschallenge.org/. Fake News Challenge, 2017. [find](https://scholar.google.com/scholar?q=D%2E+Pomerleau+and+D%2E+Rao%2E+fakenewschallenge%2Eorg%2F%2E+Fake+News+Challenge%2C+2017%2E)
16. N. Srivastava, G. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov. [Dropout: A simple way to prevent neural networks from overfitting](https://md.research.elamur.ai/free/en/papers/srivastava-2014-dropout-simple-way-prevent/). Journal of Machine Learning Research, 2014. URL http://www.

*Updated 2 November 2026*