---
title: "Finding Contradictions in Text — Marneffe 2008"
description: "The paper addresses automatic detection of conflicting statements across texts."
authors: ["Marie-Catherine de Marneffe", "Anna N. Rafferty", "Christopher D. Manning"]
year: 2008
venue: "Proceedings of ACL-08: HLT"
source: "https://aclanthology.org/P08-1118/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/marneffe-2008-finding-contradictions-text/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/marneffe-2008-finding-contradictions-text/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2008](https://md.research.elamur.ai/free/en/papers/2008/) / Finding Contradictions in Text

# Finding Contradictions in Text

**Authors:** [Marie-Catherine de Marneffe](https://md.research.elamur.ai/free/en/authors/marie-catherine-de-marneffe/), Anna N. Rafferty, [Christopher D. Manning](https://md.research.elamur.ai/free/en/authors/christopher-d-manning/) · **Year:** 2008 · **Venue:** Proceedings of ACL-08: HLT · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P08-1118/) · [PDF](https://aclanthology.org/P08-1118.pdf) · [OpenAlex](https://openalex.org/W2130359236)

**Citations:** 222 citations (openalex-snapshot · 2026-10-14) · [79](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
The paper addresses automatic detection of conflicting statements across texts. The authors argue this task differs from textual entailment and is harder. They replace the strict logical notion of contradiction with a practical one: two sentences contradict when they are highly unlikely to be true together, provided they describe the same event. Under this definition they annotated the RTE1–RTE3 datasets for contradiction. NIST later used their guidelines to label RTE3 test for the RTE3 Pilot three-way task. Analysing these data and a corpus of naturally occurring contradictions, they propose a typology with two groups:
- **"Easy" cases:** antonymy, negation, and number/date mismatch.
- **Harder cases:** factive/modal, structural, lexical, and world-knowledge conflicts.

Their detector includes an event coreference component based on topicality, and they give the first per-type performance breakdown. The system handles negation and antonymy reasonably well but rarely catches lexical and world-knowledge contradictions. It also degrades on test data more than typical entailment systems do, yet it still clearly beats the average RTE3 Pilot entry. The study shows which contradiction types are currently solvable and where effort is needed.

## Key points
- Contradictions are rare even in inference-oriented data: for example, 149 of 800 pairs in RTE1 test and 72 of 800 in RTE3 test.
- The type distribution differs sharply between the RTE3 dev set and the "real" corpus. Numeric mismatches are 8.8% in RTE3 dev versus 29.0% in the real corpus, and world-knowledge cases are 27.5% versus 13.0%. The easy category covers about a third of RTE contradictions but about half of the real ones.
- Recall by type on RTE3 test:
  - antonym 42.9% (3/7)
  - negation 60.0% (3/5)
  - lexical 0% (0/12)
  - world knowledge 8.3% (1/12)
- Further results:
  - 63% accuracy on the negation test set
  - 78.9% precision for single-word antonymy
- Earlier work by Harabagiu et al. (2006) scored 75.63% on the negation portion of the LCC data and 62.55% on its paraphrase portion, though it dealt only with those two construction types.
- Structural features were the main source of false alarms. They caused 36 false positives on RTE3 test, more than 75% of the precision errors. The heuristic "same subject, different object" signalled a real contradiction only 55% of the time on the development sets.

## Interesting ideas
- **Define the target by human plausibility, not logic.** Treating "very unlikely to co-occur" as contradiction makes annotation align with intuition and with practical use cases such as flagging inconsistent reports. Any fact-checking or data-consistency project could adopt a similar operational definition.
- **Make same-event detection a gate.** Checking whether two texts describe the same event (via event coreference) before judging conflict prevents false alarms between unrelated statements. This filter design transfers to deduplication or cross-source verification pipelines.
- **Break evaluation down by phenomenon.** Reporting recall per contradiction type exposes where a system actually works, rather than hiding failures behind one aggregate score. The same practice helps diagnose any classifier evaluated on mixed phenomena.
- **Compare benchmark and real-world distributions.** The relative frequency of contradiction types in RTE differed strongly from naturally occurring data. Checking this before choosing what to optimize keeps development aligned with real use.
- **Mismatch is not conflict.** The Nike example compares a profit figure with an orders figure, so differing values are not a real conflict. It warns that surface disagreement needs contextual grounding, a useful caution for numeric consistency checks in trade or statistical data.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

79 papers of our library cite it (of the 140884 whose bibliographies we read); 12 of them have a page here, 67 not yet.

- [Detecting Subevents using Discourse and Narrative Features](https://md.research.elamur.ai/free/en/papers/aldawsari-2019-detecting-subevents-discourse-narrative-features/)
- [A Survey of Paraphrasing and Textual Entailment Methods](https://md.research.elamur.ai/free/en/papers/androutsopoulos-2009-survey-paraphrasing-textual-entailment-methods/)
- [Joint Event Trigger Identification and Event Coreference Resolution with Structured Perceptron](https://md.research.elamur.ai/free/en/papers/araki-2015-joint-event-trigger-identification-event/)
- [Unsupervised Event Coreference Resolution with Rich Linguistic Features](https://md.research.elamur.ai/free/en/papers/bejan-2010-unsupervised-event-coreference-resolution-rich/)
- [Improving Natural Language Inference in Arabic using Transformer Models and Linguistically Informed Pre-Training](https://md.research.elamur.ai/free/en/papers/deen-2023-improving-natural-language-inference-arabic/)
- [Interpreting Verbal Irony: Linguistic Strategies and the Connection to the Type of Semantic Incongruity](https://md.research.elamur.ai/free/en/papers/ghosh-2020-interpreting-verbal-irony-linguistic-strategies/)
- [ArNLI: Arabic Natural Language Inference for Entailment and Contradiction Detection](https://md.research.elamur.ai/free/en/papers/jallad-2022-arnli-arabic-natural-language-inference/)
- [E-CONAN (Entailment, CONtradition And Neutral) Benchmarks: Arabic Textual Entailment and Natural Inference Datasets](https://md.research.elamur.ai/free/en/papers/jallad-2026-e-conan-entailment-contradition-neutral-benchmarks/)
- [Investigating Reasons for Disagreement in Natural Language Inference](https://md.research.elamur.ai/free/en/papers/jiang-2022-investigating-reasons-disagreement-natural-language/)
- [EQUATE: A Benchmark Evaluation Framework for Quantitative Reasoning in Natural Language Inference](https://md.research.elamur.ai/free/en/papers/ravichander-2019-equate-benchmark-evaluation-framework-quantitative/)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/)
- [Event Coreference Resolution with their Paraphrases and Argument-aware Embeddings](https://md.research.elamur.ai/free/en/papers/zeng-2020-event-coreference-resolution-their-paraphrases/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{marneffe2008finding,
  title = {{Finding Contradictions in Text}},
  author = {Marneffe, Marie-Catherine de and Rafferty, Anna N. and Manning, Christopher D.},
  booktitle = {Proceedings of ACL-08: HLT},
  year = {2008},
  url = {https://aclanthology.org/P08-1118/}
}
```

- [← previous Feature Forest Models for Probabilistic HPSG Parsing](https://md.research.elamur.ai/free/en/papers/cl-2008-miyao-feature-forest-models-probabilistic-hpsg/)
- [next → Forest-Based Translation](https://md.research.elamur.ai/free/en/papers/mi-2008-forest-based-translation/)
- [↑ back to papers of 2008](https://md.research.elamur.ai/free/en/papers/2008/)

## References

22 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Roy Bar-Haim, Ido Dagan, Bill Dolan, Lisa Ferro, Danilo Giampiccolo, Bernardo Magnini, and Idan Szpektor. 2006. The second PASCAL recognising textual entailment challenge. In Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment, Venice, Italy. [find](https://scholar.google.com/scholar?q=Roy+Bar%2DHaim%2C+Ido+Dagan%2C+Bill+Dolan%2C+Lisa+Ferro%2C+Danilo+Giampiccolo%2C+Bernardo+Magnini%2C+and+Idan+Szpektor%2E+2006%2E+The+second+PASCAL+recognising+textual+entailment+challenge%2E+In+Proceedings+of+the+Second)
2. Nathanael Chambers, Daniel Cer, Trond Grenager, David Hall, Chloe Kiddon, Bill MacCartney, Marie-Catherine de Marneffe, Daniel Ramage, Eric Yeh, and Christopher D. Manning. 2007. Learning alignments and leveraging natural logic. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Par… [find](https://scholar.google.com/scholar?q=Nathanael+Chambers%2C+Daniel+Cer%2C+Trond+Grenager%2C+David+Hall%2C+Chloe+Kiddon%2C+Bill+MacCartney%2C+Marie%2DCatherine+de+Marneffe%2C+Daniel+Ramage%2C+Eric+Yeh%2C+and+Christopher+D%2E+Manning%2E+2007%2E+Learning+alignments+a)
3. Timothy Chklovski and Patrick Pantel. 2004. Verbocean: Mining the web for fine-grained semantic verb relations. In Proceedings of EMNLP-04. [find](https://scholar.google.com/scholar?q=Timothy+Chklovski+and+Patrick+Pantel%2E+2004%2E+Verbocean%3A+Mining+the+web+for+fine%2Dgrained+semantic+verb+relations%2E+In+Proceedings+of+EMNLP%2D04%2E)
4. Cleo Condoravdi, Dick Crouch, Valeria de Pavia, Reinhard Stolle, and Daniel G. Bobrow. 2003. Entailment, intensionality and text understanding. Workshop on Text Meaning (2003 May 31). [find](https://scholar.google.com/scholar?q=Cleo+Condoravdi%2C+Dick+Crouch%2C+Valeria+de+Pavia%2C+Reinhard+Stolle%2C+and+Daniel+G%2E+Bobrow%2E+2003%2E+Entailment%2C+intensionality+and+text+understanding%2E+Workshop+on+Text+Meaning+%282003+May+31%29%2E)
5. Koby Crammer and Yoram Singer. 2001. Ultraconservative online algorithms for multiclass problems. In Proceedings of COLT-2001. [find](https://scholar.google.com/scholar?q=Koby+Crammer+and+Yoram+Singer%2E+2001%2E+Ultraconservative+online+algorithms+for+multiclass+problems%2E+In+Proceedings+of+COLT%2D2001%2E)
6. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006. The PASCAL recognising textual entailment challenge. In Quinonero-Candela et al., editor, MLCW 2005, LNAI Volume 3944, pages 177–190. Springer-Verlag. [find](https://scholar.google.com/scholar?q=Ido+Dagan%2C+Oren+Glickman%2C+and+Bernardo+Magnini%2E+2006%2E+The+PASCAL+recognising+textual+entailment+challenge%2E+In+Quinonero%2DCandela+et+al%2E%2C+editor%2C+MLCW+2005%2C+LNAI+Volume+3944%2C+pages+177%E2%80%93190%2E+Springer%2DVer)
7. Marie-Catherine de Marneffe, Bill MacCartney, and Christopher D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In Proceedings of the 5th International Conference on Language Resources and Evaluation (LREC-06). [find](https://scholar.google.com/scholar?q=Marie%2DCatherine+de+Marneffe%2C+Bill+MacCartney%2C+and+Christopher+D%2E+Manning%2E+2006%2E+Generating+typed+dependency+parses+from+phrase+structure+parses%2E+In+Proceedings+of+the+5th+International+Conference+on+L) [our card](https://md.research.elamur.ai/free/en/papers/map-2006-marneffe-generating-typed-dependency-parses-phrase/)
8. Christiane Fellbaum. 1998. WordNet: an electronic lexical database. MIT Press. [find](https://scholar.google.com/scholar?q=Christiane+Fellbaum%2E+1998%2E+WordNet%3A+an+electronic+lexical+database%2E+MIT+Press%2E)
9. Jan Firbas. 1971. On the concept of communicative dynamism in the theory of functional sentence perspective. Brno Studies in English, 7:23–47. [find](https://scholar.google.com/scholar?q=Jan+Firbas%2E+1971%2E+On+the+concept+of+communicative+dynamism+in+the+theory+of+functional+sentence+perspective%2E+Brno+Studies+in+English%2C+7%3A23%E2%80%9347%2E)
10. Danilo Giampiccolo, Ido Dagan, Bernardo Magnini, and Bill Dolan. 2007. The third PASCAL recognizing textual entailment challenge. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing. [find](https://scholar.google.com/scholar?q=Danilo+Giampiccolo%2C+Ido+Dagan%2C+Bernardo+Magnini%2C+and+Bill+Dolan%2E+2007%2E+The+third+PASCAL+recognizing+textual+entailment+challenge%2E+In+Proceedings+of+the+ACL%2DPASCAL+Workshop+on+Textual+Entailment+and+Pa)
11. Sanda Harabagiu, Andrew Hickl, and Finley Lacatusu. 2006. Negation, contrast, and contradiction in text processing. In Proceedings of the Twenty-First National Conference on Artificial Intelligence (AAAI-06). [find](https://scholar.google.com/scholar?q=Sanda+Harabagiu%2C+Andrew+Hickl%2C+and+Finley+Lacatusu%2E+2006%2E+Negation%2C+contrast%2C+and+contradiction+in+text+processing%2E+In+Proceedings+of+the+Twenty%2DFirst+National+Conference+on+Artificial+Intelligence+%28A)
12. Andrew Hickl, John Williams, Jeremy Bensley, Kirk Roberts, Bryan Rink, and Ying Shi. 2006. Recognizing textual entailment with LCC’s GROUNDHOG system. In Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment. [find](https://scholar.google.com/scholar?q=Andrew+Hickl%2C+John+Williams%2C+Jeremy+Bensley%2C+Kirk+Roberts%2C+Bryan+Rink%2C+and+Ying+Shi%2E+2006%2E+Recognizing+textual+entailment+with+LCC%E2%80%99s+GROUNDHOG+system%2E+In+Proceedings+of+the+Second+PASCAL+Challenges+Wo)
13. Kevin Humphreys, Robert Gaizauskas, and Saliha Azzam. 1997. Event coreference for information extraction. In Proceedings of the Workshop on Operational Factors in Pratical, Robust Anaphora Resolution for Unrestricted Texts, 35th ACL meeting. [find](https://scholar.google.com/scholar?q=Kevin+Humphreys%2C+Robert+Gaizauskas%2C+and+Saliha+Azzam%2E+1997%2E+Event+coreference+for+information+extraction%2E+In+Proceedings+of+the+Workshop+on+Operational+Factors+in+Pratical%2C+Robust+Anaphora+Resolution+) [our card](https://md.research.elamur.ai/free/en/papers/humphreys-1997-event-coreference-information-extraction/)
14. Dan Klein and Christopher D. Manning. 2003. Accurate unlexicalized parsing. In Proceedings of the 41st Annual Meeting of the Association of Computational Linguistics. [find](https://scholar.google.com/scholar?q=Dan+Klein+and+Christopher+D%2E+Manning%2E+2003%2E+Accurate+unlexicalized+parsing%2E+In+Proceedings+of+the+41st+Annual+Meeting+of+the+Association+of+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/en/papers/aclbulk-2003-klein-accurate-unlexicalized-parsing/)
15. Bill MacCartney, Trond Grenager, Marie-Catherine de Marneffe, Daniel Cer, and Christopher D. Manning. 2006. Learning to recognize features of valid textual entailments. In Proceedings of the North American Association of Computational Linguistics (NAACL-06). [find](https://scholar.google.com/scholar?q=Bill+MacCartney%2C+Trond+Grenager%2C+Marie%2DCatherine+de+Marneffe%2C+Daniel+Cer%2C+and+Christopher+D%2E+Manning%2E+2006%2E+Learning+to+recognize+features+of+valid+textual+entailments%2E+In+Proceedings+of+the+North+Ame)
16. Daniel Marcu and Abdessamad Echihabi. 2002. An unsupervised approach to recognizing discourse relations. In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Daniel+Marcu+and+Abdessamad+Echihabi%2E+2002%2E+An+unsupervised+approach+to+recognizing+discourse+relations%2E+In+Proceedings+of+the+40th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/en/papers/marcu-2002-unsupervised-approach-recognizing-discourse-relations/)
17. Rowan Nairn, Cleo Condoravdi, and Lauri Karttunen. 2006. Computing relative polarity for textual inference. In Proceedings of ICoS-5. [find](https://scholar.google.com/scholar?q=Rowan+Nairn%2C+Cleo+Condoravdi%2C+and+Lauri+Karttunen%2E+2006%2E+Computing+relative+polarity+for+textual+inference%2E+In+Proceedings+of+ICoS%2D5%2E) [our card](https://md.research.elamur.ai/free/en/papers/lingbulk-2006-nairn-computing-relative-polarity-textual-inference/)
18. Olivia Sanchez-Graillet and Massimo Poesio. 2007. Discovering contradiction protein-protein interactions in text. In Proceedings of BioNLP 2007: Biological, translational, and clinical language processing. [find](https://scholar.google.com/scholar?q=Olivia+Sanchez%2DGraillet+and+Massimo+Poesio%2E+2007%2E+Discovering+contradiction+protein%2Dprotein+interactions+in+text%2E+In+Proceedings+of+BioNLP+2007%3A+Biological%2C+translational%2C+and+clinical+language+proces)
19. Lucy Vanderwende, Arul Menezes, and Rion Snow. 2006. Microsoft research at rte-2: Syntactic contributions in the entailment task: an implementation. In Proceedings of the Second PASCAL Challenges Workshop on Recognising Textual Entailment. [find](https://scholar.google.com/scholar?q=Lucy+Vanderwende%2C+Arul+Menezes%2C+and+Rion+Snow%2E+2006%2E+Microsoft+research+at+rte%2D2%3A+Syntactic+contributions+in+the+entailment+task%3A+an+implementation%2E+In+Proceedings+of+the+Second+PASCAL+Challenges+Work)
20. Ellen Voorhees. 2008. Contradictions and justifications: Extensions to the textual entailment task. In Proceedings of the 46th Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ellen+Voorhees%2E+2008%2E+Contradictions+and+justifications%3A+Extensions+to+the+textual+entailment+task%2E+In+Proceedings+of+the+46th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2E)
21. Annie Zaenen, Lauri Karttunen, and Richard S. Crouch. 2005. Local textual inference: can it be defined or circumscribed? In ACL 2005 Workshop on Empirical Modeling of Semantic Equivalence and Entailment. [find](https://scholar.google.com/scholar?q=Annie+Zaenen%2C+Lauri+Karttunen%2C+and+Richard+S%2E+Crouch%2E+2005%2E+Local+textual+inference%3A+can+it+be+defined+or+circumscribed%3F+In+ACL+2005+Workshop+on+Empirical+Modeling+of+Semantic+Equivalence+and+Entailme)
22. Fabio Massimo Zanzotto, Marco Pennacchiotti, and Alessandro Moschitti. 2007. Shallow semantics in fast textual entailment rule learners. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing. [find](https://scholar.google.com/scholar?q=Fabio+Massimo+Zanzotto%2C+Marco+Pennacchiotti%2C+and+Alessandro+Moschitti%2E+2007%2E+Shallow+semantics+in+fast+textual+entailment+rule+learners%2E+In+Proceedings+of+the+ACL%2DPASCAL+Workshop+on+Textual+Entailment)