---
title: "Humans Require Context to Infer Ironic Intent (so Computers"
description: "The paper argues that verbal irony often cannot be detected from the words of an utterance alone. 18 references, cited by 61 papers here."
authors: ["Byron C. Wallace", "Do Kook Choe", "Laura Kertz", "Eugene Charniak"]
year: 2014
venue: "Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)"
source: "https://aclanthology.org/P14-2084/"
doi: "10.3115/v1/p14-2084"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wallace-2014-humans-require-context-infer-ironic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wallace-2014-humans-require-context-infer-ironic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2014](https://md.research.elamur.ai/free/en/papers/2014/page/2/) / Humans Require Context to Infer Ironic Intent (so Computers Probably do, too)

# Humans Require Context to Infer Ironic Intent (so Computers Probably do, too)

**Authors:** [Byron C. Wallace](https://md.research.elamur.ai/free/en/authors/byron-c-wallace/), [Do Kook Choe](https://md.research.elamur.ai/free/en/authors/do-kook-choe/), Laura Kertz and [Eugene Charniak](https://md.research.elamur.ai/free/en/authors/eugene-charniak/) · **Year:** 2014 · **Venue:** [Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2014](https://md.research.elamur.ai/free/en/venues/acl/2014/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P14-2084/) · [PDF](https://aclanthology.org/P14-2084.pdf) · [DOI 10.3115/v1/p14-2084](https://doi.org/10.3115/v1/p14-2084) · [OpenAlex](https://openalex.org/W2142112646)

**Citations:** 135 citations (OpenAlex, 2026-10-14) · [61](https://md.research.elamur.ai/free/en/papers/wallace-2014-humans-require-context-infer-ironic/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper argues that verbal irony often cannot be detected from the words of an utterance alone. Detection also depends on context outside the text, such as who is speaking and what the discussion is about. To test this, the authors built the reddit irony corpus (β 1.0): 3,020 comments from six subreddits, split into 10,401 sentences. Three undergraduates labelled each sentence as ironic or not. The custom annotation tool let annotators ask for extra context: the surrounding thread, the linked page, or the author's earlier comments. The tool logged every such request along with a confidence rating. Two logistic regressions showed two things. Annotators asked for context much more often on comments that turned out to contain irony. A bag-of-words classifier also made its errors disproportionately on those same comments. The conclusion is that intrinsic text features put a ceiling on irony detection, and future models should use speaker and conversation context. The corpus was released publicly to support that work.

## Key points
- **Corpus:** 3,020 comments and 10,401 sentences from r/politics, r/conservative, r/progressive, r/atheism, r/Christianity and r/technology. Sentences were segmented naively by punctuation.
- **Agreement:** raw average agreement was 0.844, but average pairwise Cohen's Kappa was only 0.341. Annotators were deliberately given little guidance on what counts as irony.
- **Context requests:** annotators asked for context on 30% of comments on average, ranging from 12% to 56% per person. Their confidence rose after viewing it, and they often changed their label; one annotator flipped from ironic to unironic 86 times.
- **Regression results:** the coefficient linking context requests to irony was 1.508 (95% CI 1.326–1.690, p \< 0.001). The coefficient linking context requests to classifier errors was 0.971 (CI 0.810–1.133, p \< 0.001), after adjusting for the true label.
- **Baseline classifier:** binary unigram and bigram features (top 50,000, stop-words removed) plus punctuation and emoticon flags. It reached recall of about 0.496 and average precision of about 0.315. The tokens it relied on most heavily for prediction included "!", "shocked", "oh", "yeah" and "guys".

## Interesting ideas
- During annotation, record when a human had to look beyond the text, and treat that request as a measurable signal of how much the item depends on context.
- Regress a model's errors on a human-difficulty indicator while controlling for the gold label, to show *why* a model fails rather than only *how often*.
- Model the speaker's history as context: an utterance that contradicts the author's known stance is likely ironic, as in the Ted Cruz example.
- Ask annotators for both a forced judgement before seeing context and a final judgement after it, then compare the two to measure how much context changes interpretation.
- Some sarcasm markers are shallow but intuitive, such as exclamation marks and words like "yeah" or "oh". They serve as weak cues but cannot decide on their own.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): silver-data tooling could flag items where witness parsers or LLM teachers needed extra discourse to agree, and keep those as a separate "context-dependent" evaluation slice.
- **I2** (learn): error analysis could test whether failures cluster on sentences that humans mark as context-needing, which would separate missing-knowledge errors from parser bugs.
- **I3** (frames): the irony module in prag could compare a speech act's literal evaluation with stances attributed to the same speaker elsewhere in the text, and flag a clash as possible irony with an explicit reason.
- **I4** (answer): fable and tale QA tests could pair a sentence-only question with a full-story question, checking whether readers really use the world of the text.
- **I5** (frames): treat such markers as low-weight evidence feeding an explainable irony hypothesis, never as the sole trigger for reversing a frame's polarity.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Brown University](https://md.research.elamur.ai/free/en/universities/brown-university/) — University, Providence, United States · 58 works · 25,310 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: Byron C. Wallace, Do Kook Choe, Laura Kertz and Eugene Charniak. 2014. Humans Require Context to Infer Ironic Intent (so Computers Probably do, too). Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{wallace2014humans,
  title = {{Humans Require Context to Infer Ironic Intent (so Computers Probably do, too)}},
  author = {Byron C. Wallace, Do Kook Choe, Laura Kertz and Eugene Charniak},
  booktitle = {Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2014},
  doi = {10.3115/v1/p14-2084},
  url = {https://doi.org/10.3115/v1/p14-2084}
}
```

- [← previous Creating a massively parallel Bible corpus](https://md.research.elamur.ai/free/en/papers/mayer-2014-creating-massively-parallel-bible-corpus/)
- [next → A Benchmark Dataset for Automatic Detection of Claims and Evidence in the Context of Controversial Topics](https://md.research.elamur.ai/free/en/papers/aharoni-2014-benchmark-dataset-automatic-detection-claims/)
- [↑ back to papers of 2014](https://md.research.elamur.ai/free/en/papers/2014/page/2/)

## References

18 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. C Burfoot and T Baldwin. 2009. Automatic satire detection: are you having a laugh? In ACL-IJCNLP, pages 161–164. ACL. [find](https://scholar.google.com/scholar?q=C+Burfoot+and+T+Baldwin%2E+2009%2E+Automatic+satire+detection%3A+are+you+having+a+laugh%3F+In+ACL%2DIJCNLP%2C+pages+161%E2%80%93164%2E+ACL%2E)
2. P Carvalho, L Sarmento, MJ Silva, and E de Oliveira. 2009. Clues for detecting irony in user-generated contents: oh...!! it’s so easy;-). In CIKM workshop on Topic-sentiment analysis for mass opinion, pages 53–56. ACM. [find](https://scholar.google.com/scholar?q=P+Carvalho%2C+L+Sarmento%2C+MJ+Silva%2C+and+E+de+Oliveira%2E+2009%2E+Clues+for+detecting+irony+in+user%2Dgenerated+contents%3A+oh%2E%2E%2E%21%21+it%E2%80%99s+so+easy%3B%2D%29%2E+In+CIKM+workshop+on+Topic%2Dsentiment+analysis+for+mass+opinion%2C)
3. HH Clark and RJ Gerrig. 1984. On the pretense theory of irony. Journal of Experimental Psychology, 113:121–126. [↗](https://doi.org/10.1037/0096-3445.113.1.121)
4. J Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological Measurement, 20:37–46. [↗](https://doi.org/10.1177/001316446002000104)
5. D Davidov, O Tsur, and A Rappoport. 2010. [Semi-supervised recognition of sarcastic sentences in twitter and amazon](https://md.research.elamur.ai/free/en/papers/davidov-2010-semi-supervised-recognition-sarcasm-twitter/). pages 107–116.
6. E Filatova. 2012. [Irony and sarcasm: Corpus generation and analysis using crowdsourcing](https://md.research.elamur.ai/free/en/papers/filatova-2012-irony-sarcasm-corpus-generation-analysis/). In LREC, volume 12, pages 392–398.
7. RW Gibbs and HL Colston. 2007. Irony in language and thought: a cognitive science reader. Lawrence Erlbaum. [↗](https://doi.org/10.4324/9781410616685)
8. R González-Ibáñez, S Muresan, and N Wacholder. 2011. [Identifying sarcasm in twitter: a closer look](https://md.research.elamur.ai/free/en/papers/ibanez-2011-identifying-sarcasm-twitter-closer-look/). In ACL, volume 2, pages 581–586. Citeseer.
9. HP Grice. 1975. Logic and conversation. 1975, pages 41–58. [↗](https://doi.org/10.1163/9789004368811_003)
10. A Halevy, P Norvig, and F Pereira. 2009. The unreasonable effectiveness of data. Intelligent Systems, IEEE, 24(2):8–12. [↗](https://doi.org/10.1109/mis.2009.36)
11. S Lukin and M Walker. 2013. [Really? well. apparently bootstrapping improves the performance of sarcasm and nastiness classifiers for online dialogue](https://md.research.elamur.ai/free/en/papers/lukin-2013-really-well-apparently-bootstrapping-improves/). NAACL, pages 30–40.
12. A Reyes, P Rosso, and T Veale. 2012. A multidimensional approach for detecting irony in twitter. LREC, pages 1–30. [↗](https://doi.org/10.1007/s10579-012-9196-x)
13. E Riloff, A Qadir, P Surve, LD Silva, N Gilbert, and R Huang. 2013. [Sarcasm as contrast between a positive sentiment and negative situation](https://md.research.elamur.ai/free/en/papers/riloff-2013-sarcasm-as-contrast-between-positive/). In EMNLP, pages 704–714.
14. D Sperber and D Wilson. 1981. Irony and the use-mention distinction. 1981. [find](https://scholar.google.com/scholar?q=D+Sperber+and+D+Wilson%2E+1981%2E+Irony+and+the+use%2Dmention+distinction%2E+1981%2E)
15. J Tepperman, D Traum, and S Narayanan. 2006. “Yeah Right”: Sarcasm Recognition for Spoken Dialogue Systems. [find](https://scholar.google.com/scholar?q=J+Tepperman%2C+D+Traum%2C+and+S+Narayanan%2E+2006%2E+%E2%80%9CYeah+Right%E2%80%9D%3A+Sarcasm+Recognition+for+Spoken+Dialogue+Systems%2E)
16. O Tsur, D Davidov, and A Rappoport. 2010. ICWSM-a great catchy name: Semi-supervised recognition of sarcastic sentences in online product reviews. In AAAI Conference on Weblogs and Social Media. [find](https://scholar.google.com/scholar?q=O+Tsur%2C+D+Davidov%2C+and+A+Rappoport%2E+2010%2E+ICWSM%2Da+great+catchy+name%3A+Semi%2Dsupervised+recognition+of+sarcastic+sentences+in+online+product+reviews%2E+In+AAAI+Conference+on+Weblogs+and+Social+Media%2E)
17. AJ Viera and JM Garrett. 2005. Understanding interobserver agreement: the kappa statistic. Family Medicine, 37(5):360–363. [find](https://scholar.google.com/scholar?q=AJ+Viera+and+JM+Garrett%2E+2005%2E+Understanding+interobserver+agreement%3A+the+kappa+statistic%2E+Family+Medicine%2C+37%285%29%3A360%E2%80%93363%2E)
18. MA Walker, JEF Tree, P Anand, R Abbott, and J King. 2012. [A corpus for research on deliberation and debate](https://md.research.elamur.ai/free/en/papers/walker-2012-corpus-research-deliberation-debate/). In LREC, pages 812–817.

*Updated 24 October 2026*