---
title: "RiQuA — paper by Papay and Padó (2020)"
description: "Quoted speech carries much of a novel's content, but understanding it requires more than the quoted words. 20 references, cited by 16 papers here."
authors: ["Sean Papay", "Sebastian Padó"]
year: 2020
venue: "Proceedings of the Twelfth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2020.lrec-1.104/"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/papay-2020-riqua-corpus-rich-quotation-annotation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/papay-2020-riqua-corpus-rich-quotation-annotation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/6/) / RiQuA: A Corpus of Rich Quotation Annotation for English Literary Text

# RiQuA: A Corpus of Rich Quotation Annotation for English Literary Text

**Authors:** Sean Papay and [Sebastian Padó](https://md.research.elamur.ai/free/en/authors/sebastian-pado/) · **Year:** 2020 · **Venue:** [Proceedings of the Twelfth Language Resources and Evaluation Conference](https://md.research.elamur.ai/free/en/venues/lrec/) ([LREC 2020](https://md.research.elamur.ai/free/en/venues/lrec/2020/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.lrec-1.104/) · [PDF](https://aclanthology.org/2020.lrec-1.104.pdf) · [OpenAlex](https://openalex.org/W3028857185)

**Citations:** 11 citations (OpenAlex, 2026-10-14) · 16 in our library, of the 141,840 papers whose bibliographies we read

## Summary
Quoted speech carries much of a novel's content, but understanding it requires more than the quoted words. A reader also needs to know who spoke, to whom, and through which speech word. Existing quotation corpora rarely capture all of this. They typically skip indirect speech, omit addressees, or rely on automatically detected spans. RiQuA addresses this gap for English literary text. The authors took the same 11 nineteenth-century works used in the Columbia attribution corpus and fully double-annotated them by hand. For every direct and indirect quotation they marked the span, the cue, the speaker and the addressee wherever these exist. Their definition of a speech event draws on FrameNet's communication-related frames and is intentionally broad. It includes nouns, idioms and implied communication, and cues are optional. The final corpus holds roughly 6,000 quotations. In their error analysis, the authors find that many disagreements, especially for addressees, involve annotators choosing different mentions of the same character. These are not true errors of identification. RiQuA is freely available and supports work on dialogue structure, social network extraction and discourse modeling.

## Key points
- The corpus covers 11 works of 19th-century fiction, including Austen's *Emma*, Dickens' *A Christmas Carol*, *Madame Bovary*, *Tom Sawyer*, four Sherlock Holmes stories and three Chekhov pieces. One work is translated from French and three from Russian.
- It contains about 6,000 quotations, each with span, cue, speaker and addressee where present. All annotation is manual and was done by two annotators.
- Among the prior corpora compared (CQSAC 3,176 quotations, PARC 3 19,712, STOP 13,237, ACDS 1,245, RWG 9,451), only ACDS marks addressees. ACDS covers just the Gospels, handles only direct speech and is not public.
- Speech events count whether or not they are factual, at any modality and at any narrative depth, including quotations nested inside other quotations.
- Disagreements were split into plain errors and "CoRef" errors, meaning the right referent but a different mention. CoRef cases make up most addressee disagreements. Speakers actually had more plain errors than addressees (46 vs. 33).

## Interesting ideas
- The authors split annotator disagreement into "wrong entity" and "same entity, different mention." This shows that the addressee task only looked harder because of where the mentions fall, not because annotators misidentified characters.
- Speech events are defined broadly through FrameNet's Communication and Statement frames, then narrowed with structural filters such as "has a cue" or "single verb or noun" to recover the canonical cases.
- Cues are optional, so quotations in uncued dialogue still get a speaker and addressee from the surrounding context.
- The authors show that guessing conversation partners from a 300-word proximity window, as earlier work did, adds noise. Explicit addressee links avoid this.
- Speech is annotated at every narrative level, including quotations inside quotations, whether or not the event is factual.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): When scoring speaker or addressee output against gold or silver data, give credit for any mention in the same coreference chain, so that mention choice does not hide real attribution errors.
- **I2** (frames): The prag crate could detect communication frames broadly, including noun cues, idioms and implied speech, and store structural flags so that later stages can filter down to plain cases.
- **I3** (world): The world builder should track turn-taking between characters, so that uncued lines in a fable can be attributed from the dialogue state rather than left without an owner.
- **I4** (answer): QA over tales ("whom did the fox tell…?") should answer from explicit addressee links in the meaning graph, not from which characters appear near each other.
- **I5** (frames): Frames could carry a nesting or embedding depth and a factuality marker, so that reported or hypothetical speech does not enter the text's world as asserted fact.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Stuttgart](https://md.research.elamur.ai/free/en/universities/university-of-stuttgart/) — University, Stuttgart, Germany · 75 works · 46,113 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [📚 Literature and Literary Theory](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/literature-and-literary-theory/) · 340 works

🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 1,373 works

## Cite

Cite as: Sean Papay and Sebastian Padó. 2020. RiQuA: A Corpus of Rich Quotation Annotation for English Literary Text. Proceedings of the Twelfth Language Resources and Evaluation Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{papay2020riqua,
  title = {{RiQuA: A Corpus of Rich Quotation Annotation for English Literary Text}},
  author = {Sean Papay and Sebastian Padó},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  url = {https://aclanthology.org/2020.lrec-1.104/}
}
```

- [← previous New Protocols and Negative Results for Textual Entailment Data Collection](https://md.research.elamur.ai/free/en/papers/bowman-2020-new-protocols-negative-results-textual/)
- [next → Using Type Information to Improve Entity Coreference Resolution](https://md.research.elamur.ai/free/en/papers/khosla-2020-type-information-improve-entity-coreference/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/6/)

## References

20 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Almeida, M. S. C., Almeida, M. B., and Martins, A. F. T. (2014). A joint model for quotation attribution and coreference resolution. In Proceedings of EACL, pages 39–48, Gothenburg, Sweden. [find](https://scholar.google.com/scholar?q=Almeida%2C+M%2E+S%2E+C%2E%2C+Almeida%2C+M%2E+B%2E%2C+and+Martins%2C+A%2E+F%2E+T%2E+%282014%29%2E+A+joint+model+for+quotation+attribution+and+coreference+resolution%2E+In+Proceedings+of+EACL%2C+pages+39%E2%80%9348%2C+Gothenburg%2C+Sweden%2E)
2. Artstein, R. and Poesio, M. (2008). Inter-coder agreement for computational linguistics. Computational Linguistics, 34(4):555–596. [↗](https://doi.org/10.1162/coli.07-034-r2) [our card](https://md.research.elamur.ai/free/en/papers/artstein-2008-survey-article-inter-coder-agreement/)
3. Brunner, A. (2013). Automatic recognition of speech, thought, and writing representation in German narrative texts. Literary and Linguistic Computing, 28(4):563–575. [↗](https://doi.org/10.1093/llc/fqt024)
4. Elson, D. K. and McKeown, K. R. (2010). Automatic attribution of quoted speech in literary narrative. In Proceedings of AAAI, pages 1013–1019, Atlanta, GA. [↗](https://doi.org/10.7916/d8ks70v3)
5. Elson, D. K., Dames, N., and McKeown, K. R. (2010). Extracting social networks from literary fiction. In Proceedings of ACL, pages 138–147, Uppsala, Sweden. [↗](https://doi.org/10.7916/d8m90j1s)
6. Faruqui, M. and Padó, S. (2012). Towards a model of formal and informal address in English. In Proceedings of EACL, pages 623–633, Avignon, France. [find](https://scholar.google.com/scholar?q=Faruqui%2C+M%2E+and+Pad%C3%B3%2C+S%2E+%282012%29%2E+Towards+a+model+of+formal+and+informal+address+in+English%2E+In+Proceedings+of+EACL%2C+pages+623%E2%80%93633%2C+Avignon%2C+France%2E)
7. Fillmore, C. J., Johnson, C. R., and Petruck, M. R. (2003). Background to FrameNet. International Journal of Lexicography, 16(3):235–250. [↗](https://doi.org/10.1093/ijl/16.3.235)
8. Fillmore, C. J. (1986). Pragmatically controlled zero anaphora. In V Nikiforidou, et al., editors, Proceedings of the Berkeley Linguistics Society, volume 12, pages 95–107. [↗](https://doi.org/10.3765/bls.v12i0.1866)
9. Iosif, E. and Mishra, T. (2014). From speaker identification to affective analysis: a multi-step system for analyzing children’s stories. In Proceedings of the 3rd Workshop on Computational Linguistics for Literature, pages 40–49, Gothenburg, Sweden. [find](https://scholar.google.com/scholar?q=Iosif%2C+E%2E+and+Mishra%2C+T%2E+%282014%29%2E+From+speaker+identification+to+affective+analysis%3A+a+multi%2Dstep+system+for+analyzing+children%E2%80%99s+stories%2E+In+Proceedings+of+the+3rd+Workshop+on+Computational+Linguistic)
10. Kim, E. and Klinger, R. (2018). Who feels what and why? annotation of a literature corpus with semantic roles of emotions. In Proceedings of COLING, pages 1345–1359, Santa Fe, NM. [find](https://scholar.google.com/scholar?q=Kim%2C+E%2E+and+Klinger%2C+R%2E+%282018%29%2E+Who+feels+what+and+why%3F+annotation+of+a+literature+corpus+with+semantic+roles+of+emotions%2E+In+Proceedings+of+COLING%2C+pages+1345%E2%80%931359%2C+Santa+Fe%2C+NM%2E)
11. Lee, J. and Yeung, C. Y. (2016). An annotated corpus of direct speech. In Proceedings of the Tenth International Conference on Language Resources and Evaluation, pages 1059–1063, Portorož, Slovenia. [find](https://scholar.google.com/scholar?q=Lee%2C+J%2E+and+Yeung%2C+C%2E+Y%2E+%282016%29%2E+An+annotated+corpus+of+direct+speech%2E+In+Proceedings+of+the+Tenth+International+Conference+on+Language+Resources+and+Evaluation%2C+pages+1059%E2%80%931063%2C+Portoro%C5%BE%2C+Slovenia%2E)
12. Liu, B. (2012). Sentiment Analysis and Opinion Mining. Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers. [↗](https://doi.org/10.2200/s00416ed1v01y201204hlt016)
13. Nalisnick, E. T. and Baird, H. S. (2013). Character-to-character sentiment analysis in Shakespeare’s plays. In Proceedings of ACL, pages 479–483, Sofia, Bulgaria. [find](https://scholar.google.com/scholar?q=Nalisnick%2C+E%2E+T%2E+and+Baird%2C+H%2E+S%2E+%282013%29%2E+Character%2Dto%2Dcharacter+sentiment+analysis+in+Shakespeare%E2%80%99s+plays%2E+In+Proceedings+of+ACL%2C+pages+479%E2%80%93483%2C+Sofia%2C+Bulgaria%2E)
14. Page, N. (1988). Speech in the English novel. Springer. [↗](https://doi.org/10.1007/978-1-349-19047-8)
15. Papay, S. and Padó, S. (2019). Quotation detection and classification with a corpus-agnostic model. In Proceedings of RANLP, Varna, Bulgaria. [find](https://scholar.google.com/scholar?q=Papay%2C+S%2E+and+Pad%C3%B3%2C+S%2E+%282019%29%2E+Quotation+detection+and+classification+with+a+corpus%2Dagnostic+model%2E+In+Proceedings+of+RANLP%2C+Varna%2C+Bulgaria%2E)
16. Pareti, S. (2016). Parc 3.0: A corpus of attribution relations. In Proceedings of LREC, pages 3914–3920, Portorož, Slovenia. [find](https://scholar.google.com/scholar?q=Pareti%2C+S%2E+%282016%29%2E+Parc+3%2E0%3A+A+corpus+of+attribution+relations%2E+In+Proceedings+of+LREC%2C+pages+3914%E2%80%933920%2C+Portoro%C5%BE%2C+Slovenia%2E)
17. Redeker, G. and Egg, M. (2006). Says who? On the treatment of speech attributions in discourse structure. In Proceedings of the Workshop on Constraints in Discourse, pages 141–147, Maynooth, Ireland. [find](https://scholar.google.com/scholar?q=Redeker%2C+G%2E+and+Egg%2C+M%2E+%282006%29%2E+Says+who%3F+On+the+treatment+of+speech+attributions+in+discourse+structure%2E+In+Proceedings+of+the+Workshop+on+Constraints+in+Discourse%2C+pages+141%E2%80%93147%2C+Maynooth%2C+Ireland%2E)
18. Roesiger, I., Schulz, S., and Reiter, N. (2018). Towards coreference for literary text: Analyzing domain-specific phenomena. In Proceedings of the Workshop on Computational Linguistics for Cultural Heritage, Social Sciences, Humanities and Literature, pages 129–138, Santa Fe, NM. [our card](https://md.research.elamur.ai/free/en/papers/roesiger-2018-coreference-literary-text-analyzing-domain/)
19. Semino, E. and Short, M. (2004). Corpus Stylistics: Speech, Writing And Thought Presentation In A Corpus Of English Writing. Routledge Advances In Corpus Linguistics. Routledge, London. [find](https://scholar.google.com/scholar?q=Semino%2C+E%2E+and+Short%2C+M%2E+%282004%29%2E+Corpus+Stylistics%3A+Speech%2C+Writing+And+Thought+Presentation+In+A+Corpus+Of+English+Writing%2E+Routledge+Advances+In+Corpus+Linguistics%2E+Routledge%2C+London%2E)
20. Stenetorp, P., Pyysalo, S., Topić, G., Ohta, T., Ananiadou, S., and Tsujii, J. (2012). brat: a web-based tool for NLP-assisted text annotation. In Proceedings of EACL System Demonstrations, pages 102–107, Avignon, France. [our card](https://md.research.elamur.ai/free/en/papers/stenetorp-2012-brat-web-based-tool-nlp/)

*Updated 24 October 2026*