---
title: "Rollenwechsel-English: a large-scale semantic role corpus (2018)"
description: "The authors build a very large English resource of automatically labelled semantic frames, aimed at… 19 references, cited by 2 papers here."
authors: ["Asad Sayeed", "Pavel Shkadzko", "Vera Demberg"]
year: 2018
venue: "Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)"
source: "https://aclanthology.org/L18-1488/"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sayeed-2018-rollenwechsel-english-large-scale-semantic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sayeed-2018-rollenwechsel-english-large-scale-semantic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/6/) / Rollenwechsel-English: a large-scale semantic role corpus

# Rollenwechsel-English: a large-scale semantic role corpus

**Authors:** Asad Sayeed, Pavel Shkadzko and [Vera Demberg](https://md.research.elamur.ai/free/en/authors/vera-demberg/) · **Year:** 2018 · **Venue:** [Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)](https://md.research.elamur.ai/free/en/venues/lrec/) ([LREC 2018](https://md.research.elamur.ai/free/en/venues/lrec/2018/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/L18-1488/) · [PDF](https://aclanthology.org/L18-1488.pdf) · [OpenAlex](https://openalex.org/W2807526605)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · 2 in our library, of the 141,840 papers whose bibliographies we read

## Summary
The authors build a very large English resource of automatically labelled semantic frames, aimed at modelling generalized event knowledge, meaning how typical a noun is as, say, the agent or instrument of a verb. Psycholinguistic and distributional work lacked big role-annotated data for this. They ran the SENNA labeller over ukWaC and the BNC to get PropBank-style role spans for verbal predicates. They chose SENNA because it does not lean on a syntactic parse, so the roles are not fully tied to one syntactic theory. A cascade of heuristics then picks a single head word for each role span, using POS tags and MaltParser dependency trees. The result, RW-eng, is distributed as XML with lemmas, POS tags and word positions but without full sentences, because of licensing. It comes with Python tools for building verb–role–filler tensors and for sampling the corpus. Unsupervised thematic-fit models built on a development version beat comparable syntax-based models when correlated with human judgements. This makes the resource useful for psycholinguistic modelling and for semantically aware language models.

## Key points
- Scale: roughly 78M sentences in 2.3M documents, with about 210M predicates and 704M role-fillers. For parallel processing the source was split into 3500 segments.
- Labelling uses SENNA (75.49% on CoNLL 2005) with PropBank roles. Head finding uses three fallbacks: MALT, MALT-SPAN and LINEAR. If all three fail, the span is marked FAILED and kept as a whole constituent.
- Manual check of 200 heads, taking SENNA's labels as given:
  - in 39% of cases the span was already a single word;
  - in 48.5% the heuristic found a correct head.
- Thematic-fit correlation (Padó data, ρ):
  - TypeDM+SDDM: 59
  - SDDM: 56
  - TypeDM: 51
  - Padó: 51
  - ParCos: 48
  - DepDM: 35
- Tooling:
  - ukwac2tensor converts the corpus to XML and then to an order-3 Pandas/HDF5 tensor of counts or PMI.
  - ukwac-heads-api provides filtered vocabularies, generators and filtering by head-finding algorithm.

## Interesting ideas
- The head-finding cascade has three steps: follow dependency links from the predicate into the span, then try the word attached furthest left, then fall back to a POS-based linear scan that skips adjuncts. Each head records which rule produced it, which makes errors traceable.
- They deliberately used a role labeller that is independent of the parser, so that role statistics are not just syntax in disguise. Independent sources make agreement more informative.
- With ambiguous spans (conjunctions, compounds, unclear pronouns), the heuristics return one head and favour precision over recall.
- Verb × role × filler counts are compiled into a PMI tensor, giving role-specific vectors for selectional preference, or thematic fit.
- They evaluated one component in isolation: the head heuristic was checked by hand while the upstream labels were assumed correct, in addition to extrinsic task scores.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): role-filler heads in prag frames could carry a provenance tag (tree-path, span-attachment, linear, failed), so explanations show how a filler was chosen and failures stay visible.
- **I2** (learn): when witness parsers or labellers vote on silver data, prefer witnesses with different architectures and inputs, so their consensus is not shared bias.
- **I3** (world): when filling entity slots from frames, commit only to a confident single filler and keep the full span otherwise, rather than inventing entities.
- **I4** (world): a similar compiled table over VerbNet roles could score how plausible a candidate filler is, to help rank readings and coreference choices.
- **I5** (learn): evaluate each pipeline stage with gold inputs from the stage before, so a downstream error can be attributed to the right component.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Saarland University](https://md.research.elamur.ai/free/en/universities/saarland-university/) — University, Saarbrücken, Germany · 102 works · 17,447 citations
- [University of Gothenburg](https://md.research.elamur.ai/free/en/universities/university-of-gothenburg/) — University, Gothenburg, Sweden · 20 works · 1,231 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 1,373 works · [🎭 Semantic role labeling](https://md.research.elamur.ai/free/en/topics/semantic-role-labeling/) · 132 works

## Cite

Cite as: Asad Sayeed, Pavel Shkadzko and Vera Demberg. 2018. Rollenwechsel-English: a large-scale semantic role corpus. Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{sayeed2018rollenwechsel,
  title = {{Rollenwechsel-English: a large-scale semantic role corpus}},
  author = {Asad Sayeed, Pavel Shkadzko and Vera Demberg},
  booktitle = {Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018)},
  year = {2018},
  doi = {10.63317/4atchm3v4amc},
  url = {https://doi.org/10.63317/4atchm3v4amc}
}
```

- [← previous Prosodic entrainment in dialog acts](https://md.research.elamur.ai/free/en/papers/reichel-2018-prosodic-entrainment-dialog-acts/)
- [next → A Study on Dialog Act Recognition using Character-Level Tokenization](https://md.research.elamur.ai/free/en/papers/ribeiro-2018-study-dialog-act-recognition-using/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/6/)

## References

19 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Amsel, B. D., DeLong, K. A., and Kutas, M. (2015). Close, but no garlic: Perceptuomotor and event knowledge activation during language comprehension. Journal of Memory and Language, 82(Supplement C):118 – 132. [↗](https://doi.org/10.1016/j.jml.2015.03.009)
2. Baroni, M. and Lenci, A. (2010). [Distributional memory: A general framework for corpus-based semantics](https://md.research.elamur.ai/free/en/papers/baroni-2010-distributional-memory-general-framework-corpus/). Computational Linguistics, 36(4):673–721. [↗](https://doi.org/10.1162/coli_a_00016)
3. Bicknell, K., Elman, J. L., Hare, M., McRae, K., and Kutas, M. (2010). Effects of event knowledge in processing verbal arguments. Journal of Memory and Language, 63(4):489–505. [↗](https://doi.org/10.1016/j.jml.2010.08.004)
4. BNC Consortium. (2007). The British National Corpus, version 3 (BNC XML Edition). Distributed by Oxford University Computing Services on behalf of the BNC Consortium. [find](https://scholar.google.com/scholar?q=BNC+Consortium%2E+%282007%29%2E+The+British+National+Corpus%2C+version+3+%28BNC+XML+Edition%29%2E+Distributed+by+Oxford+University+Computing+Services+on+behalf+of+the+BNC+Consortium%2E)
5. Collobert, R. and Weston, J. (2007). Fast semantic extraction using a novel neural network architecture. In Proceedings of the 45th Annual Meeting of the Association of Computational Linguistics, pages 560–567, Prague, Czech Republic, June. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Collobert%2C+R%2E+and+Weston%2C+J%2E+%282007%29%2E+Fast+semantic+extraction+using+a+novel+neural+network+architecture%2E+In+Proceedings+of+the+45th+Annual+Meeting+of+the+Association+of+Computational+Linguistics%2C+page)
6. Collobert, R., Weston, J., Bottou, L., Karlen, M., Kavukcuoglu, K., and Kuksa, P. (2011). [Natural language processing (almost) from scratch](https://md.research.elamur.ai/free/en/papers/collobert-2011-natural-language-processing-almost-scratch/). The Journal of Machine Learning Research, 12:2493–2537.
7. Ferraresi, A., Zanchetta, E., Baroni, M., and Bernardini, S. (2008). Introducing and evaluating ukWaC, a very large web-derived corpus of english. Proceedings of the 4th Web as Corpus Workshop (WAC-4) Can we beat Google, pages 47–54. [find](https://scholar.google.com/scholar?q=Ferraresi%2C+A%2E%2C+Zanchetta%2C+E%2E%2C+Baroni%2C+M%2E%2C+and+Bernardini%2C+S%2E+%282008%29%2E+Introducing+and+evaluating+ukWaC%2C+a+very+large+web%2Dderived+corpus+of+english%2E+Proceedings+of+the+4th+Web+as+Corpus+Workshop+%28WAC%2D4%29)
8. Ferretti, T. R., McRae, K., and Hatherell, A. (2001). Integrating verbs, situation schemas, and thematic role concepts. Journal of Memory and Language, 44(4):516– 547. [↗](https://doi.org/10.1006/jmla.2000.2728)
9. Greenberg, C., Sayeed, A., and Demberg, V. (2015). Improving unsupervised vector-space thematic fit evaluation via role-filler prototype clustering. In Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics Human Language Technologies (NAACL HLT). [find](https://scholar.google.com/scholar?q=Greenberg%2C+C%2E%2C+Sayeed%2C+A%2E%2C+and+Demberg%2C+V%2E+%282015%29%2E+Improving+unsupervised+vector%2Dspace+thematic+fit+evaluation+via+role%2Dfiller+prototype+clustering%2E+In+Proceedings+of+the+2015+Conference+of+the+North+)
10. Kingsbury, P. and Palmer, M. (2002). [From TreeBank to PropBank](https://md.research.elamur.ai/free/en/papers/kingsbury-2002-treebank-propbank/). In LREC. European Language Resources Association.
11. Magerman, D. M. (1994). Natural Lagnuage Parsing as Statistical Pattern Recognition. Ph.D. thesis, Stanford University. [find](https://scholar.google.com/scholar?q=Magerman%2C+D%2E+M%2E+%281994%29%2E+Natural+Lagnuage+Parsing+as+Statistical+Pattern+Recognition%2E+Ph%2ED%2E+thesis%2C+Stanford+University%2E)
12. McRae, K., Spivey-Knowlton, M. J., and Tanenhaus, M. K. (1998). Modeling the influence of thematic fit (and other constraints) in on-line sentence comprehension. Journal of Memory and Language, 38(3):283–312. [↗](https://doi.org/10.1006/jmla.1997.2543)
13. McRae, K., Hare, M., Elman, J. L., and Ferretti, T. (2005). A basis for generating expectancies for verbs from nouns. Memory and Cognition, 33(7):1174–1184. [↗](https://doi.org/10.3758/bf03193221)
14. Nivre, J., Hall, J., Nilsson, J., Chanev, A., Eryigit, G., Kübler, S., Marinov, S., and Marsi, E. (2007). Maltparser: A language-independent system for data-driven dependency parsing. Natural Language Engineering, 13(2):95–135. [↗](https://doi.org/10.1017/s1351324906004505)
15. Padó, U. (2007). The integration of syntax and semantic plausibility in a wide-coverage model of human sentence processing. Ph.D. thesis, Saarland University. [↗](https://doi.org/10.22028/d291-23491)
16. Sayeed, A., Demberg, V., and Shkadzko, P. (2015). An exploration of semantic features in an unsupervised thematic fit evaluation framework. In IJCoL vol. 1, n. 1 december 2015: Emerging Topics at the First Italian Conference on Computational Linguistics, pages 25–40. Accademia University Press. [↗](https://doi.org/10.4000/ijcol.298)
17. Sayeed, A., Hong, X., and Demberg, V. (2016). Roleo: Visualising thematic fit spaces on the web. In Proceedings of ACL-2016 System Demonstrations, pages 139– 144, Berlin, Germany. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sayeed%2C+A%2E%2C+Hong%2C+X%2E%2C+and+Demberg%2C+V%2E+%282016%29%2E+Roleo%3A+Visualising+thematic+fit+spaces+on+the+web%2E+In+Proceedings+of+ACL%2D2016+System+Demonstrations%2C+pages+139%E2%80%93+144%2C+Berlin%2C+Germany%2E+Association+for+Comp)
18. Tilk, O., Demberg, V., Sayeed, A. B., Klakow, D., and Thater, S. (2016). Event participant modelling with neural networks. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 171–182. [find](https://scholar.google.com/scholar?q=Tilk%2C+O%2E%2C+Demberg%2C+V%2E%2C+Sayeed%2C+A%2E+B%2E%2C+Klakow%2C+D%2E%2C+and+Thater%2C+S%2E+%282016%29%2E+Event+participant+modelling+with+neural+networks%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Languag)
19. Titov, I. and Klementiev, A. (2012). A Bayesian approach to unsupervised semantic role induction. In Proceedings of the 13th Conference of the European Chapter of the Association for Computational Linguistics, pages 12–22. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Titov%2C+I%2E+and+Klementiev%2C+A%2E+%282012%29%2E+A+Bayesian+approach+to+unsupervised+semantic+role+induction%2E+In+Proceedings+of+the+13th+Conference+of+the+European+Chapter+of+the+Association+for+Computational+Lin)

*Updated 24 October 2026*