---
title: "Anaphoric Annotation of Wikipedia and Blogs — Rodríguez 2010"
description: "Italian has few corpora annotated for anaphora. Social Web text such as Wikipedia articles and blog comments also lacks annotated data for training…"
authors: ["Kepa Joseba Rodríguez", "Francesca Delogu", "Yannick Versley", "Egon W. Stemle", "Massimo Poesio"]
year: 2010
venue: "Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)"
source: "https://aclanthology.org/L10-1295/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/rodriguez-2010-anaphoric-annotation-wikipedia-blogs-live/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/rodriguez-2010-anaphoric-annotation-wikipedia-blogs-live/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2010](https://md.research.elamur.ai/free/papers/2010/) / Anaphoric Annotation of Wikipedia and Blogs in the Live Memories Corpus

# Anaphoric Annotation of Wikipedia and Blogs in the Live Memories Corpus

**Authors:** [Kepa Joseba Rodríguez](https://md.research.elamur.ai/free/authors/kepa-joseba-rodriguez/), Francesca Delogu, [Yannick Versley](https://md.research.elamur.ai/free/authors/yannick-versley/), [Egon W. Stemle](https://md.research.elamur.ai/free/authors/egon-w-stemle/), [Massimo Poesio](https://md.research.elamur.ai/free/authors/massimo-poesio/) · **Year:** 2010 · **Venue:** Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/L10-1295/) · [PDF](https://aclanthology.org/L10-1295.pdf) · [OpenAlex](https://openalex.org/W116192430)

**Citations:** 38 citations (openalex-snapshot · 2026-10-14) · [18](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Italian has few corpora annotated for anaphora. Social Web text such as Wikipedia articles and blog comments also lacks annotated data for training entity and coreference systems, and earlier work showed that resolvers trained on news perform much worse on user comments. The authors address both gaps with the Live Memories Corpus (LMC). It contains Italian Wikipedia articles and Creative Commons–licensed blog posts with user comments, all about the Trentino/Südtirol region, and newspaper text from l'Adige is planned. Web pages are cleaned with KrdWrd. NP markables are extracted automatically using TextPro and the MALT parser trained on the TUT treebank, then annotated by hand in MMAX2. The scheme adapts ARRAU and borrows from VENEX. It covers agreement, information status, semantic class, discourse deixis, bridging and ambiguous antecedents. It also handles Italian-specific cases: empty subjects, clitics attached to verbs, and discontinuous coordination markables. Agreement between two annotators reaches Kappa 0.79–0.93, which suggests the scheme is reliable and reusable. The Wikipedia portion is used in the SemEval 2010 multilingual coreference task.

## Key points
- The Wikipedia dataset has 144 files (about 142K words) and 44.5K markables. Of these markables, 57.8% are discourse-new, 28.5% discourse-old and 13.7% non-referring. The target size is about 150,000 words per genre.
- Empty subjects make up about 4.5% of anaphoric expressions in the Wikipedia data, and every one is linked to an antecedent.
- Inter-annotator agreement (Kappa) was:
  - information status: 0.80
  - basic markable annotation: 0.79
  - semantic type: 0.85
  - antecedent links: 0.88
  - clitic antecedents: 0.84
  - empty-pronoun antecedents: 0.93
- The blog data differs from Wikipedia in several ways:
  - Predicates make up only about 25% of its non-referring expressions.
  - Idioms are about three times as frequent.
  - Expletives and clitics are more common.
- The corpus will be released in MMAX2 stand-off format and in a CoNLL-style tabular format, under a Creative Commons Attribution license.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

Topics: [Discourse analysis](https://md.research.elamur.ai/free/topics/discourse-analysis/) · [Coreference resolution](https://md.research.elamur.ai/free/topics/coreference-resolution/)

## Cited by

Cited by

18 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 12 not yet.

- [WikiCoref: An English Coreference-annotated Corpus of Wikipedia Articles](https://md.research.elamur.ai/free/papers/ghaddar-2016-wikicoref-english-coreference-annotated-corpus/)
- [A Cross-Lingual ILP Solution to Zero Anaphora Resolution](https://md.research.elamur.ai/free/papers/iida-2011-cross-lingual-ilp-solution-zero/)
- [Bridging Resolution: A Survey of the State of the Art](https://md.research.elamur.ai/free/papers/kobayashi-2020-bridging-resolution-survey-state-art/)
- [An Empirical Study of Contextual Data Augmentation for Japanese Zero Anaphora Resolution](https://md.research.elamur.ai/free/papers/konno-2020-empirical-study-contextual-data-augmentation/)
- [Coreference meets Universal Dependencies – a pilot experiment on harmonizing coreference datasets for 11 languages](https://md.research.elamur.ai/free/papers/nedoluzhko-2021-coreference-meets-universal-dependencies/)
- [Introducing the Prague Discourse Treebank 1.0](https://md.research.elamur.ai/free/papers/polakova-2013-introducing-prague-discourse-treebank-1/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{rodriguez2010anaphoric,
  title = {{Anaphoric Annotation of Wikipedia and Blogs in the Live Memories Corpus}},
  author = {Rodríguez, Kepa Joseba and Delogu, Francesca and Versley, Yannick and Stemle, Egon W. and Poesio, Massimo},
  booktitle = {Proceedings of the Seventh International Conference on Language Resources and Evaluation (LREC'10)},
  year = {2010},
  doi = {10.63317/4hbf54ijjdr9},
  url = {https://doi.org/10.63317/4hbf54ijjdr9}
}
```

- [← previous An Overview of the CRAFT Concept Annotation Guidelines](https://md.research.elamur.ai/free/papers/bada-2010-overview-craft-concept-annotation-guidelines/)
- [next → Annotation of Discourse Relations for Conversational Spoken Dialogs](https://md.research.elamur.ai/free/papers/tonelli-2010-annotation-discourse-relations-conversational-spoken/)
- [↑ back to papers of 2010](https://md.research.elamur.ai/free/papers/2010/)

## References

20 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. A. Balahur, E. Boldrini, A. Montoyo, and P. Martı́nez-Barco. 2009a. Opinion and generic question answering systems: a performance analysis. In ACL-IJCNLP ’09: Proceedings of the ACL-IJCNLP 2009 Conference Short Papers, pages 157–160, Morristown, NJ, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=A%2E+Balahur%2C+E%2E+Boldrini%2C+A%2E+Montoyo%2C+and+P%2E+Mart%C4%B1%CC%81nez%2DBarco%2E+2009a%2E+Opinion+and+generic+question+answering+systems%3A+a+performance+analysis%2E+In+ACL%2DIJCNLP+%E2%80%9909%3A+Proceedings+of+the+ACL%2DIJCNLP+2009+Confer)
2. A. Balahur, E. Lloret, E. Boldrini, A. Montoyo, M. Palomar, and P. Martinez-Barco. 2009b. Summarizing threads in blogs using opinion polarity. In Constantin Orăsan, Laura Hasler, and Corina Forascu, editors, Proceedings of the International Workshop on Events in Emerging Text Types (eETTs), pages 5… [find](https://scholar.google.com/scholar?q=A%2E+Balahur%2C+E%2E+Lloret%2C+E%2E+Boldrini%2C+A%2E+Montoyo%2C+M%2E+Palomar%2C+and+P%2E+Martinez%2DBarco%2E+2009b%2E+Summarizing+threads+in+blogs+using+opinion+polarity%2E+In+Constantin+Ora%CC%86san%2C+Laura+Hasler%2C+and+Corina+Forascu%2C+)
3. P. Bo and L. Lee. 2008. Opinion mining and sentiment analysis. Foundations and Trends in Information Retrival, 2(1 & 2):1 – 135. [find](https://scholar.google.com/scholar?q=P%2E+Bo+and+L%2E+Lee%2E+2008%2E+Opinion+mining+and+sentiment+analysis%2E+Foundations+and+Trends+in+Information+Retrival%2C+2%281+%26+2%29%3A1+%E2%80%93+135%2E)
4. Ester Boldrini, Alexandra Balahur, Patricio Martnez-Barco, and Andrs Montoyo. 2009. Emotiblog: an annotation scheme for emotion detection and analysis in non-traditional textual genres. In Robert Stahlbock, Sven F. Crone, and Stefan Lessmann, editors, DMIN, pages 491– 497. CSREA Press. [find](https://scholar.google.com/scholar?q=Ester+Boldrini%2C+Alexandra+Balahur%2C+Patricio+Martnez%2DBarco%2C+and+Andrs+Montoyo%2E+2009%2E+Emotiblog%3A+an+annotation+scheme+for+emotion+detection+and+analysis+in+non%2Dtraditional+textual+genres%2E+In+Robert+Stah)
5. C. Bosco, V. Lombardo, D. Vassallo, and L. Lesmo. 2000. Building a treebank for italian: a data-driven annotation scheme. In Proceedings of the LREC-00, Athens. Greece. [find](https://scholar.google.com/scholar?q=C%2E+Bosco%2C+V%2E+Lombardo%2C+D%2E+Vassallo%2C+and+L%2E+Lesmo%2E+2000%2E+Building+a+treebank+for+italian%3A+a+data%2Ddriven+annotation+scheme%2E+In+Proceedings+of+the+LREC%2D00%2C+Athens%2E+Greece%2E)
6. Jean Carletta. 1996. Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22:249–254. [find](https://scholar.google.com/scholar?q=Jean+Carletta%2E+1996%2E+Assessing+agreement+on+classification+tasks%3A+The+kappa+statistic%2E+Computational+Linguistics%2C+22%3A249%E2%80%93254%2E) [our card](https://md.research.elamur.ai/free/papers/carletta-1996-assessing-agreement-classification-tasks-kappa/)
7. I. Hendrickx and V. Hoste. 2009. Coreference resolution on blogs and commented news. In DAARC ’09: Proceedings of the 7th Discourse Anaphora and Anaphor Resolution Colloquium on Anaphora Processing and Applications, pages 43–53, Berlin, Heidelberg. Springer-Verlag. [find](https://scholar.google.com/scholar?q=I%2E+Hendrickx+and+V%2E+Hoste%2E+2009%2E+Coreference+resolution+on+blogs+and+commented+news%2E+In+DAARC+%E2%80%9909%3A+Proceedings+of+the+7th+Discourse+Anaphora+and+Anaphor+Resolution+Colloquium+on+Anaphora+Processing+an)
8. D.J. Hopkins and G. King. 2010. A method of automated nonparametric contente analysis for social science. American Journal of Political Science, 54(1):229 – 247. [find](https://scholar.google.com/scholar?q=D%2EJ%2E+Hopkins+and+G%2E+King%2E+2010%2E+A+method+of+automated+nonparametric+contente+analysis+for+social+science%2E+American+Journal+of+Political+Science%2C+54%281%29%3A229+%E2%80%93+247%2E)
9. Linguistic Data Consortium LDC, 2004. ACE (Automatic Content Extraction) English Annotation Guidelines for Entities, version 5.6.1. [find](https://scholar.google.com/scholar?q=Linguistic+Data+Consortium+LDC%2C+2004%2E+ACE+%28Automatic+Content+Extraction%29+English+Annotation+Guidelines+for+Entities%2C+version+5%2E6%2E1%2E)
10. B. Magnini and M. Poesio. 2009. Content extraction meets the social web in the livememories project. In Proceedings of the AT4DL. [find](https://scholar.google.com/scholar?q=B%2E+Magnini+and+M%2E+Poesio%2E+2009%2E+Content+extraction+meets+the+social+web+in+the+livememories+project%2E+In+Proceedings+of+the+AT4DL%2E)
11. B. Magnini, E. Pianta, Ch. Girardi, M. Negri, L. Romano, M. Speranza, and R. Sprugnoli. 2006. I-cab: the italian content annotation bank. In Proceedings of the LREC-06, Genova, Italia. [find](https://scholar.google.com/scholar?q=B%2E+Magnini%2C+E%2E+Pianta%2C+Ch%2E+Girardi%2C+M%2E+Negri%2C+L%2E+Romano%2C+M%2E+Speranza%2C+and+R%2E+Sprugnoli%2E+2006%2E+I%2Dcab%3A+the+italian+content+annotation+bank%2E+In+Proceedings+of+the+LREC%2D06%2C+Genova%2C+Italia%2E)
12. R. Mihalcea. 2007. Using wikipedia for automatic word sense disambiguation. In Proceedings of the NAACL. [find](https://scholar.google.com/scholar?q=R%2E+Mihalcea%2E+2007%2E+Using+wikipedia+for+automatic+word+sense+disambiguation%2E+In+Proceedings+of+the+NAACL%2E)
13. Ch. Müller and M. Strube. 2006. Multi-level annotation of linguistic data with mmax2. In S. Braun, K. Kohn, and J. Mukherjee, editors, Corpus Technology and Language Pedagogy: New Resources, New Tools, New Methods. Peter Lang, Frankfurt a. M., Germany. [find](https://scholar.google.com/scholar?q=Ch%2E+Mu%CC%88ller+and+M%2E+Strube%2E+2006%2E+Multi%2Dlevel+annotation+of+linguistic+data+with+mmax2%2E+In+S%2E+Braun%2C+K%2E+Kohn%2C+and+J%2E+Mukherjee%2C+editors%2C+Corpus+Technology+and+Language+Pedagogy%3A+New+Resources%2C+New+Tool)
14. N. Nicolov, F. Salvetti, and S. Ivanova. 2008. Sentiment analysis: Does coreference matter? In Proceedings of the Symposium on Affective Lenguage in Human and Machine, Aberden. UK. [find](https://scholar.google.com/scholar?q=N%2E+Nicolov%2C+F%2E+Salvetti%2C+and+S%2E+Ivanova%2E+2008%2E+Sentiment+analysis%3A+Does+coreference+matter%3F+In+Proceedings+of+the+Symposium+on+Affective+Lenguage+in+Human+and+Machine%2C+Aberden%2E+UK%2E)
15. J. Nivre, J. Hall, J. Nilsson, A. Chanev, G. Eryigit, S. Kbler, S. Marinov, and E. Marsi. 2007. Maltparser: A language-independent system for data-driven dependency parsing. Natural Language Engineering, 2(13). [find](https://scholar.google.com/scholar?q=J%2E+Nivre%2C+J%2E+Hall%2C+J%2E+Nilsson%2C+A%2E+Chanev%2C+G%2E+Eryigit%2C+S%2E+Kbler%2C+S%2E+Marinov%2C+and+E%2E+Marsi%2E+2007%2E+Maltparser%3A+A+language%2Dindependent+system+for+data%2Ddriven+dependency+parsing%2E+Natural+Language+Engineeri)
16. E. Pianta, Ch. Girardi, and R. Zanoli. 2008. The textpro tool suite. In Proceedings of LREC-08. [find](https://scholar.google.com/scholar?q=E%2E+Pianta%2C+Ch%2E+Girardi%2C+and+R%2E+Zanoli%2E+2008%2E+The+textpro+tool+suite%2E+In+Proceedings+of+LREC%2D08%2E)
17. M. Poesio and R. Artstein. 2008. Anaphoric annotation in the arrau corpus. In Proceedings of LREC-08, Marrakech, Marocco. [find](https://scholar.google.com/scholar?q=M%2E+Poesio+and+R%2E+Artstein%2E+2008%2E+Anaphoric+annotation+in+the+arrau+corpus%2E+In+Proceedings+of+LREC%2D08%2C+Marrakech%2C+Marocco%2E) [our card](https://md.research.elamur.ai/free/papers/poesio-2008-anaphoric-annotation-arrau-corpus/)
18. M. Poesio, R. Delmonte, A. Bristot, L. Chiran, and S. Tonelli. 2004. The venex corpus of anaphora and deixis in spoken and written italian. Manuscript. [find](https://scholar.google.com/scholar?q=M%2E+Poesio%2C+R%2E+Delmonte%2C+A%2E+Bristot%2C+L%2E+Chiran%2C+and+S%2E+Tonelli%2E+2004%2E+The+venex+corpus+of+anaphora+and+deixis+in+spoken+and+written+italian%2E+Manuscript%2E)
19. S. Ponzetto and M. Strube. 2007. Knowledge derived from wikipedia for computing semantic relatedness. Journal of Artificial Intelligence Research, 30. [find](https://scholar.google.com/scholar?q=S%2E+Ponzetto+and+M%2E+Strube%2E+2007%2E+Knowledge+derived+from+wikipedia+for+computing+semantic+relatedness%2E+Journal+of+Artificial+Intelligence+Research%2C+30%2E)
20. J. M. Steger and E. W. Stemle. 2009. The architecture for unified processing of web content. In Proceedings of the Fifth Web as Corpus Workshop (WAC5), Donostia-San Sebastian. Basque Country. [find](https://scholar.google.com/scholar?q=J%2E+M%2E+Steger+and+E%2E+W%2E+Stemle%2E+2009%2E+The+architecture+for+unified+processing+of+web+content%2E+In+Proceedings+of+the+Fifth+Web+as+Corpus+Workshop+%28WAC5%29%2C+Donostia%2DSan+Sebastian%2E+Basque+Country%2E)