---
title: "An Annotation Scheme for Information Status in Dialogue (2004)"
description: "The paper proposes a scheme for labelling the information status (IS) of every noun phrase in… 25 references, cited by 47 papers here."
authors: ["Malvina Nissim", "Shipra Dingare", "Jean Carletta", "Mark Steedman"]
year: 2004
venue: "Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC'04)"
source: "https://aclanthology.org/L04-1402/"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/nissim-2004-annotation-scheme-information-status-dialogue/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nissim-2004-annotation-scheme-information-status-dialogue/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2004](https://md.research.elamur.ai/free/en/papers/2004/) / An Annotation Scheme for Information Status in Dialogue

# An Annotation Scheme for Information Status in Dialogue

**Authors:** [Malvina Nissim](https://md.research.elamur.ai/free/en/authors/malvina-nissim/), Shipra Dingare, [Jean Carletta](https://md.research.elamur.ai/free/en/authors/jean-carletta/) and [Mark Steedman](https://md.research.elamur.ai/free/en/authors/mark-steedman/) · **Year:** 2004 · **Venue:** [Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC'04)](https://md.research.elamur.ai/free/en/venues/lrec/) ([LREC 2004](https://md.research.elamur.ai/free/en/venues/lrec/2004/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/L04-1402/) · [PDF](https://aclanthology.org/L04-1402.pdf) · [OpenAlex](https://openalex.org/W101214240)

**Citations:** 71 citations (OpenAlex, 2026-10-14) · [47](https://md.research.elamur.ai/free/en/papers/nissim-2004-annotation-scheme-information-status-dialogue/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper proposes a scheme for labelling the information status (IS) of every noun phrase in spontaneous dialogue. IS reflects what the speaker assumes the hearer already knows. Earlier efforts mostly covered only definites or demonstratives, used flat inventories of many classes, and often reported low or no agreement figures. The authors instead use a coarse three-way split based on Prince: old, mediated (inferrable or generally known) and new. Optional finer subtypes sit beneath old and mediated. Two categories handle markables that should not be judged: non-applicable and not-understood. Markables were extracted automatically from the Penn Treebank parses of Switchboard and annotated with the NITE XML Toolkit, using written guidelines and a precedence decision tree. On held-out dialogues, two annotators reached kappa of about .86 on the top-level categories after training. The scheme was then applied to 147 dialogues, producing what the authors describe as the largest IS-annotated corpus available. Such a corpus supports studies of syntactic choice, anaphora resolution, generation and discourse analysis.

## Key points
- The top-level labels are old, mediated and new, plus non-applicable and not-understood. Old has 6 subtypes (identity, event, general, generic, ident generic, relative). Nine subtypes fall under mediated: aggregation, func value, poss, set, event, situation, part, bound and general. New has no subtypes.
- Reliability was tested on 3 Switchboard dialogues totalling 1738 NPs, with two annotators:
  - The training dialogue DiaA gave K = .794.
  - DiaB and DiaC gave K = .860 and .857.
  - Agreement on non-applicable alone rose from .614 to .916 and .888 after training.
- The final corpus covers 147 dialogues and 43,358 sentences. It contains 69,004 valid NPs and 16,324 coreference links.
  - 35,299 NPs are old, 23,816 mediated and 9,889 new.
  - 8,127 markables were marked non-applicable and 160 not-understood.
- Markables are all NPs plus possessive pronouns. Locative, directional and adverbial NPs, disfluencies and traces are excluded or left untagged.
- Comparable schemes for definite descriptions report lower agreement, roughly K = .45 to .79 depending on the study and language. These include Poesio & Vieira, Spenader, and Salmon-Alt & Vieira.

## Interesting ideas
- Organise labels as a hierarchy: a small, reliably agreed top layer with optional subtypes underneath, so coarse agreement survives even when fine distinctions are disputed.
- A separate "non-applicable" bucket catches markables that are artefacts of extraction, such as "course" in "of course", "there" in "there is" and expletive "it".
- An explicit precedence decision tree resolves cases where several labels fit (e.g. old/identity beats mediated/general), which the annotators credited with much of the agreement gain.
- Bridging subtypes are anchored in substitution tests and lexical resources: WordNet for part-whole links and FrameNet for situation links.
- Two old entities joined in a coordination form a new mediated "aggregation" unless that group was mentioned before.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): Coreference and bridging output could always commit to old/mediated/new and attach a subtype only when confident, so downstream readers get a dependable coarse signal.
- **I2** (world): Mova could filter such NPs before creating world entities, so idioms and expletives never become referents.
- **I3** (learn): When several witness parsers or LLM teachers give overlapping labels, a fixed priority order could merge them deterministically into silver data.
- **I4** (world): Mova's WordNet meronymy and FrameNet frames could license inferred entities, such as "the door" of a home, giving an explainable reason for each mediated link.
- **I5** (world): Mova could create a fresh group entity for coordinations like "John and Mary", linked to its members, so a later "they" resolves to the group.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Edinburgh](https://md.research.elamur.ai/free/en/universities/university-of-edinburgh/) — University, Edinburgh, United Kingdom · 381 works · 80,747 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 654 works

## Cite

Cite as: Malvina Nissim, Shipra Dingare, Jean Carletta and Mark Steedman. 2004. An Annotation Scheme for Information Status in Dialogue. Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC'04).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{nissim2004annotation,
  title = {{An Annotation Scheme for Information Status in Dialogue}},
  author = {Malvina Nissim, Shipra Dingare, Jean Carletta and Mark Steedman},
  booktitle = {Proceedings of the Fourth International Conference on Language Resources and Evaluation (LREC'04)},
  year = {2004},
  doi = {10.63317/5e8ipnwca99c},
  url = {https://doi.org/10.63317/5e8ipnwca99c}
}
```

- [← previous The MATE/GNOME Proposals for Anaphoric Annotation, Revisited](https://md.research.elamur.ai/free/en/papers/poesio-2004-mate-gnome-proposals-anaphoric-annotation/)
- [next → Unsupervised Learning of Contextual Role Knowledge for Coreference Resolution](https://md.research.elamur.ai/free/en/papers/bean-2004-unsupervised-learning-contextual-role-knowledge/)
- [↑ back to papers of 2004](https://md.research.elamur.ai/free/en/papers/2004/)

## References

25 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. M. Ariel. 1990. Accessing Noun Phrase Antecedents. Routledge, New York. [find](https://scholar.google.com/scholar?q=M%2E+Ariel%2E+1990%2E+Accessing+Noun+Phrase+Antecedents%2E+Routledge%2C+New+York%2E)
2. C. F. Baker, C. J. Fillmore, and J. B. Lowe. 1998. [The Berkeley FrameNet project](https://md.research.elamur.ai/free/en/papers/baker-1998-berkeley-framenet/). In Christian Boitet and Pete Whitelock, editors, Proceedings of COLING-ACL, pages 86–90.
3. B. Birner and G. Ward. 1998. Information Status and Noncanonical Word Order in English. Cambridge University Press. [find](https://scholar.google.com/scholar?q=B%2E+Birner+and+G%2E+Ward%2E+1998%2E+Information+Status+and+Noncanonical+Word+Order+in+English%2E+Cambridge+University+Press%2E)
4. J. Carletta, S. Dingare, M. Nissim, and T. Nikitina. 2004. Using the NITE XML Toolkit on the Switchboard Corpus to study syntactic choice: a case study. In Proc. of LREC2004. [find](https://scholar.google.com/scholar?q=J%2E+Carletta%2C+S%2E+Dingare%2C+M%2E+Nissim%2C+and+T%2E+Nikitina%2E+2004%2E+Using+the+NITE+XML+Toolkit+on+the+Switchboard+Corpus+to+study+syntactic+choice%3A+a+case+study%2E+In+Proc%2E+of+LREC2004%2E)
5. J. Carletta. 1996. [Assessing agreement on classification tasks: the kappa statistic](https://md.research.elamur.ai/free/en/papers/carletta-1996-assessing-agreement-classification-tasks-kappa/). Computational Linguistics, 22(2):249–254.
6. H. H. Clark. 1975. Bridging. In R. Schank and B. Nash-Webber, editors, Theoretical Issues in Natural Language Processing. The MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=H%2E+H%2E+Clark%2E+1975%2E+Bridging%2E+In+R%2E+Schank+and+B%2E+Nash%2DWebber%2C+editors%2C+Theoretical+Issues+in+Natural+Language+Processing%2E+The+MIT+Press%2C+Cambridge%2C+MA%2E)
7. S. Davies, M. Poesio, F. Bruneseaux, and L. Romary. 1998. Annotating coreference in dialogues: Proposal for a scheme for MATE, July. [http://www.hcrc](http://www.hcrc). ed.ac.uk/˜poesio/anno_manual.html. [↗](http://www.hcrc)
8. M. Eckert and M. Strube. 2001. Dialogue acts, synchronising units and anaphora resolution. Journal of Semantics, 17(1):51–89. [find](https://scholar.google.com/scholar?q=M%2E+Eckert+and+M%2E+Strube%2E+2001%2E+Dialogue+acts%2C+synchronising+units+and+anaphora+resolution%2E+Journal+of+Semantics%2C+17%281%29%3A51%E2%80%9389%2E)
9. C. Fellbaum, editor. 1998. WordNet: An Electronic Lexical Database. The MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=C%2E+Fellbaum%2C+editor%2E+1998%2E+WordNet%3A+An+Electronic+Lexical+Database%2E+The+MIT+Press%2C+Cambridge%2C+MA%2E)
10. J. Godfrey, E. Holliman, and J. McDaniel. 1992. SWITCHBOARD: Telephone speech corpus for research and development. In Proceedings of ICASSP-92, pages 517–520. [find](https://scholar.google.com/scholar?q=J%2E+Godfrey%2C+E%2E+Holliman%2C+and+J%2E+McDaniel%2E+1992%2E+SWITCHBOARD%3A+Telephone+speech+corpus+for+research+and+development%2E+In+Proceedings+of+ICASSP%2D92%2C+pages+517%E2%80%93520%2E)
11. J. Gundel, N. Hedberg, and R. Zacharski. 1993. Cognitive status and the form of referring expressions in discourse. Language, 69(2):274–307. [find](https://scholar.google.com/scholar?q=J%2E+Gundel%2C+N%2E+Hedberg%2C+and+R%2E+Zacharski%2E+1993%2E+Cognitive+status+and+the+form+of+referring+expressions+in+discourse%2E+Language%2C+69%282%29%3A274%E2%80%93307%2E)
12. J. A. Hawkins. 1978. Definiteness and Indefiniteness. Croom Helm, London. [find](https://scholar.google.com/scholar?q=J%2E+A%2E+Hawkins%2E+1978%2E+Definiteness+and+Indefiniteness%2E+Croom+Helm%2C+London%2E)
13. L. Hirschman and N. Chinchor. 1997. [MUC-7 coreference task definition](https://md.research.elamur.ai/free/en/papers/hirschman-1998-appendix-f-muc-7-coreference/). In Proc. of MUC-7.
14. K. Lambrecht. 1994. Information structure and sentence form. Topic, focus, and the mental representation of discourse referents. CUP, Cambridge. [find](https://scholar.google.com/scholar?q=K%2E+Lambrecht%2E+1994%2E+Information+structure+and+sentence+form%2E+Topic%2C+focus%2C+and+the+mental+representation+of+discourse+referents%2E+CUP%2C+Cambridge%2E)
15. S. Löbner. 1985. Definites. Journal of Semantics, 4:279–326. [find](https://scholar.google.com/scholar?q=S%2E+Lo%CC%88bner%2E+1985%2E+Definites%2E+Journal+of+Semantics%2C+4%3A279%E2%80%93326%2E)
16. R. Passonneau. 1996. Instructions for applying discourse reference annotation for multiple applications (DRAMA). Unpublished manuscript, December. [find](https://scholar.google.com/scholar?q=R%2E+Passonneau%2E+1996%2E+Instructions+for+applying+discourse+reference+annotation+for+multiple+applications+%28DRAMA%29%2E+Unpublished+manuscript%2C+December%2E)
17. M. Poesio and R. Vieira. 1998. A corpus-based investigation of definite descriptions use. Computational Linguistics, 24(2):183–216. [find](https://scholar.google.com/scholar?q=M%2E+Poesio+and+R%2E+Vieira%2E+1998%2E+A+corpus%2Dbased+investigation+of+definite+descriptions+use%2E+Computational+Linguistics%2C+24%282%29%3A183%E2%80%93216%2E)
18. M. Poesio. 2000. The GNOME annotation scheme manual. [http://www.hcrc](http://www.hcrc). ed.ac.uk/˜gnome/anno_manual.html. [↗](http://www.hcrc)
19. E. F. Prince. 1981. Toward a taxonomy of given-new information. In Peter Cole, editor, Radical Pragmatics. Academic Press, New York. [find](https://scholar.google.com/scholar?q=E%2E+F%2E+Prince%2E+1981%2E+Toward+a+taxonomy+of+given%2Dnew+information%2E+In+Peter+Cole%2C+editor%2C+Radical+Pragmatics%2E+Academic+Press%2C+New+York%2E)
20. E. Prince. 1992. The ZPG letter: subjects, definiteness, and information-status. In S. Thompson and W. Mann, editors, Discourse description: diverse analyses of a fund raising text, pages 295–325. John Benjamins, Philadelphia/Amsterdam. [find](https://scholar.google.com/scholar?q=E%2E+Prince%2E+1992%2E+The+ZPG+letter%3A+subjects%2C+definiteness%2C+and+information%2Dstatus%2E+In+S%2E+Thompson+and+W%2E+Mann%2C+editors%2C+Discourse+description%3A+diverse+analyses+of+a+fund+raising+text%2C+pages+295%E2%80%93325%2E+Joh)
21. S. Salmon-Alt and R. Vieira. 2002. Nominal expressions in multilingual corpora: Definites and demonstratives. In Proc. of LREC2002, 2002, pages 1627–1634. [find](https://scholar.google.com/scholar?q=S%2E+Salmon%2DAlt+and+R%2E+Vieira%2E+2002%2E+Nominal+expressions+in+multilingual+corpora%3A+Definites+and+demonstratives%2E+In+Proc%2E+of+LREC2002%2C+2002%2C+pages+1627%E2%80%931634%2E)
22. J. Spenader. 2001. Between binding and accommodation. In P. Kühnlein, H. Rieser, and H. Zeevat, editors, Proceedings of Bi-Dialog 2001, pages 162–173. [find](https://scholar.google.com/scholar?q=J%2E+Spenader%2E+2001%2E+Between+binding+and+accommodation%2E+In+P%2E+Ku%CC%88hnlein%2C+H%2E+Rieser%2C+and+H%2E+Zeevat%2C+editors%2C+Proceedings+of+Bi%2DDialog+2001%2C+pages+162%E2%80%93173%2E)
23. M. Steedman. 2000. The Syntactic Process. The MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=M%2E+Steedman%2E+2000%2E+The+Syntactic+Process%2E+The+MIT+Press%2C+Cambridge%2C+MA%2E)
24. E. Vallduvı́. 1992. The Informational Component. Garland, New York. [find](https://scholar.google.com/scholar?q=E%2E+Vallduv%C4%B1%CC%81%2E+1992%2E+The+Informational+Component%2E+Garland%2C+New+York%2E)
25. T. Wasow. 2002. Postverbal Behavior. CSLI Publications. [find](https://scholar.google.com/scholar?q=T%2E+Wasow%2E+2002%2E+Postverbal+Behavior%2E+CSLI+Publications%2E)

*Updated 2 November 2026*