---
title: "Contrasting the Automatic Identification of Two Discourse"
description: "The paper addresses a binary decision task: is a given occurrence of like or well a discourse marker (DM) or an… 0.78. recall. 20 references."
authors: ["Andrei Popescu-Belis", "Sandrine Zufferey"]
year: 2007
venue: "Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue"
source: "https://aclanthology.org/2007.sigdial-1.3/"
doi: "10.18653/v1/2007.sigdial-1.3"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/belis-2007-contrasting-automatic-identification-two-discourse/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/belis-2007-contrasting-automatic-identification-two-discourse/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2007](https://md.research.elamur.ai/free/en/papers/2007/page/2/) / Contrasting the Automatic Identification of Two Discourse Markers in Multiparty Dialogues

# Contrasting the Automatic Identification of Two Discourse Markers in Multiparty Dialogues

**Authors:** [Andrei Popescu-Belis](https://md.research.elamur.ai/free/en/authors/andrei-popescu-belis/) and [Sandrine Zufferey](https://md.research.elamur.ai/free/en/authors/sandrine-zufferey/) · **Year:** 2007 · **Venue:** [Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue](https://md.research.elamur.ai/free/en/venues/sigdial/) ([SIGDIAL 2007](https://md.research.elamur.ai/free/en/venues/sigdial/2007/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/2007.sigdial-1.3/) · [PDF](https://aclanthology.org/2007.sigdial-1.3.pdf) · [DOI 10.18653/v1/2007.sigdial-1.3](https://doi.org/10.18653/v1/2007.sigdial-1.3) · [OpenAlex](https://openalex.org/W4411117043)

**Citations:** 4 citations (OpenAlex, 2026-10-14)

## Summary
The paper addresses a binary decision task: is a given occurrence of *like* or *well* a discourse marker (DM) or an ordinary word? Both items are highly ambiguous and appear often in conversation. The authors annotated every occurrence of the two words in the ICSI Meeting Recorder Corpus, which contains 75 multi-party meetings, about 80 hours of audio and roughly 800,000 transcribed words. That yields more than 4,000 instances per item, far more than earlier studies, which often spread about 1,000 tokens over 30–40 DM types.

Using only surface cues, they trained C4.5 decision trees and other learners. The cues were:
- neighbouring words,
- utterance position,
- simple timing measures,
- speaker metadata.

The best accuracy is around 90%, with κ near 0.78. This beats earlier reported results and matches the agreement measured among human non-expert annotators.

Two findings stand out:
- Separate classifiers for each word work clearly better than a single pooled model.
- Lexical collocations, especially the word just before the candidate, carry most of the signal.

The work matters because DM detection is a prerequisite for tagging, parsing and understanding spontaneous dialogue.

## Key points
- **Data:**
  - ICSI Meeting Recorder Corpus, 75 meetings with 5–8 speakers, including native and non-native speakers.
  - *like* has 4,519 occurrences, of which 2,052 (45%) are DMs.
  - *well* has 4,136 occurrences, of which 3,639 (88%) are DMs.
- **Features:**
  - Lexical window features: WORD(−N)…WORD(+N), with a frequency cutoff F.
  - Positional features: INITIAL, FINAL.
  - Temporal features: PAUSE-BEFORE, PAUSE-AFTER, DURATION, taken from forced alignment.
  - Sociolinguistic features: GENDER, AGE, EDUCATION, NATIVE, ORIGIN.
  - TYPE, which marks whether the item is *like* or *well*.
- **Best combined result (Bayesian Network):**
  - CCI = 90%, κ = 0.78.
  - Recall 0.96, precision 0.90, f = 0.93.
  - C4.5 and SVM scores were only slightly lower.
- **Feature selection:**
  - CFS selected {TYPE, PAUSE-BEFORE, INITIAL, WORD(−1)}.
  - Lexical features alone come within 5% of the best overall scores.
  - Sociolinguistic features help only marginally, and mostly for *like*.
- **Human ceiling:** Non-expert inter-annotator agreement was κ = 0.74. An earlier trivial baseline in Siegel and McKeown's study, using only an utterance-initial feature, already reached 79.16% accuracy.

## Interesting ideas
- Train one classifier per ambiguous lexical item instead of one model for the whole DM class, since the useful cues differ from word to word.
- Treat human inter-annotator agreement as the upper bound on achievable scores, and report a chance-corrected κ rather than raw accuracy when classes are skewed (88% of *well* tokens are DMs).
- Compare against very simple baselines, such as a single positional feature, because they can already score high and make sophisticated models look better than they are.
- Run attribute selection on individual neighbouring words to surface readable collocation cues, such as *something/things/seems* before *like*, or *as well / very well / how well*, that signal a non-DM reading.
- When *well* is a DM, it can mark rejection of a request, disagreement with a prior utterance, hesitation, or turn-taking. Detecting it therefore also signals the speaker's stance.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): Add per-word DM detectors for *like*, *well* and similar items, running before the tagger, so that discourse uses are not mistagged as prepositions or adverbs.
- **I2** (learn): Report κ next to accuracy and compare against measured agreement between witness parsers or annotators, so we can tell when further training is pointless and labels need adjudication.
- **I3** (learn): Include one-feature or majority-class classifiers as negative controls in every evaluation of a new component.
- **I4** (learn): Mine such collocation cues from silver data and turn them into explicit, explainable repair rules, instead of opaque weights.
- **I5** (frames): Once a token is classified as a DM, use it as a cue for speech-act labels such as disagreement or hedging, and treat DM *like* as a marker of approximation or reported speech.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Geneva](https://md.research.elamur.ai/free/en/universities/university-of-geneva/) — University, Geneva, Switzerland · 29 works · 3,158 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 624 works · [🧵 Discourse analysis](https://md.research.elamur.ai/free/en/topics/discourse-analysis/) · 466 works

## Cite

Cite as: Andrei Popescu-Belis and Sandrine Zufferey. 2007. Contrasting the Automatic Identification of Two Discourse Markers in Multiparty Dialogues. Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{popescubelis2007contrasting,
  title = {{Contrasting the Automatic Identification of Two Discourse Markers in Multiparty Dialogues}},
  author = {Andrei Popescu-Belis and Sandrine Zufferey},
  booktitle = {Proceedings of the 8th SIGdial Workshop on Discourse and Dialogue},
  year = {2007},
  doi = {10.18653/v1/2007.sigdial-1.3},
  url = {https://doi.org/10.18653/v1/2007.sigdial-1.3}
}
```

- [← previous Arabic Speech Recognition System using CMU-Sphinx4](https://md.research.elamur.ai/free/en/papers/satori-2007-arabic-speech-recognition-system-using/)
- [next → JU-SKNSB: Extended WordNet Based WSD on the English All-Words Task at SemEval-1](https://md.research.elamur.ai/free/en/papers/naskar-2007-ju-sknsb-extended-wordnet-based/)
- [↑ back to papers of 2007](https://md.research.elamur.ai/free/en/papers/2007/page/2/)

## References

20 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Gisle Andersen. 2001. Pragmatic Markers of Sociolinguistic Variation. John Benjamins, Amsterdam. [find](https://scholar.google.com/scholar?q=Gisle+Andersen%2E+2001%2E+Pragmatic+Markers+of+Sociolinguistic+Variation%2E+John+Benjamins%2C+Amsterdam%2E)
2. Jean Carletta. 1996. [Assessing agreement on classification tasks: The kappa statistic](https://md.research.elamur.ai/free/en/papers/carletta-1996-assessing-agreement-classification-tasks-kappa/). Computational Linguistics, 22(2):249–254. [↗](https://doi.org/10.48550/arxiv.cmp-lg/9602004)
3. Barbara J. Grosz and Candace L. Sidner. 1986. Attentions, intentions and the structure of discourse. Computational Linguistics, 12(3):175–204. [find](https://scholar.google.com/scholar?q=Barbara+J%2E+Grosz+and+Candace+L%2E+Sidner%2E+1986%2E+Attentions%2C+intentions+and+the+structure+of+discourse%2E+Computational+Linguistics%2C+12%283%29%3A175%E2%80%93204%2E)
4. Peter A. Heeman and James F. Allen. 1999. Speech repairs, intonational phrases and discourse markers: Modeling speakers utterances in spoken dialogue. Computational Linguistics, 25(4):1–45. [find](https://scholar.google.com/scholar?q=Peter+A%2E+Heeman+and+James+F%2E+Allen%2E+1999%2E+Speech+repairs%2C+intonational+phrases+and+discourse+markers%3A+Modeling+speakers+utterances+in+spoken+dialogue%2E+Computational+Linguistics%2C+25%284%29%3A1%E2%80%9345%2E)
5. Julia Hirschberg and Diane Litman. 1993. [Empirical studies on the disambiguation of cue phrases](https://md.research.elamur.ai/free/en/papers/hirschberg-1993-empirical-studies-disambiguation-cue-phrases/). Computational Linguistics, 19(3):501–530.
6. Ben Hutchinson. 2004. [Acquiring the meaning of discourse markers](https://md.research.elamur.ai/free/en/papers/hutchinson-2004-acquiring-meaning-discourse-markers/). In Proceedings of ACL 2004 (42nd Annual Meeting of the Association for Computational Linguistics), pages 685–692, Barcelona, Spain.
7. Adam Janin, Don Baron, Jane A. Edwards, Dan Ellis, David Gelbart, Nelson Morgan, Barbara Peskin, Thilo Pfau, Elizabeth Shriberg, Andreas Stolcke, and Chuck Wooters. 2003. The ICSI Meeting Corpus. In Proceedings of ICASSP 2003 (IEEE International Conference on Acoustics, Speech, and Signal Processing), Hong Kong, China. [↗](https://doi.org/10.7916/d83b68g8)
8. J. Richard Landis and Gary G. Koch. 1977. The measurement of observer agreement for categorical data. Biometrics, 33(1):159–174. [↗](https://doi.org/10.2307/2529310)
9. Diane J. Litman. 1996. Cue phrase classification using machine learning. Journal of Artificial Intelligence Research, 5:53–94. [↗](https://doi.org/10.1613/jair.327)
10. Daniel Marcu. 1998. [A surface-based approach to identifying discourse markers and elementary textual units in unrestricted texts](https://md.research.elamur.ai/free/en/papers/marcu-1998-surface-based-approach-identifying-discourse/). In ACL/COLING-98 Workshop on Discourse Relations and Discourse Markers, pages 1–7, Montreal, Canada.
11. Rebecca J. Passonneau and Diane J. Litman. 1997. [Discourse segmentation by human and automated means](https://md.research.elamur.ai/free/en/papers/passonneau-1997-discourse-segmentation-human-automated-means/). Computational Linguistics, 23(1):103–140. [↗](https://doi.org/10.5555/972684.972689)
12. Andrei Popescu-Belis and Sandrine Zufferey. 2006. Automatic identification of discourse markers in multi-party dialogues. Working paper 65, ISSCO, University of Geneva, December 2006. [find](https://scholar.google.com/scholar?q=Andrei+Popescu%2DBelis+and+Sandrine+Zufferey%2E+2006%2E+Automatic+identification+of+discourse+markers+in+multi%2Dparty+dialogues%2E+Working+paper+65%2C+ISSCO%2C+University+of+Geneva%2C+December+2006%2E)
13. John R. Quinlan. 1993. [C4.5: Programs for Machine Learning](https://md.research.elamur.ai/free/en/papers/quinlan-1993-c45-programs-machine-learning/). Morgan Kaufman, San Francisco, CA.
14. Rachel Reichman. 1985. Getting Computers to Talk Like You and Me: Discourse Context, Focus, and Semantics (An ATN Model). MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Rachel+Reichman%2E+1985%2E+Getting+Computers+to+Talk+Like+You+and+Me%3A+Discourse+Context%2C+Focus%2C+and+Semantics+%28An+ATN+Model%29%2E+MIT+Press%2C+Cambridge%2C+MA%2E)
15. Deborah Schiffrin. 1987. Discourse Markers. Cambridge University Press, Cambridge, UK. [find](https://scholar.google.com/scholar?q=Deborah+Schiffrin%2E+1987%2E+Discourse+Markers%2E+Cambridge+University+Press%2C+Cambridge%2C+UK%2E)
16. Lawrence C. Schourup. 2001. Rethinking ‘well’. Journal of Pragmatics, 33:1025–1060. [find](https://scholar.google.com/scholar?q=Lawrence+C%2E+Schourup%2E+2001%2E+Rethinking+%E2%80%98well%E2%80%99%2E+Journal+of+Pragmatics%2C+33%3A1025%E2%80%931060%2E)
17. Elizabeth Shriberg, Raj Dhillon, Sonali Bhagat, Jeremy Ang, and Hannah Carvey. 2004. [The ICSI meeting recorder dialog act (MRDA) corpus](https://md.research.elamur.ai/free/en/papers/shriberg-2004-icsi-meeting-recorder-dialog-act/). In Proceedings of SIGdial 2004 (5th SIGdial Workshop on Discourse and Dialogue), pages 97–100, Cambridge, MA. [↗](https://doi.org/10.21236/ada460980)
18. Eric V. Siegel and Kathleen R. McKeown. 1994. Emergent linguistic rules from inducing decision trees: Disambiguating discourse clue words. In Proceedings of AAAI 1994 (12th National Conference on Artificial Intelligence), pages 820–826, Seattle, WA. [↗](https://doi.org/10.48550/arxiv.cmp-lg/9408007)
19. Iain Witten and Eibe Frank. 2000. Data Mining: Practical Machine Learning Tools with Java Implementations. Morgan Kaufmann, San Francisco, CA. [find](https://scholar.google.com/scholar?q=Iain+Witten+and+Eibe+Frank%2E+2000%2E+Data+Mining%3A+Practical+Machine+Learning+Tools+with+Java+Implementations%2E+Morgan+Kaufmann%2C+San+Francisco%2C+CA%2E)
20. Sandrine Zufferey and Andrei Popescu-Belis. 2004. [Towards automatic identification of discourse markers in dialogs: The case of like](https://md.research.elamur.ai/free/en/papers/zufferey-2004-automatic-identification-discourse-markers-dialogs/). In Proceedings of SIGdial 2004 (5th SIGdial Workshop on Discourse and Dialogue), pages 63–71, Cambridge, MA.

*Updated 1 November 2026*