---
title: "Towards Automatic Identification of Discourse Markers in"
description: "They then build automatic detectors from three kinds of evidence: collocation filters… 70% precision. 17 references, cited by 5 papers here."
authors: ["Sandrine Zufferey", "Andrei Popescu-Belis"]
year: 2004
venue: "Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue at HLT-NAACL 2004"
source: "https://aclanthology.org/W04-2313/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zufferey-2004-automatic-identification-discourse-markers-dialogs/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zufferey-2004-automatic-identification-discourse-markers-dialogs/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2004](https://md.research.elamur.ai/free/en/papers/2004/page/2/) / Towards Automatic Identification of Discourse Markers in Dialogs: The Case of Like

# Towards Automatic Identification of Discourse Markers in Dialogs: The Case of Like

**Authors:** [Sandrine Zufferey](https://md.research.elamur.ai/free/en/authors/sandrine-zufferey/) and [Andrei Popescu-Belis](https://md.research.elamur.ai/free/en/authors/andrei-popescu-belis/) · **Year:** 2004 · **Venue:** [Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue at HLT-NAACL 2004](https://md.research.elamur.ai/free/en/venues/sigdial/) ([SIGDIAL 2004](https://md.research.elamur.ai/free/en/venues/sigdial/2004/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W04-2313/) · [PDF](https://aclanthology.org/W04-2313.pdf) · [OpenAlex](https://openalex.org/W2265630291)

**Citations:** 17 citations (OpenAlex, 2026-10-14) · [5](https://md.research.elamur.ai/free/en/papers/zufferey-2004-automatic-identification-discourse-markers-dialogs/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses a first step in pragmatic dialog analysis: deciding when an ambiguous word is a discourse marker (DM) rather than an ordinary lexical item. The focus is *like*, which can also be a verb, preposition, conjunction, adjective, noun or adverb. As a DM, it mostly signals that what follows is a loose approximation. Using transcribed staff meetings from the ICSI Meeting Recording corpus, the authors first measure how well human judges agree, comparing transcripts alone with transcripts plus audio. They then build automatic detectors from three kinds of evidence: collocation filters, POS tags, and timing features such as pause and word duration. These are combined with C4.5 decision trees in WEKA. Humans reach reliable agreement, around κ ≈ 0.74, only when they can hear the recordings. Automatically, collocation filters dominate: about 70% precision at near-full recall for *like*, and about 91% precision for *well*. Off-the-shelf POS taggers fail on DMs. The results suggest high-recall filters can pre-screen data for annotators and support later parsing and tagging.

## Key points
- Data: ICSI-MR corpus, made of 75 one-hour meetings with up to eight speakers each. The authors use an initial release of 50 dialogs, segmented into roughly 65,000 prosodic utterances. In about 50 hours, 37% of 2,116 occurrences of *like* are DMs.
- Human annotation:
  - The first experiment used 6 judges (native English and French speakers) working from transcripts only.
  - The second used 3 judges who could also listen to the audio.
  - With sound, agreement reached κ ≈ 0.74.
- For *like*, a tree using only the collocation filters gets 0.96 recall and 0.67 precision (80% correct, κ = 0.62), close to the best tree. Without collocations, timing features alone give 67% correct with κ = 0.23.
- Timing cue: a pause longer than 240 ms before *like* marked a DM in 70 of 78 remaining cases. Very short tokens (under 120 ms) tended not to be DMs.
- For *well*, 579 of 873 occurrences are DMs (a 66% baseline). Collocations plus the pause after *well* give 91% accuracy, κ = 0.8, 91% precision and 97% recall.

## Interesting ideas
- Measure human agreement under different information conditions (text only vs. text plus audio) before building a classifier, so that the automatic score has a realistic ceiling.
- Use cheap lexical-context collocation filters that rule out non-DM readings (e.g. *as well*, *I like*) as a high-recall first pass, and leave the rest to finer analysis or humans.
- Also add positive filters that confirm a DM (e.g. *oh well*, *OK well*, a word ending an abandoned utterance), alongside the negative ones.
- Treat DM *like* as a loose-talk signal: what follows is an approximation, whether of a quantity or of a wording.
- Features that generalize across a whole class of markers, such as timing, are distinguished from item-specific collocations that win for individual words.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Evaluate pragmatic labels against agreement measured when annotators see only the text, the same input Mova has, rather than an idealized gold standard.
- **I2** (tokens): Add explainable pre-tagging rules that flag possible DM readings of *like*/*well*/*so*, so that the tagger does not force them into verb or preposition tags.
- **I3** (tree): Detect such markers before parsing and attach them as `discourse` dependents, so they do not distort the rest of the UD tree.
- **I4** (frames): When *like* precedes a number or an NP, mark the value or description as approximate, so the frame carries hedged rather than exact content.
- **I5** (world): Store hedged quantities from *like twenty minutes* as ranges, not point values, so that later reasoning about measures and event durations stays honest.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Geneva](https://md.research.elamur.ai/free/en/universities/university-of-geneva/) — University, Geneva, Switzerland · 29 works · 3,158 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 624 works · [🧵 Discourse analysis](https://md.research.elamur.ai/free/en/topics/discourse-analysis/) · 466 works

## Cite

Cite as: Sandrine Zufferey and Andrei Popescu-Belis. 2004. Towards Automatic Identification of Discourse Markers in Dialogs: The Case of Like. Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue at HLT-NAACL 2004.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zufferey2004automatic,
  title = {{Towards Automatic Identification of Discourse Markers in Dialogs: The Case of Like}},
  author = {Sandrine Zufferey and Andrei Popescu-Belis},
  booktitle = {Proceedings of the 5th SIGdial Workshop on Discourse and Dialogue at HLT-NAACL 2004},
  year = {2004},
  doi = {10.7892/boris.78686},
  url = {https://doi.org/10.7892/boris.78686}
}
```

- [← previous Parsing Ungrammatical Input: An Evaluation Procedure](https://md.research.elamur.ai/free/en/papers/foster-2004-parsing-ungrammatical-input-evaluation-procedure/)
- [next → An Unsupervised Approach for Bootstrapping Arabic Sense Tagging](https://md.research.elamur.ai/free/en/papers/diab-2004-unsupervised-approach-bootstrapping-arabic-sense/)
- [↑ back to papers of 2004](https://md.research.elamur.ai/free/en/papers/2004/page/2/)

## References

17 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aijmer, K. English Discourse Particles: Evidence from a Corpus. Amsterdam: John Benjamins, 2002. [find](https://scholar.google.com/scholar?q=Aijmer%2C+K%2E+English+Discourse+Particles%3A+Evidence+from+a+Corpus%2E+Amsterdam%3A+John+Benjamins%2C+2002%2E)
2. Andersen, G. Pragmatic Markers of Sociolinguistic Variation: a Relevance-Theoretic Approach to the Language of Adolescents. Amsterdam: John Benjamins, 2001. [find](https://scholar.google.com/scholar?q=Andersen%2C+G%2E+Pragmatic+Markers+of+Sociolinguistic+Variation%3A+a+Relevance%2DTheoretic+Approach+to+the+Language+of+Adolescents%2E+Amsterdam%3A+John+Benjamins%2C+2001%2E)
3. Blakemore, D. Meaning and Relevance: the Semantics and Pragmatics of Discourse Markers. Cambridge: CUP, 2002. [find](https://scholar.google.com/scholar?q=Blakemore%2C+D%2E+Meaning+and+Relevance%3A+the+Semantics+and+Pragmatics+of+Discourse+Markers%2E+Cambridge%3A+CUP%2C+2002%2E)
4. Fraser, B. An Approach to Discourse Markers. Journal of Pragmatics. 1990, vol.14, pp. 383-395. [find](https://scholar.google.com/scholar?q=Fraser%2C+B%2E+An+Approach+to+Discourse+Markers%2E+Journal+of+Pragmatics%2E+1990%2C+vol%2E14%2C+pp%2E+383%2D395%2E)
5. Heeman, P., Byron, D., Allen, J. [Identifying Discourse Markers in Spoken Dialog](https://md.research.elamur.ai/free/en/papers/heeman-1998-identifying-discourse-markers-spoken-dialog/). Proceedings of AAAI Spring Symposium on Applying Machine Learning and Discourse Processing. Stanford, 1998.
6. Lenk, U. Marking Discourse Coherence: Functions of Discourse Markers in Spoken English. Tübingen: Gunter Narr Verlag, 1998. [find](https://scholar.google.com/scholar?q=Lenk%2C+U%2E+Marking+Discourse+Coherence%3A+Functions+of+Discourse+Markers+in+Spoken+English%2E+T%C3%BCbingen%3A+Gunter+Narr+Verlag%2C+1998%2E)
7. Litman, D. Cue Phrase Classification Using Machine Learning. Journal of Artificial Intelligence Research. 1996, vol.5, pp. 53-94. [find](https://scholar.google.com/scholar?q=Litman%2C+D%2E+Cue+Phrase+Classification+Using+Machine+Learning%2E+Journal+of+Artificial+Intelligence+Research%2E+1996%2C+vol%2E5%2C+pp%2E+53%2D94%2E)
8. Mann, W., Thompson, S. Rhetorical Structure Theory: Toward a Functional Theory of Text Organisation. Text. 1988, vol.8(3), pp. 243-281. [find](https://scholar.google.com/scholar?q=Mann%2C+W%2E%2C+Thompson%2C+S%2E+Rhetorical+Structure+Theory%3A+Toward+a+Functional+Theory+of+Text+Organisation%2E+Text%2E+1988%2C+vol%2E8%283%29%2C+pp%2E+243%2D281%2E)
9. Marcu, D. The Theory and Practice of Discourse Parsing and Summarization. Cambridge: MIT Press, 2000. [find](https://scholar.google.com/scholar?q=Marcu%2C+D%2E+The+Theory+and+Practice+of+Discourse+Parsing+and+Summarization%2E+Cambridge%3A+MIT+Press%2C+2000%2E)
10. Mason, O. Programming for Corpus Linguistics: How to do Text Analysis in Java. Edinburgh: Edinburgh University Press, 2000. [find](https://scholar.google.com/scholar?q=Mason%2C+O%2E+Programming+for+Corpus+Linguistics%3A+How+to+do+Text+Analysis+in+Java%2E+Edinburgh%3A+Edinburgh+University+Press%2C+2000%2E)
11. Meteer, M. Dysfluency Annotation Stylebook for the Switchboard Corpus. LDC, Working Paper. 1995, 28p. [http://www.ldc.upenn.edu/Catalog/CatalogList/LDC99T42/DFLGUIDE.PS](http://www.ldc.upenn.edu/Catalog/CatalogList/LDC99T42/DFLGUIDE.PS) (06/17/2003). [↗](http://www.ldc.upenn.edu/Catalog/CatalogList/LDC99T42/DFLGUIDE.PS)
12. Quinlan, J. R. [C4.5: Programs for Machine Learning](https://md.research.elamur.ai/free/en/papers/quinlan-1993-c45-programs-machine-learning/). San Francisco: Morgan Kaufman, 1993.
13. Schiffrin, D. Discourse Markers. Cambridge: CUP, 1987. [find](https://scholar.google.com/scholar?q=Schiffrin%2C+D%2E+Discourse+Markers%2E+Cambridge%3A+CUP%2C+1987%2E)
14. Sperber, D., Wilson, D. Relevance: Communication and Cognition. Oxford: Blackwell, 1986/1995. [find](https://scholar.google.com/scholar?q=Sperber%2C+D%2E%2C+Wilson%2C+D%2E+Relevance%3A+Communication+and+Cognition%2E+Oxford%3A+Blackwell%2C+1986%2F1995%2E)
15. Taboada, M. Discourse Markers as Signals (or not) of Rhetorical Relations in Conversation. Proceedings of the 8th International Pragmatics Conference. Toronto, 2003. [find](https://scholar.google.com/scholar?q=Taboada%2C+M%2E+Discourse+Markers+as+Signals+%28or+not%29+of+Rhetorical+Relations+in+Conversation%2E+Proceedings+of+the+8th+International+Pragmatics+Conference%2E+Toronto%2C+2003%2E)
16. Witten, I., Frank, E. Data Mining: Practical Machine Learning Tools with Java Implementations. San Francisco: Morgan Kaufmann, 2000. [find](https://scholar.google.com/scholar?q=Witten%2C+I%2E%2C+Frank%2C+E%2E+Data+Mining%3A+Practical+Machine+Learning+Tools+with+Java+Implementations%2E+San+Francisco%3A+Morgan+Kaufmann%2C+2000%2E)
17. Zufferey, S. Une Analyse des Connecteurs Pragmatiques Fondée sur la Théorie de la Pertinence et son Application au TALN. Cahiers de linguistique française. 2004, vol.25, pp. 257-272. [find](https://scholar.google.com/scholar?q=Zufferey%2C+S%2E+Une+Analyse+des+Connecteurs+Pragmatiques+Fond%C3%A9e+sur+la+Th%C3%A9orie+de+la+Pertinence+et+son+Application+au+TALN%2E+Cahiers+de+linguistique+fran%C3%A7aise%2E+2004%2C+vol%2E25%2C+pp%2E+257%2D272%2E)

*Updated 29 October 2026*