---
title: "Identifying Agreement and Disagreement in — Galley 2004"
description: "The paper addresses automatic detection of agreement and disagreement between participants in multi-party meetings, which the authors see as a step…"
authors: ["Michel Galley", "Kathleen McKeown", "Julia Hirschberg", "Elizabeth Shriberg"]
year: 2004
venue: "Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)"
source: "https://aclanthology.org/P04-1085/"
doi: "10.3115/1218955.1219040"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/galley-2004-identifying-agreement-disagreement-conversational-speech/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/galley-2004-identifying-agreement-disagreement-conversational-speech/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2004](https://md.research.elamur.ai/free/papers/2004/) / Identifying Agreement and Disagreement in Conversational Speech: Use of Bayesian Networks to Model Pragmatic Dependencies

# Identifying Agreement and Disagreement in Conversational Speech: Use of Bayesian Networks to Model Pragmatic Dependencies

**Authors:** [Michel Galley](https://md.research.elamur.ai/free/authors/michel-galley/), [Kathleen McKeown](https://md.research.elamur.ai/free/authors/kathleen-mckeown/), [Julia Hirschberg](https://md.research.elamur.ai/free/authors/julia-hirschberg/), [Elizabeth Shriberg](https://md.research.elamur.ai/free/authors/elizabeth-shriberg/) · **Year:** 2004 · **Venue:** Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P04-1085/) · [PDF](https://aclanthology.org/P04-1085.pdf) · [DOI 10.3115/1218955.1219040](https://doi.org/10.3115/1218955.1219040) · [OpenAlex](https://openalex.org/W2102800123)

**Citations:** 219 citations (openalex-snapshot · 2026-10-14) · [59](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
The paper addresses automatic detection of agreement and disagreement between participants in multi-party meetings, which the authors see as a step toward meeting summarization. The work uses the ICSI Meeting corpus. Speech is split into pause-delimited spurts, and each spurt is labeled agreement, disagreement, backchannel, or other.

The method has two stages:
1. **Finding addressees.** A maximum entropy ranker identifies adjacency pairs, which tell the system who is responding to whom. It ranks candidate speakers using structural, durational, and lexical cues drawn from both preceding and following context.
2. **Labeling spurts.** A sequential classifier, structured as a Bayesian network, assigns the labels. Beyond local features, it encodes pragmatic dependencies on earlier labels, for instance whether a speaker who previously disagreed with someone tends to do so again.

On three-way classification, the system reaches 86.9% accuracy, compared with 82% for Hillard et al. (2003). Adding the label-dependency features contributes about 1.3 points of that gain. The results suggest that modeling conversational context and addressee structure improves pragmatic tagging in multi-party dialog.

## Key points
- **Data:** The ICSI Meeting corpus has 75 meetings averaging 6.5 participants. Seven meetings were annotated at the spurt level, giving 8135 spurts. The class split is 11.9% agreement, 6.8% disagreement, 23.2% backchannel, and 58.1% other. Inter-annotator kappa was 0.63.
- **Adjacency-pair identification:** Simply choosing the most recent prior speaker is correct 79.8% of the time. The maxent ranker, trained with GIS in YASMET, reaches 89.38%. Adding Gaussian smoothing and feature selection raises this to 90.2%, and dialog-act tags bring it to 91.34%.
- **Forward-looking features:** Dropping them lowers ranking accuracy to 86.99%.
- **Contextual dependencies:** Four label-dependency features are modeled: the previous tag, the previous tag between the same interactants, reflexivity, and transitivity. Decoding uses beam search with a beam size of 100.
- **Four-way cross-validation:** Accuracy is 84.07% with label dependencies versus 83.11% without them. Lexical features are the strongest of the local feature sets.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cited by

Cited by

59 papers of our library cite it (of the 140673 whose bibliographies we read); 2 of them have a page here, 57 not yet.

- [Detecting Disagreement in Conversations using Pseudo-Monologic Rhetorical Structure](https://md.research.elamur.ai/free/papers/allen-2014-detecting-disagreement-conversations-pseudo-monologic/)
- [A Corpus for Research on Deliberation and Debate](https://md.research.elamur.ai/free/papers/walker-2012-corpus-research-deliberation-debate/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{galley2004identifying,
  title = {{Identifying Agreement and Disagreement in Conversational Speech: Use of Bayesian Networks to Model Pragmatic Dependencies}},
  author = {Galley, Michel and McKeown, Kathleen and Hirschberg, Julia and Shriberg, Elizabeth},
  booktitle = {Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)},
  year = {2004},
  doi = {10.3115/1218955.1219040},
  url = {https://doi.org/10.3115/1218955.1219040}
}
```

- [← previous From information structure, topic and focus, to theme in Biblical Hebrew](https://md.research.elamur.ai/free/papers/hbo-2004-floor-information-structure-topic-focus-theme/)
- [next → Identifying Broken Plurals in Unvowelised Arabic Text](https://md.research.elamur.ai/free/papers/aclbulk-2004-goweder-identifying-broken-plurals-unvowelised-arabic/)
- [↑ back to papers of 2004](https://md.research.elamur.ai/free/papers/2004/)

## References

19 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. A. Berger, S. Della Pietra, and V Della Pietra. 1996. A maximum entropy approach to natural language processing. Computational Linguistics, 22(1):39–72. [find](https://scholar.google.com/scholar?q=A%2E+Berger%2C+S%2E+Della+Pietra%2C+and+V+Della+Pietra%2E+1996%2E+A+maximum+entropy+approach+to+natural+language+processing%2E+Computational+Linguistics%2C+22%281%29%3A39%E2%80%9372%2E) [our card](https://md.research.elamur.ai/free/papers/berger-1996-maximum-entropy-approach-natural-language/)
2. J. Cohen. 1960. A coefficient of agreement for nominal scales. Educational and Psychological measurements, 20:37–46. [find](https://scholar.google.com/scholar?q=J%2E+Cohen%2E+1960%2E+A+coefficient+of+agreement+for+nominal+scales%2E+Educational+and+Psychological+measurements%2C+20%3A37%E2%80%9346%2E)
3. S. Cohen. 2002. A computerized scale for monitoring levels of agreement during a conversation. In Proc. of the 26th Penn Linguistics Colloquium. [find](https://scholar.google.com/scholar?q=S%2E+Cohen%2E+2002%2E+A+computerized+scale+for+monitoring+levels+of+agreement+during+a+conversation%2E+In+Proc%2E+of+the+26th+Penn+Linguistics+Colloquium%2E)
4. M. Collins. 2000. Discriminative reranking for natural language parsing. In Proc. 17th International Conf. on Machine Learning, pages 175– 182. [find](https://scholar.google.com/scholar?q=M%2E+Collins%2E+2000%2E+Discriminative+reranking+for+natural+language+parsing%2E+In+Proc%2E+17th+International+Conf%2E+on+Machine+Learning%2C+pages+175%E2%80%93+182%2E) [our card](https://md.research.elamur.ai/free/papers/collins-2000-discriminative-reranking-natural-language-parsing/)
5. J. N. Darroch and D. Ratcliff. 1972. Generalized iterative scaling for log-linear models. Annals of Mathematical Statistics, 43:1470–1480. [find](https://scholar.google.com/scholar?q=J%2E+N%2E+Darroch+and+D%2E+Ratcliff%2E+1972%2E+Generalized+iterative+scaling+for+log%2Dlinear+models%2E+Annals+of+Mathematical+Statistics%2C+43%3A1470%E2%80%931480%2E)
6. R. Dhillon, S. Bhagat, H. Carvey, and E. Shriberg. 2004. Meeting recorder project: Dialog act labeling guide. Technical Report TR-04-002, ICSI. [find](https://scholar.google.com/scholar?q=R%2E+Dhillon%2C+S%2E+Bhagat%2C+H%2E+Carvey%2C+and+E%2E+Shriberg%2E+2004%2E+Meeting+recorder+project%3A+Dialog+act+labeling+guide%2E+Technical+Report+TR%2D04%2D002%2C+ICSI%2E)
7. V. Hatzivassiloglou and K. McKeown. 1997. Predicting the semantic orientation of adjectives. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=V%2E+Hatzivassiloglou+and+K%2E+McKeown%2E+1997%2E+Predicting+the+semantic+orientation+of+adjectives%2E+In+Proc%2E+of+ACL%2E)
8. D. Hillard, M. Ostendorf, and E Shriberg. 2003. Detection of agreement vs. disagreement in meetings: training with unlabeled data. In Proc. of HLT/NAACL. [find](https://scholar.google.com/scholar?q=D%2E+Hillard%2C+M%2E+Ostendorf%2C+and+E+Shriberg%2E+2003%2E+Detection+of+agreement+vs%2E+disagreement+in+meetings%3A+training+with+unlabeled+data%2E+In+Proc%2E+of+HLT%2FNAACL%2E)
9. J. Hirschberg and D. Litman. 1994. Empirical studies on the disambiguation of cue phrases. Computational Linguistics, 19(3):501–530. [find](https://scholar.google.com/scholar?q=J%2E+Hirschberg+and+D%2E+Litman%2E+1994%2E+Empirical+studies+on+the+disambiguation+of+cue+phrases%2E+Computational+Linguistics%2C+19%283%29%3A501%E2%80%93530%2E)
10. A. Janin, D. Baron, J. Edwards, D. Ellis, D. Gelbart, N. Morgan, B. Peskin, T. Pfau, E. Shriberg, A. Stolcke, and C. Wooters. 2003. The ICSI meeting corpus. In Proc. of ICASSP-03, Hong Kong. [find](https://scholar.google.com/scholar?q=A%2E+Janin%2C+D%2E+Baron%2C+J%2E+Edwards%2C+D%2E+Ellis%2C+D%2E+Gelbart%2C+N%2E+Morgan%2C+B%2E+Peskin%2C+T%2E+Pfau%2C+E%2E+Shriberg%2C+A%2E+Stolcke%2C+and+C%2E+Wooters%2E+2003%2E+The+ICSI+meeting+corpus%2E+In+Proc%2E+of+ICASSP%2D03%2C+Hong+Kong%2E)
11. D. Klein and C. D. Manning. 2002. Conditional structure versus conditional estimation in NLP models. Technical report. [find](https://scholar.google.com/scholar?q=D%2E+Klein+and+C%2E+D%2E+Manning%2E+2002%2E+Conditional+structure+versus+conditional+estimation+in+NLP+models%2E+Technical+report%2E)
12. S. Levinson. 1983. Pragmatics. Cambridge University Press. [find](https://scholar.google.com/scholar?q=S%2E+Levinson%2E+1983%2E+Pragmatics%2E+Cambridge+University+Press%2E)
13. A. McCallum, D. Freitag, and F. Pereira. 2000. Maximum entropy markov models for information extraction and segmentation. In Proc. of ICML. [find](https://scholar.google.com/scholar?q=A%2E+McCallum%2C+D%2E+Freitag%2C+and+F%2E+Pereira%2E+2000%2E+Maximum+entropy+markov+models+for+information+extraction+and+segmentation%2E+In+Proc%2E+of+ICML%2E)
14. A. Pomerantz. 1984. Agreeing and disagreeing with assessments: some features of preferred/dispreferred turn shapes. In J.M. Atkinson and J.C. Heritage, editors, Structures of Social Action, pages 57–101. [find](https://scholar.google.com/scholar?q=A%2E+Pomerantz%2E+1984%2E+Agreeing+and+disagreeing+with+assessments%3A+some+features+of+preferred%2Fdispreferred+turn+shapes%2E+In+J%2EM%2E+Atkinson+and+J%2EC%2E+Heritage%2C+editors%2C+Structures+of+Social+Action%2C+pages+57%E2%80%931)
15. A. Ratnaparkhi. 1996. A maximum entropy part-of-speech tagger. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=A%2E+Ratnaparkhi%2E+1996%2E+A+maximum+entropy+part%2Dof%2Dspeech+tagger%2E+In+Proc%2E+of+EMNLP%2E)
16. D. Ravichandran, E. Hovy, and F. J. Och. 2003. Statistical QA - classifier vs re-ranker: What’s the difference? In Proc. of the ACL Workshop on Multilingual Summarization and Question Answering. [find](https://scholar.google.com/scholar?q=D%2E+Ravichandran%2C+E%2E+Hovy%2C+and+F%2E+J%2E+Och%2E+2003%2E+Statistical+QA+%2D+classifier+vs+re%2Dranker%3A+What%E2%80%99s+the+difference%3F+In+Proc%2E+of+the+ACL+Workshop+on+Multilingual+Summarization+and+Question+Answering%2E)
17. E. A. Schegloff and H Sacks. 1973. Opening up closings. Semiotica, 7-4:289–327. [find](https://scholar.google.com/scholar?q=E%2E+A%2E+Schegloff+and+H+Sacks%2E+1973%2E+Opening+up+closings%2E+Semiotica%2C+7%2D4%3A289%E2%80%93327%2E)
18. E. Shriberg, R. Dhillon, S. Bhagat, J. Ang, and H. Carvey. 2004. The ICSI meeting recorder dialog act (MRDA) corpus. In SIGdial Workshop on Discourse and Dialogue, pages 97–100. [find](https://scholar.google.com/scholar?q=E%2E+Shriberg%2C+R%2E+Dhillon%2C+S%2E+Bhagat%2C+J%2E+Ang%2C+and+H%2E+Carvey%2E+2004%2E+The+ICSI+meeting+recorder+dialog+act+%28MRDA%29+corpus%2E+In+SIGdial+Workshop+on+Discourse+and+Dialogue%2C+pages+97%E2%80%93100%2E) [our card](https://md.research.elamur.ai/free/papers/shriberg-2004-icsi-meeting-recorder-dialog-act/)
19. A. Stolcke, K. Ries, N. Coccaro, E. Shriberg, R. Bates, D. Jurafsky, P. Taylor, R. Martin, C. Van Ess-Dykema, and M. Meteer. 2000. Dialogue act modeling for automatic tagging and recognition of conversational speech. Computational Linguistics, 26(3):339–373. [find](https://scholar.google.com/scholar?q=A%2E+Stolcke%2C+K%2E+Ries%2C+N%2E+Coccaro%2C+E%2E+Shriberg%2C+R%2E+Bates%2C+D%2E+Jurafsky%2C+P%2E+Taylor%2C+R%2E+Martin%2C+C%2E+Van+Ess%2DDykema%2C+and+M%2E+Meteer%2E+2000%2E+Dialogue+act+modeling+for+automatic+tagging+and+recognition+of+conver) [our card](https://md.research.elamur.ai/free/papers/prag-2000-stolcke-dialogue-act-modeling-automatic-tagging/)