---
title: "A CUP of CoFee — paper by Prévot et al. (2016)"
description: "This paper presents a large French resource from the CoFee project. 22 references, cited by 4 papers here."
authors: ["Laurent Prévot", "Jan Gorisch", "Roxane Bertrand"]
year: 2016
venue: "Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)"
source: "https://aclanthology.org/L16-1507/"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/prevot-2016-cup-cofee-large-collection-feedback/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/prevot-2016-cup-cofee-large-collection-feedback/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/4/) / A CUP of CoFee: A large Collection of feedback Utterances Provided with communicative function annotations

# A CUP of CoFee: A large Collection of feedback Utterances Provided with communicative function annotations

**Authors:** [Laurent Prévot](https://md.research.elamur.ai/free/en/authors/laurent-prevot/), Jan Gorisch and Roxane Bertrand · **Year:** 2016 · **Venue:** [Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)](https://md.research.elamur.ai/free/en/venues/lrec/) ([LREC 2016](https://md.research.elamur.ai/free/en/venues/lrec/2016/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/L16-1507/) · [PDF](https://aclanthology.org/L16-1507.pdf) · [OpenAlex](https://openalex.org/W2572372148)

**Citations:** 7 citations (OpenAlex, 2026-10-14) · 4 in our library, of the 141,840 papers whose bibliographies we read

## Summary
Short reactions such as "yeah", "mh" or a nod are among the most frequent dialogue moves. Labelling what they do in conversation is hard, and earlier schemes often never checked whether annotators agree. This paper presents a large French resource from the CoFee project. Its purpose is to support later statistical and machine-learning studies of how the form of a feedback signal relates to its function. The authors used four corpora: free conversation, a remote MapTask, a face-to-face MapTask and DVD negotiation dialogues. From these they gathered roughly 13,000 candidate verbal and gestural feedback items. Verbal candidates came from short Inter-Pausal Units. Gestural candidates came from a recall-oriented video pre-segmentation. They designed a schema with four dimensions: base, attitude, previous and visual. Twelve student raters then labelled about 9,500 instances. Agreement on the base dimension was good, while the more peripheral dimensions were much weaker. The average best-pair kappa was about 0.6. The annotations come with automatically extracted lexical, positional and acoustic features and are freely released. This makes the resource a reusable testbed for feedback classification.

## Key points
- Source material comes from four corpora:
  - CID: about 8 h of conversation.
  - MTR: a 2.5 h remote MapTask.
  - MTX: a 2.5 h face-to-face MapTask with video.
  - DVD: about 4 h of negotiation recordings.
  
  All are hosted on Ortolang-SLDR and are aligned at word and phone level.
- Verbal candidates were found by looking at token frequencies in IPUs of three tokens or fewer. The most frequent forms were ouais (2781), mh (2321), d'accord (1082), laughter (920) and oui (888). The authors excluded tu, et and alors as unrelated to feedback.
- About 9,500 of the ~13,000 candidates were annotated by 12 students. Each item received 2 to 7 raters.
- Five labels are available in the base dimension (acknowledgement, answer, evaluation, elicit, plus a catch-all "other"), and annotators agree well on it. The attitude, previous and visual dimensions agree poorly. The overall average best-pair κ is around 0.6.
- Video drifted out of sync with audio because of clock differences and dropped frames. CID was fixed by hand. MTX and DVD were realigned with a dedicated synchronisation method. A future release in the ISO-DiaML (ISO 24617-2) format is planned.

## Interesting ideas
- Candidate feedback items were found from the form side: the authors counted which tokens dominate very short isolated units, then removed by hand the frequent words that have no feedback role. This is a cheap, transparent way to bootstrap a phenomenon-specific dataset.
- The gesture pass was deliberately tuned for recall, with broad instructions, and a later precise annotation step was allowed to throw out false hits. This splits a hard labelling job into a permissive sweep and a strict filter.
- The schema keeps a stable core dimension (base) apart from the harder interpretive layers (attitude, previous), and measures agreement for each layer separately. This shows clearly which parts of the analysis can be trusted.
- Agreement is reported as the best rater pair for each category and each corpus, even though the number of raters varies per item. This exposes where disagreement is concentrated instead of hiding it in one global score.
- One schema was tested on deliberately different settings (chat, map task, negotiation) to check whether it generalises across interaction types.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): Count the tokens in short standalone dialogue turns of tales and fables ("yes", "okay", "right", "oh") to build a small inventory of feedback markers, then give each one explicit speech-act candidates such as acknowledgement or answer.
- **I2** (learn): Let a cheap high-recall pass propose silver candidates, then require agreement from several witness parsers to discard the bad ones, instead of demanding precision at the first step.
- **I3** (frames): Store a speech act as a reliable core function plus optional attitude or irony layers, each with its own confidence, so that later reasoning leans on the core and treats the periphery as tentative.
- **I4** (learn): Report witness-parser agreement broken down by dependency relation and by text genre, so it is visible which labels are good enough for silver data.
- **I5** (answer): Evaluate the readers on several genres side by side (school tests, word problems, tales) and compare the per-genre scores, to spot components that only work in one setting.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Aix-Marseille Université](https://md.research.elamur.ai/free/en/universities/aix-marseille-universite/) — University, Marseille, France · 14 works · 38,957 citations
- [Laboratoire Parole et Langage](https://md.research.elamur.ai/free/en/labs/laboratoire-parole-et-langage/) — Research institute, Aix-en-Provence, France · 5 works · 61 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: Laurent Prévot, Jan Gorisch and Roxane Bertrand. 2016. A CUP of CoFee: A large Collection of feedback Utterances Provided with communicative function annotations. Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{prevot2016cup,
  title = {{A CUP of CoFee: A large Collection of feedback Utterances Provided with communicative function annotations}},
  author = {Laurent Prévot, Jan Gorisch and Roxane Bertrand},
  booktitle = {Proceedings of the Tenth International Conference on Language Resources and Evaluation (LREC'16)},
  year = {2016},
  doi = {10.63317/2wdsku8hqi2s},
  url = {https://doi.org/10.63317/2wdsku8hqi2s}
}
```

- [← previous Korean FrameNet Expansion Based on Projection of Japanese FrameNet](https://md.research.elamur.ai/free/en/papers/kim-2016-korean-framenet-expansion-based-projection/)
- [next → Abstract Coreference in a Multilingual Perspective: a View on Czech and German](https://md.research.elamur.ai/free/en/papers/nedoluzhko-2016-abstract-coreference-multilingual-perspective-view/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/4/)

## References

22 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Allwood, J., Nivre, J., and Ahlsen, E. (1992). On the semantics and pragmatics of linguistic feedback. Journal of Semantics, 9:1–26. [↗](https://doi.org/10.1093/jos/9.1.1)
2. Allwood, J., Cerrato, L., Jokinen, K., Navarretta, C., and Paggio, P. (2007). The MUMIN coding scheme for the annotation of feedback, turn management and sequenc-(a) Best pair of raters on CID (c) Best pair of raters on MTR Figure 3: κ for the best rater pair for all corpora (approbation = approval ; attente = expectation ; precedent = previous) ing phenomena. Language Resources and Evaluation, 41(3):273–287. [find](https://scholar.google.com/scholar?q=Allwood%2C+J%2E%2C+Cerrato%2C+L%2E%2C+Jokinen%2C+K%2E%2C+Navarretta%2C+C%2E%2C+and+Paggio%2C+P%2E+%282007%29%2E+The+MUMIN+coding+scheme+for+the+annotation+of+feedback%2C+turn+management+and+sequenc%2D%28a%29+Best+pair+of+raters+on+CID+%28c%29+Bes)
3. Anderson, A., Bader, M., Bard, E., Boyle, E., Doherty, G. M., Garrod, S., Isard, S., Kowtko, J., McAllister, J., Miller, J., Sotillo, C., Thompson, H. S., and Weinert, R. (1991). The HCRC map task corpus. Language and Speech, 34:351–366. [↗](https://doi.org/10.1177/002383099103400404)
4. Bard, E. G., Astésano, C., D’Imperio, M., Turk, A., Nguyen, N., Prévot, L., and Bigi, B. (2013). Aix Map-Task: A new French resource for prosodic and discourse studies. In Proceedings of Tools and Resources for the Analysis of Speech Prosody (TRASP), Aix-en-Provence, France. [find](https://scholar.google.com/scholar?q=Bard%2C+E%2E+G%2E%2C+Aste%CC%81sano%2C+C%2E%2C+D%E2%80%99Imperio%2C+M%2E%2C+Turk%2C+A%2E%2C+Nguyen%2C+N%2E%2C+Pre%CC%81vot%2C+L%2E%2C+and+Bigi%2C+B%2E+%282013%29%2E+Aix+Map%2DTask%3A+A+new+French+resource+for+prosodic+and+discourse+studies%2E+In+Proceedings+of+Tools+and+R)
5. Bertrand, R., Blache, P., Espesser, R., Ferré, G., Meunier, C., Priego-Valverde, B., and Rauzy, S. (2008). Le CID-Corpus of interactional data-annotation et exploitation multimodale de parole conversationnelle. Traitement Automatique des Langues, 49(3):1–30. [find](https://scholar.google.com/scholar?q=Bertrand%2C+R%2E%2C+Blache%2C+P%2E%2C+Espesser%2C+R%2E%2C+Ferre%CC%81%2C+G%2E%2C+Meunier%2C+C%2E%2C+Priego%2DValverde%2C+B%2E%2C+and+Rauzy%2C+S%2E+%282008%29%2E+Le+CID%2DCorpus+of+interactional+data%2Dannotation+et+exploitation+multimodale+de+parole+convers)
6. Bunt, H., Alexandersson, J., Choe, J.-W., Fang, A. C., Hasida, K., Petukhova, V., Popescu-Belis, A., and Traum, D. R. (2012). [ISO 24617-2: A semantically-based standard for dialogue annotation](https://md.research.elamur.ai/free/en/papers/bunt-2012-iso-24617-2-semantically-based/). In Proceedings of the Eighth International Conference on Language Re-(b) Best pair of raters on DVD (d) Best pair of raters on MTX sources and Evaluation (LREC’12), pages 430–437, Istanbul, Turkey.
7. Bunt, H. (1994). Context and dialogue control. Think Quarterly, 3(1):19–31. [find](https://scholar.google.com/scholar?q=Bunt%2C+H%2E+%281994%29%2E+Context+and+dialogue+control%2E+Think+Quarterly%2C+3%281%29%3A19%E2%80%9331%2E)
8. Gorisch, J. and Prévot, L. (2015). Audio synchronisation with a tunnel matrix for time series and dynamic programming. In Proceedings of ICASSP 2015 IEEE International Conference on Acoustics, Speech, and Signal Processing, pages 3846–3850, Brisbane, Australia. [find](https://scholar.google.com/scholar?q=Gorisch%2C+J%2E+and+Pre%CC%81vot%2C+L%2E+%282015%29%2E+Audio+synchronisation+with+a+tunnel+matrix+for+time+series+and+dynamic+programming%2E+In+Proceedings+of+ICASSP+2015+IEEE+International+Conference+on+Acoustics%2C+Speech)
9. Gorisch, J., Astésano, C., Bard, E., Bigi, B., and Prévot, L. (2014). Aix Map Task corpus: The French multimodal corpus of task-oriented dialogue. In Proceedings of The Ninth International Conference on Language Resources and Evaluation (LREC’14), Reykjavik, Iceland. [find](https://scholar.google.com/scholar?q=Gorisch%2C+J%2E%2C+Aste%CC%81sano%2C+C%2E%2C+Bard%2C+E%2E%2C+Bigi%2C+B%2E%2C+and+Pre%CC%81vot%2C+L%2E+%282014%29%2E+Aix+Map+Task+corpus%3A+The+French+multimodal+corpus+of+task%2Doriented+dialogue%2E+In+Proceedings+of+The+Ninth+International+Conferenc)
10. Gravano, A., Hirschberg, J., and Beňuš, Š. (2012). Affirmative cue words in task-oriented dialogue. Computational Linguistics, 38(1):1–39. [↗](https://doi.org/10.1162/coli_a_00083)
11. Neiberg, D., Salvi, G., and Gustafson, J. (2013). Semi-supervised methods for exploring the acoustics of simple productive feedback. Speech Communication, 55:451– 469. [↗](https://doi.org/10.1016/j.specom.2012.12.007)
12. Prévot, L. and Bertrand, R. (2012). Cofee-toward a multidimensional analysis of conversational feedback, the case of french language. In Proceedings of the Workshop on Feedback Behaviors. (poster). [find](https://scholar.google.com/scholar?q=Pre%CC%81vot%2C+L%2E+and+Bertrand%2C+R%2E+%282012%29%2E+Cofee%2Dtoward+a+multidimensional+analysis+of+conversational+feedback%2C+the+case+of+french+language%2E+In+Proceedings+of+the+Workshop+on+Feedback+Behaviors%2E+%28poster%29%2E)
13. Prévot, L., Gorisch, J., Bertrand, R., Gorene, E., and Bigi, B. (2015a). A SIP of CoFee: A Sample of Interesting Productions of Conversational Feedback. In 16th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGdial), pages 149–153. [find](https://scholar.google.com/scholar?q=Pre%CC%81vot%2C+L%2E%2C+Gorisch%2C+J%2E%2C+Bertrand%2C+R%2E%2C+Gorene%2C+E%2E%2C+and+Bigi%2C+B%2E+%282015a%29%2E+A+SIP+of+CoFee%3A+A+Sample+of+Interesting+Productions+of+Conversational+Feedback%2E+In+16th+Annual+Meeting+of+the+Special+Interest)
14. Prévot, L., Gorisch, J., and Mukherjee, S. (2015b). Annotation and classification of french feedback communicative functions. In Proceedings of the The 29th Pacific Asia Conference on Language, Information and Computation (PACLIC 29), Shangai. [find](https://scholar.google.com/scholar?q=Pre%CC%81vot%2C+L%2E%2C+Gorisch%2C+J%2E%2C+and+Mukherjee%2C+S%2E+%282015b%29%2E+Annotation+and+classification+of+french+feedback+communicative+functions%2E+In+Proceedings+of+the+The+29th+Pacific+Asia+Conference+on+Language%2C+Infor)
15. Pympi, D. (2015). A python module for processing ELAN and Praat annotation files. [https://github.com/dopefishh/pympi](https://github.com/dopefishh/pympi). Accessed: 2015-02-25. [↗](https://github.com/dopefishh/pympi)
16. Schegloff, E. A. (1982). Discourse as an interactional achievement: Some use of uh-huh and other things that come between sentences. Georgetown University Round Table on Languages and Linguistics, Analyzing discourse: Text and talk, pages 71–93. [find](https://scholar.google.com/scholar?q=Schegloff%2C+E%2E+A%2E+%281982%29%2E+Discourse+as+an+interactional+achievement%3A+Some+use+of+uh%2Dhuh+and+other+things+that+come+between+sentences%2E+Georgetown+University+Round+Table+on+Languages+and+Linguistics%2C+Ana)
17. Wittenburg, P., Brugman, H., Russel, A., Klassmann, A., and Sloetjes, H. (2006). [Elan: a professional framework for multimodality research](https://md.research.elamur.ai/free/en/papers/wittenburg-2006-elan-professional-framework-multimodality-research/). In Proceedings of Language Resources and Evaluation Conference (LREC). Citeseer.
18. Yngve, V. H. (1970). On getting a word in edgewise. In Papers from the Sixth Regional Meeting of the Chicago Linguistic Society, pages 567–578. 10. [find](https://scholar.google.com/scholar?q=Yngve%2C+V%2E+H%2E+%281970%29%2E+On+getting+a+word+in+edgewise%2E+In+Papers+from+the+Sixth+Regional+Meeting+of+the+Chicago+Linguistic+Society%2C+pages+567%E2%80%93578%2E+10%2E)
19. Language Resource References Ellen Bard and Corine Astésano and Cheryl Frenck-Mestre and Mariapaola D’imperio and Alice Turk and Noël Nguyen. (2010). MAPTASK-AIX. Laboratoire Parole et Langage - UMR 7309 (LPL, Aix-en-Provence FR), Speech and Language Data Repository (SLDR/ORTOLANG), ISLRN oai:sldr.org:sldr000732. [find](https://scholar.google.com/scholar?q=Language+Resource+References+Ellen+Bard+and+Corine+Aste%CC%81sano+and+Cheryl+Frenck%2DMestre+and+Mariapaola+D%E2%80%99imperio+and+Alice+Turk+and+Noe%CC%88l+Nguyen%2E+%282010%29%2E+MAPTASK%2DAIX%2E+Laboratoire+Parole+et+Langage+%2D+UMR)
20. Roxane Bertrand and Béatrice Priego-Valverde. (2008). Transcriptions du corpus CID. Laboratoire Parole et Langage - UMR 7309 (LPL, Aix-en-Provence FR), Speech and Language Data Repository (SLDR/ORTOLANG), ISLRN oai:sldr.org:sldr000720. [find](https://scholar.google.com/scholar?q=Roxane+Bertrand+and+Be%CC%81atrice+Priego%2DValverde%2E+%282008%29%2E+Transcriptions+du+corpus+CID%2E+Laboratoire+Parole+et+Langage+%2D+UMR+7309+%28LPL%2C+Aix%2Den%2DProvence+FR%29%2C+Speech+and+Language+Data+Repository+%28SLDR%2FORTOL)
21. Jan Gorisch and Laurent Prévot. (2014). Audio-visual condition of Aix Map Task. Laboratoire Parole et Langage - UMR 7309 (LPL, Aix-en-Provence FR), Speech and Language Data Repository (SLDR/ORTOLANG), ISLRN oai:sldr.org:sldr000875. [find](https://scholar.google.com/scholar?q=Jan+Gorisch+and+Laurent+Pre%CC%81vot%2E+%282014%29%2E+Audio%2Dvisual+condition+of+Aix+Map+Task%2E+Laboratoire+Parole+et+Langage+%2D+UMR+7309+%28LPL%2C+Aix%2Den%2DProvence+FR%29%2C+Speech+and+Language+Data+Repository+%28SLDR%2FORTOLANG%29)
22. Laurent Prévot and Jan Gorisch and Roxane Bertrand. (2015). CoFee project Corpus Collection. Laboratoire Parole et Langage - UMR 7309 (LPL, Aix-en-Provence FR), Speech and Language Data Repository (SLDR/ORTOLANG), ISLRN oai:sldr.org:sldr000911. [find](https://scholar.google.com/scholar?q=Laurent+Pre%CC%81vot+and+Jan+Gorisch+and+Roxane+Bertrand%2E+%282015%29%2E+CoFee+project+Corpus+Collection%2E+Laboratoire+Parole+et+Langage+%2D+UMR+7309+%28LPL%2C+Aix%2Den%2DProvence+FR%29%2C+Speech+and+Language+Data+Repository+%28S)

*Updated 24 October 2026*