---
title: "Event Chronography in Multi-modal Data — Murat 2026"
description: "Researchers who annotate multimodal interaction in Elan often struggle to export overlapping tiers for statistical work."
authors: ["Anaïs Claire Murat", "Maria Koutsombogera", "Carl Vogel"]
year: 2026
venue: "Proceedings of the Fifteenth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2026.lrec-1.724/"
doi: "10.63317/3yhonsvzhzq9"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/murat-2026-event-chronography-multi-modal-data/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/murat-2026-event-chronography-multi-modal-data/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2026](https://md.research.elamur.ai/free/papers/2026/page/2/) / Event Chronography in Multi-modal Data: The BME Method for Quantitative Analyses

# Event Chronography in Multi-modal Data: The BME Method for Quantitative Analyses

**Authors:** [Anaïs Claire Murat](https://md.research.elamur.ai/free/authors/anais-claire-murat/), Maria Koutsombogera, [Carl Vogel](https://md.research.elamur.ai/free/authors/carl-vogel/) · **Year:** 2026 · **Venue:** Proceedings of the Fifteenth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.lrec-1.724/) · [PDF](https://aclanthology.org/2026.lrec-1.724.pdf) · [DOI 10.63317/3yhonsvzhzq9](https://doi.org/10.63317/3yhonsvzhzq9) · [OpenAlex](https://openalex.org/W7160069748)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14)

## Summary
Researchers who annotate multimodal interaction in Elan often struggle to export overlapping tiers for statistical work. Speech, gaze and gesture rarely share start or end points. As a result, simple per-segment exports either duplicate content when one annotation spans several others, or lose track of where it began and ended. Annotations falling outside the reference tier can disappear entirely. Murat, Koutsombogera and Vogel propose the BME method, which sets exact timestamps aside and keeps only the relative order of events. Each annotation is reduced to a Beginning and an End. Every event from another tier that falls between them adds an indexed Middle. Counting Middles yields an event-based alternative to measuring duration in milliseconds. The authors release an open-source Python implementation that produces Elan-compatible output. They illustrate the method with three case studies:
- inter-tier dynamics in the Multisimo corpus;
- comparing time-based and event-based gaze durations against linguistic repetition;
- assessing how well annotators agree on boundary placement, using data from Sasmita and Swallow (2022).

The method offers a simple, count-based first look at how modalities are arranged relative to one another.

## Key points
- **Data source:** The Multisimo corpus contains 18 English three-party dialogues of roughly 2,000–3,000 words each. In each one, two players and a facilitator take part in a collaborative ranking game.
- **Case study 1 (conversation S07):** This dialogue has 252 mutual gazes and 685 turns. Chi-square tests on whether Beginnings and Ends fall inside or outside the other tier gave p ≤ 2.22e−16, with standardized residuals around 14.5–14.9. Turns mostly begin and end outside mutual gaze, while mutual gazes mostly begin and end inside turns. The authors read this as evidence that the turn channel takes priority.
- **Case study 2 (gaze and repetition):** Duration was measured both in milliseconds and as a count of Middles. The analysis found a significant link between gaze and repetition that earlier work on the same dataset had missed.
- **Code:** The implementation is available at github.com/anaismu/BME_Method.
- **Limitations:** The authors present BME as an initial seed rather than a complete analysis framework.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{murat2026event,
  title = {{Event Chronography in Multi-modal Data: The BME Method for Quantitative Analyses}},
  author = {Murat, Anaïs Claire and Koutsombogera, Maria and Vogel, Carl},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/3yhonsvzhzq9},
  url = {https://doi.org/10.63317/3yhonsvzhzq9}
}
```

- [← previous Evaluation of Paralinguistic-Aware Spoken Dialogue Systems using Next-Utterance Classification](https://md.research.elamur.ai/free/papers/miyazawa-2026-evaluation-paralinguistic-aware-spoken-dialogue/)
- [next → Expanding the Lexicon of Ge'ez Based African Languages: A Comparative Study of Amharic and Tigrinya](https://md.research.elamur.ai/free/papers/langstream-2026-teklehaymanot-expanding-lexicon-ge-ez-based/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/papers/2026/page/2/)

## References

22 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ron Artstein. 2017. Inter-annotator Agreement. In Nancy Ide and James Pustejovsky, editors, Handbook of Linguistic Annotation, pages 297– 313. Springer Netherlands, Dordrecht. [find](https://scholar.google.com/scholar?q=Ron+Artstein%2E+2017%2E+Inter%2Dannotator+Agreement%2E+In+Nancy+Ide+and+James+Pustejovsky%2C+editors%2C+Handbook+of+Linguistic+Annotation%2C+pages+297%E2%80%93+313%2E+Springer+Netherlands%2C+Dordrecht%2E)
2. Jean Carletta. 1996. Assessing Agreement on Classification Tasks: The Kappa Statistic. Computational Linguistics, 22(2):249–254. Place: Cambridge, MA Publisher: MIT Press. [find](https://scholar.google.com/scholar?q=Jean+Carletta%2E+1996%2E+Assessing+Agreement+on+Classification+Tasks%3A+The+Kappa+Statistic%2E+Computational+Linguistics%2C+22%282%29%3A249%E2%80%93254%2E+Place%3A+Cambridge%2C+MA+Publisher%3A+MIT+Press%2E) [our card](https://md.research.elamur.ai/free/papers/carletta-1996-assessing-agreement-classification-tasks-kappa/)
3. Tim Fernando. 2019. Pictorial Narratives and Temporal Refinement. Semantics and Linguistic Theory, 29(0):43–62. Number: 0. [find](https://scholar.google.com/scholar?q=Tim+Fernando%2E+2019%2E+Pictorial+Narratives+and+Temporal+Refinement%2E+Semantics+and+Linguistic+Theory%2C+29%280%29%3A43%E2%80%9362%2E+Number%3A+0%2E)
4. Emer Gilmartin. 2021. Composition and Dynamics of Multiparty Casual Conversation: A Corpus-based Analysis. Ph.D. thesis. [find](https://scholar.google.com/scholar?q=Emer+Gilmartin%2E+2021%2E+Composition+and+Dynamics+of+Multiparty+Casual+Conversation%3A+A+Corpus%2Dbased+Analysis%2E+Ph%2ED%2E+thesis%2E)
5. Milan Gnjatović, Jovica Tasevski, Branislav Borovac, and Nemanja Maček. 2018. An Entropy-Based Approach to Automatic Detection of Critical Changes in Human-Machine Interaction. In 2018 9th IEEE International Conference on Cognitive Infocommunications (CogInfoCom), pages 000175–000178. ISSN: 2380-735… [find](https://scholar.google.com/scholar?q=Milan+Gnjatovi%C4%87%2C+Jovica+Tasevski%2C+Branislav+Borovac%2C+and+Nemanja+Ma%C4%8Dek%2E+2018%2E+An+Entropy%2DBased+Approach+to+Automatic+Detection+of+Critical+Changes+in+Human%2DMachine+Interaction%2E+In+2018+9th+IEEE+Intern)
6. Jing Han, Zixing Zhang, Maja Pantic, and Björn Schuller. 2021. Internet of emotional people: Towards continual affective computing cross cultures via audiovisual signals. Future Generation Computer Systems, 114:294–306. [find](https://scholar.google.com/scholar?q=Jing+Han%2C+Zixing+Zhang%2C+Maja+Pantic%2C+and+Bj%C3%B6rn+Schuller%2E+2021%2E+Internet+of+emotional+people%3A+Towards+continual+affective+computing+cross+cultures+via+audiovisual+signals%2E+Future+Generation+Computer+Sy)
7. Henning Holle and Robert Rein. 2015. EasyDIAg: A tool for easy determination of interrater agreement. Behavior Research Methods, 47(3):837– 847. [find](https://scholar.google.com/scholar?q=Henning+Holle+and+Robert+Rein%2E+2015%2E+EasyDIAg%3A+A+tool+for+easy+determination+of+interrater+agreement%2E+Behavior+Research+Methods%2C+47%283%29%3A837%E2%80%93+847%2E)
8. Laszlo Hunyadi. 2019. Agreeing/Disagreeing in a Dialogue: Multimodal Patterns of Its Expression. Frontiers in Psychology, 10:1373. [find](https://scholar.google.com/scholar?q=Laszlo+Hunyadi%2E+2019%2E+Agreeing%2FDisagreeing+in+a+Dialogue%3A+Multimodal+Patterns+of+Its+Expression%2E+Frontiers+in+Psychology%2C+10%3A1373%2E)
9. Spyros Kousidis, David Dorran, Ciaran Mcdonnell, and Eugene Coyle. 2009. Convergence in human dialogues time series analysis of acoustic feature. Convergence, 2009:01–01. [find](https://scholar.google.com/scholar?q=Spyros+Kousidis%2C+David+Dorran%2C+Ciaran+Mcdonnell%2C+and+Eugene+Coyle%2E+2009%2E+Convergence+in+human+dialogues+time+series+analysis+of+acoustic+feature%2E+Convergence%2C+2009%3A01%E2%80%9301%2E)
10. Magnus S. Magnusson. 2020. T-Pattern Detection and Analysis (TPA) With THEMETM: A Mixed Methods Approach. Frontiers in Psychology, 10. [find](https://scholar.google.com/scholar?q=Magnus+S%2E+Magnusson%2E+2020%2E+T%2DPattern+Detection+and+Analysis+%28TPA%29+With+THEMETM%3A+A+Mixed+Methods+Approach%2E+Frontiers+in+Psychology%2C+10%2E)
11. Angeliki Metallinou and Shrikanth Narayanan. 2013. Annotation and processing of continuous emotional attributes: Challenges and opportunities. In 2013 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG), pages 1–8. [find](https://scholar.google.com/scholar?q=Angeliki+Metallinou+and+Shrikanth+Narayanan%2E+2013%2E+Annotation+and+processing+of+continuous+emotional+attributes%3A+Challenges+and+opportunities%2E+In+2013+10th+IEEE+International+Conference+and+Workshops+)
12. Erwan Moreau, Ashjan Alsulaimani, Alfredo Maldonado, Lifeng Han, Carl Vogel, and Koel Dutta Chowdhury. 2018. Semantic reranking of CRF label sequences for verbal multiword expression identification. In Multiword expressions at length and in depth, pages 177–207. Language Science Press, Berlin. [find](https://scholar.google.com/scholar?q=Erwan+Moreau%2C+Ashjan+Alsulaimani%2C+Alfredo+Maldonado%2C+Lifeng+Han%2C+Carl+Vogel%2C+and+Koel+Dutta+Chowdhury%2E+2018%2E+Semantic+reranking+of+CRF+label+sequences+for+verbal+multiword+expression+identification%2E+I)
13. Anais Murat, Maria Koutsombogera, and Carl Vogel. 2022. Mutual Gaze and Linguistic Repetition in a Multimodal Corpus. In Proceedings of the Language Resources and Evaluation Conference, pages 2771–2780, Marseille, France. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Anais+Murat%2C+Maria+Koutsombogera%2C+and+Carl+Vogel%2E+2022%2E+Mutual+Gaze+and+Linguistic+Repetition+in+a+Multimodal+Corpus%2E+In+Proceedings+of+the+Language+Resources+and+Evaluation+Conference%2C+pages+2771%E2%80%93278)
14. Yukiko Nakano, Gabe Reinstein, Tom Stocky, and Justine Cassell. 2003. Towards a Model of Face-to-Face Grounding. In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 553–561, Sapporo, Japan. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yukiko+Nakano%2C+Gabe+Reinstein%2C+Tom+Stocky%2C+and+Justine+Cassell%2E+2003%2E+Towards+a+Model+of+Face%2Dto%2DFace+Grounding%2E+In+Proceedings+of+the+41st+Annual+Meeting+of+the+Association+for+Computational+Linguist)
15. Christophe Parisse, Marion Blondel, Stéphanie Caët, Claire Danet, Coralie Vincent, and Aliyah Morgenstern. 2022. Multidimensional coding of multimodal languaging in multi-party settings. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 2781–2787, Marseille, France… [find](https://scholar.google.com/scholar?q=Christophe+Parisse%2C+Marion+Blondel%2C+St%C3%A9phanie+Ca%C3%ABt%2C+Claire+Danet%2C+Coralie+Vincent%2C+and+Aliyah+Morgenstern%2E+2022%2E+Multidimensional+coding+of+multimodal+languaging+in+multi%2Dparty+settings%2E+In+Proceeding)
16. Martin J. Pickering and Simon Garrod. 2004. Toward a mechanistic psychology of dialogue. Behavioral and Brain Sciences, 27(02). [find](https://scholar.google.com/scholar?q=Martin+J%2E+Pickering+and+Simon+Garrod%2E+2004%2E+Toward+a+mechanistic+psychology+of+dialogue%2E+Behavioral+and+Brain+Sciences%2C+27%2802%29%2E)
17. Karen Sasmita and Khena M. Swallow. 2022. Measuring event segmentation: An investigation into the stability of event boundary agreement across groups. Behavior Research Methods, 55(1):428– 447. [find](https://scholar.google.com/scholar?q=Karen+Sasmita+and+Khena+M%2E+Swallow%2E+2022%2E+Measuring+event+segmentation%3A+An+investigation+into+the+stability+of+event+boundary+agreement+across+groups%2E+Behavior+Research+Methods%2C+55%281%29%3A428%E2%80%93+447%2E)
18. Hesheng Xu, Bin Hu, and Suneet Kumar Gupta. 2022. Legal text recognition using lstm-crf deep learning model. [find](https://scholar.google.com/scholar?q=Hesheng+Xu%2C+Bin+Hu%2C+and+Suneet+Kumar+Gupta%2E+2022%2E+Legal+text+recognition+using+lstm%2Dcrf+deep+learning+model%2E)
19. Intell. Neuroscience, 2022. 13. [find](https://scholar.google.com/scholar?q=Intell%2E+Neuroscience%2C+2022%2E+13%2E)
20. Language Resource References Koutsombogera, Maria and Vogel, Carl. 2018. Modeling Collaborative Multimodal Behavior in Group Dialogues: The MULTISIMO Corpus. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Language+Resource+References+Koutsombogera%2C+Maria+and+Vogel%2C+Carl%2E+2018%2E+Modeling+Collaborative+Multimodal+Behavior+in+Group+Dialogues%3A+The+MULTISIMO+Corpus%2E+European+Language+Resources+Association+%28E)
21. Sasmita, Karen and Swallow, Khena M. 2022. Measuring event segmentation: An investigation into the stability of event boundary agreement across groups. [find](https://scholar.google.com/scholar?q=Sasmita%2C+Karen+and+Swallow%2C+Khena+M%2E+2022%2E+Measuring+event+segmentation%3A+An+investigation+into+the+stability+of+event+boundary+agreement+across+groups%2E)
22. Wittenburg, Peter and Brugman, Hennie and Russel, Albert and Klassmann, Alex and Sloetjes, Han. 2006. ELAN: a Professional Framework for Multimodality Research. European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Wittenburg%2C+Peter+and+Brugman%2C+Hennie+and+Russel%2C+Albert+and+Klassmann%2C+Alex+and+Sloetjes%2C+Han%2E+2006%2E+ELAN%3A+a+Professional+Framework+for+Multimodality+Research%2E+European+Language+Resources+Association) [our card](https://md.research.elamur.ai/free/papers/wittenburg-2006-elan-professional-framework-multimodality-research/)