--- title: "Message Understanding Conference- 6: A Brief History (1996)" description: "The authors show how the task grew steadily more complex. 57% recall. Cited by 276 papers here." authors: ["Ralph Grishman", "Beth Sundheim"] year: 1996 venue: "COLING 1996 Volume 1: The 16th International Conference on Computational Linguistics" source: "https://aclanthology.org/C96-1079/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/grishman-1996-message-understanding-conference-6-brief/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/grishman-1996-message-understanding-conference-6-brief/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [1996](https://md.research.elamur.ai/free/en/papers/1996/) / Message Understanding Conference- 6: A Brief History # Message Understanding Conference- 6: A Brief History **Authors:** [Ralph Grishman](https://md.research.elamur.ai/free/en/authors/ralph-grishman/) and Beth Sundheim · **Year:** 1996 · **Venue:** [COLING 1996 Volume 1: The 16th International Conference on Computational Linguistics](https://md.research.elamur.ai/free/en/venues/coling/) ([COLING 1996](https://md.research.elamur.ai/free/en/venues/coling/1996/)) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/C96-1079/) · [PDF](https://aclanthology.org/C96-1079.pdf) **Citations:** [276](https://md.research.elamur.ai/free/en/papers/grishman-1996-message-understanding-conference-6-brief/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary The paper reviews the history of the Message Understanding Conferences (MUC), a series of evaluation campaigns organized since 1987 by NRaD/NOSC with DARPA funding. The series centers on information extraction, i.e., the automatic extraction of structured facts from texts. The authors show how the task grew steadily more complex. It began with military messages, then moved to terrorism, and in MUC-5 covered joint ventures and microchip fabrication in English and Japanese. This complexity made system development increasingly costly. In response, MUC-6 introduced a set of separate tasks: named entity, template element, a "mini-MUC" scenario template, and a SemEval component for evaluating deeper text understanding (coreference, word sense disambiguation, predicate-argument structure). The results showed that systems could be adapted to a new scenario within a few weeks while achieving performance close to the best results of earlier MUCs. At the same time, most participants converged on a similar architecture: a cascade of finite-state automata that recognize patterns. The paper is important because it defined the standard tasks and metrics (recall, precision) on which subsequent NER and coreference research was built. ## Key points - History of the series: MUC-1 (1987) had no formal evaluation yet. The template to be filled had 10 slots in MUC-2 (1989), 18 in MUC-3 (1991), and 24 in MUC-4 (1992). In MUC-5 (1993, Tipster program), the joint ventures domain already had 11 templates with 47 slots, and a nested (hierarchical) template structure was introduced. - Best overall result in MUC-5: 57% recall and 64% precision. - Named entity in MUC-6 covers persons, organizations, locations, temporal expressions, monetary amounts, and percentages. The annotation is done in SGML using the ENAMEX and NUMEX tags. - Template element: the best system achieved 75% recall and 86% precision. Descriptive noun phrases proved to be the weakest point, with only 38% recall and 51% precision. - Scenario template (domain: management changes in companies): the best result was 47% recall and 70% precision, with most systems' scores falling within a narrow range. The coreference task still needs to be defined more precisely. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (2): [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works 💡 Concepts (2): [💡 Natural language processing](https://md.research.elamur.ai/free/en/concepts/natural-language-processing/) · 131 works · [💡 Information extraction](https://md.research.elamur.ai/free/en/concepts/information-extraction/) · 50 works ## Cite Cite as: Ralph Grishman and Beth Sundheim. 1996. Message Understanding Conference- 6: A Brief History. COLING 1996 Volume 1: The 16th International Conference on Computational Linguistics. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{grishman1996message, title = {{Message Understanding Conference- 6: A Brief History}}, author = {Ralph Grishman and Beth Sundheim}, booktitle = {COLING 1996 Volume 1: The 16th International Conference on Computational Linguistics}, year = {1996}, url = {https://aclanthology.org/C96-1079/} } ``` - [← previous MBT: A Memory-Based Part of Speech Tagger-Generator](https://md.research.elamur.ai/free/en/papers/daelemans-1996-mbt-memory-based-part-speech/) - [next → A Polynomial-Time Algorithm for Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/wu-1996-polynomial-time-algorithm-statistical-machine/) - [↑ back to papers of 1996](https://md.research.elamur.ai/free/en/papers/1996/) *Updated 29 October 2026*