---
title: "Overview of AMALGUM – Large Silver Quality Annotations"
description: "Discourse-oriented research faces a gap. 9 references. Among them: 1."
authors: ["Luke Gessler", "Siyao Peng", "Yang Liu", "Yilun Zhu", "Shabnam Behzad", "Amir Zeldes"]
year: 2021
venue: "Proceedings of the Society for Computation in Linguistics 2021"
source: "https://aclanthology.org/2021.scil-1.52/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/gessler-2021-overview-amalgum-large-silver-quality/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/gessler-2021-overview-amalgum-large-silver-quality/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/8/) / Overview of AMALGUM – Large Silver Quality Annotations across English Genres

# Overview of AMALGUM – Large Silver Quality Annotations across English Genres

**Authors:** [Luke Gessler](https://md.research.elamur.ai/free/en/authors/luke-gessler/), [Siyao Peng](https://md.research.elamur.ai/free/en/authors/siyao-peng/), [Yang Liu](https://md.research.elamur.ai/free/en/authors/yang-liu/), [Yilun Zhu](https://md.research.elamur.ai/free/en/authors/yilun-zhu/), [Shabnam Behzad](https://md.research.elamur.ai/free/en/authors/shabnam-behzad/) and [Amir Zeldes](https://md.research.elamur.ai/free/en/authors/amir-zeldes/) · **Year:** 2021 · **Venue:** [Proceedings of the Society for Computation in Linguistics 2021](https://md.research.elamur.ai/free/en/venues/scil/) ([SCiL 2021](https://md.research.elamur.ai/free/en/venues/scil/2021/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.scil-1.52/) · [PDF](https://aclanthology.org/2021.scil-1.52.pdf) · [OpenAlex](https://openalex.org/W3197065044)

**Citations:** 1 citations (OpenAlex, 2026-10-14)

## Summary
Discourse-oriented research faces a gap. Huge web corpora carry only shallow annotation and have unclear makeup, while richly annotated corpora tend to be small. AMALGUM (A Machine-Annotated Lookalike of GUM) is meant to sit between the two. The authors used the gold GUM corpus (130K tokens, eight genres) as a template. They collected about 4M tokens of openly licensed web text in the same eight genres and annotated it automatically with every layer GUM has: tokens, POS, lemmas, UD trees with morphology, sentence types, document structure, nested entities, coreference and RST discourse trees. Three strategies push quality above off-the-shelf tools:
- retraining components on genre-matched GUM data,
- stacking several models' outputs,
- feeding the output of one layer into another as features.

Evaluation used the GUM test set plus a hand-corrected 2,000-token AMALGUM sample. The pipeline improved over off-the-shelf tools on most tasks, most clearly for tagging and parsing. Discourse-level layers remain far from gold quality. The result is a free, genre-balanced silver resource for linguistic variation studies, augmentation, active learning and pretraining.

## Key points
- The eight sources each contribute about 500K tokens, in documents of 500–1K tokens:
  - academic: MDPI
  - biography: Wikipedia
  - fiction: Gutenberg
  - forum: Reddit
  - how-to: wikiHow
  - interview: Wikinews
  - news: Wikinews
  - travel: Wikivoyage
- The POS tagger is an ensemble over four base taggers: Flair and StanfordNLP, each trained on OntoNotes and on GUM. On AMALGUM, tagging accuracy rose from 93.99 to 97.37.
- StanfordNLP was retrained on GUM and given the stacked POS tags instead of its own. LAS on AMALGUM went from 81.41 to 85.77, and on GUM from 83.66 to 85.89, beating the prior best GUM result.
- Coreference combines the rule/knowledge-based xrenner with BERT embeddings. A nested NER system supplies entity types for mentions whose spans match exactly. RST parsing extends DPLP with structure, sentence-type, genre and dependency features, with segmentation done by ToNy.
- Error analysis by layer:
  - **Parsing:** about 20% of errors come from Reddit. PP attachment (obl vs. nmod) causes about 25% of all parsing errors. Biographies show odd attachment of numbers and dates.
  - **Coreference:** paraphrased mentions are the main failure. Interviews and news are the weakest genres.

## Interesting ideas
- Stacking taggers trained on different corpora (a broad one and an in-domain one) lets a meta-model learn which source to trust in which context.
- The parser is told to use the upstream ensemble's tokenization and tags rather than its own internal guesses, so better tags propagate into better trees.
- Two entity systems are fused at matching spans: one is good at finding mentions, the other at rejecting non-entities and typing them.
- Silver annotation of a large corpus is checked against a small hand-corrected, genre-balanced sample from the corpus itself, not just a standard test set.
- Per-genre error analysis exposed genre-specific failures that a single averaged score hides, such as "Chapter I" tagged as PERSON in fiction or ATTRIBUTION guessed from the news genre alone.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): The tagger could combine predictions from models trained on different treebanks via a small learned arbiter, keeping the voting record as an explanation.
- **I2** (tree): gparse and the perceptron parser could consume the best available tag layer instead of re-predicting tags. This would make tagging errors and their parse consequences traceable.
- **I3** (frames): Coreference in prag could accept mention candidates from one witness and entity types from another, joined only on identical spans, keeping each contribution auditable.
- **I4** (learn): Each silver batch agreed by the witness parsers could get a small corrected, genre-stratified slice for checking, to catch domain drift that benchmark scores hide.
- **I5** (answer): Evaluation of readers on fables, tests and word problems could be broken down by text genre to reveal shortcuts the system learned from genre priors.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Georgetown University](https://md.research.elamur.ai/free/en/universities/georgetown-university/) — University, Washington, United States · 88 works · 4,732 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [📚 Literature and Literary Theory](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/literature-and-literary-theory/) · 519 works

🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works

## Cite

Cite as: Luke Gessler, Siyao Peng, Yang Liu, Yilun Zhu, Shabnam Behzad and Amir Zeldes. 2021. Overview of AMALGUM – Large Silver Quality Annotations across English Genres. Proceedings of the Society for Computation in Linguistics 2021.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{gessler2021overview,
  title = {{Overview of AMALGUM – Large Silver Quality Annotations across English Genres}},
  author = {Luke Gessler, Siyao Peng, Yang Liu, Yilun Zhu, Shabnam Behzad and Amir Zeldes},
  booktitle = {Proceedings of the Society for Computation in Linguistics 2021},
  year = {2021},
  doi = {10.7275/ep47-3t54},
  url = {https://doi.org/10.7275/ep47-3t54}
}
```

- [← previous Overcoming the challenges in morphological annotation of Turkish in universal dependencies framework](https://md.research.elamur.ai/free/en/papers/bedir-2021-overcoming-challenges-morphological-annotation-turkish/)
- [next → Points of Contact: The Shared Intellectual History of Vocalisation in Syriac, Arabic, and Hebrew](https://md.research.elamur.ai/free/en/papers/posegay-2021-points-contact-shared-intellectual/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/8/)

## References

9 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alan Akbik, Tanja Bergmann, and Roland Vollgraf. 2019. [Pooled contextualized embeddings for named entity recognition](https://md.research.elamur.ai/free/en/papers/akbik-2019-pooled-contextualized-embeddings-named-entity/). In Proceedings of NAACL 2019, pages 724–728.
2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805. [↗](https://arxiv.org/abs/1810.04805)
3. Eduard Hovy, Mitchell Marcus, Martha Palmer, Lance Ramshaw, and Ralph Weischedel. 2006. OntoNotes: The 90% solution. In Proceedings of NAACL 2006, pages 57–60, New York. [find](https://scholar.google.com/scholar?q=Eduard+Hovy%2C+Mitchell+Marcus%2C+Martha+Palmer%2C+Lance+Ramshaw%2C+and+Ralph+Weischedel%2E+2006%2E+OntoNotes%3A+The+90%25+solution%2E+In+Proceedings+of+NAACL+2006%2C+pages+57%E2%80%9360%2C+New+York%2E)
4. Yangfeng Ji and Jacob Eisenstein. 2014. [Representation learning for text-level discourse parsing](https://md.research.elamur.ai/free/en/papers/ji-2014-representation-learning-text-level-discourse/). In Proceedings of ACL 2014, pages 13–24, Baltimore, MD.
5. Philippe Muller, Chloé Braud, and Mathieu Morey. 2019. ToNy: Contextual embeddings for accurate multilingual discourse segmentation of full documents. In Proceedings of DISRPT 2019, pages 115– 124, Minneapolis, MN. [find](https://scholar.google.com/scholar?q=Philippe+Muller%2C+Chloe%CC%81+Braud%2C+and+Mathieu+Morey%2E+2019%2E+ToNy%3A+Contextual+embeddings+for+accurate+multilingual+discourse+segmentation+of+full+documents%2E+In+Proceedings+of+DISRPT+2019%2C+pages+115%E2%80%93+124%2C+M)
6. Peng Qi, Timothy Dozat, Yuhao Zhang, and Christopher D. Manning. 2018. Universal Dependency parsing from scratch. In Proceedings of CoNLL 2018, pages 160–170, Brussels, Belgium. [find](https://scholar.google.com/scholar?q=Peng+Qi%2C+Timothy+Dozat%2C+Yuhao+Zhang%2C+and+Christopher+D%2E+Manning%2E+2018%2E+Universal+Dependency+parsing+from+scratch%2E+In+Proceedings+of+CoNLL+2018%2C+pages+160%E2%80%93170%2C+Brussels%2C+Belgium%2E)
7. Takashi Shibuya and Eduard Hovy. 2020. Nested named entity recognition via second-best sequence learning and decoding. TACL, 8:605–620. [↗](https://doi.org/10.1162/tacl_a_00334)
8. Amir Zeldes and Shuo Zhang. 2016. [When annotation schemes change rules help: A configurable approach to coreference resolution beyond OntoNotes](https://md.research.elamur.ai/free/en/papers/zeldes-2016-when-annotation-schemes-change-rules/). In Proceedings of CORBON 2016, pages 92–101.
9. Daniel Zeman, Jan Hajič, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, and Slav Petrov. 2018. CoNLL 2018 shared task. In Proceedings of CoNLL 2018, pages 1–21, Brussels, Belgium. [find](https://scholar.google.com/scholar?q=Daniel+Zeman%2C+Jan+Hajic%CC%8C%2C+Martin+Popel%2C+Martin+Potthast%2C+Milan+Straka%2C+Filip+Ginter%2C+Joakim+Nivre%2C+and+Slav+Petrov%2E+2018%2E+CoNLL+2018+shared+task%2E+In+Proceedings+of+CoNLL+2018%2C+pages+1%E2%80%9321%2C+Brussels%2C+Be)

*Updated 1 November 2026*