---
title: "UPOS-DEPREL Mismatches — paper by Yih and Dai (2023)"
description: "Two case studies then propose guideline changes, one for multiword expressions and one for modifier relations. 2 references."
authors: ["Tsy Yih", "Zheyuan Dai"]
year: 2023
venue: "Proceedings of the 37th Pacific Asia Conference on Language, Information and Computation"
source: "https://aclanthology.org/2023.paclic-1.5/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yih-2023-upos-deprel-mismatches-detecting-annotation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yih-2023-upos-deprel-mismatches-detecting-annotation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/17/) / UPOS-DEPREL Mismatches: Detecting Annotation Errors and Improving UD Guidelines based on Linguistic Knowledge

# UPOS-DEPREL Mismatches: Detecting Annotation Errors and Improving UD Guidelines based on Linguistic Knowledge

**Authors:** [Tsy Yih](https://md.research.elamur.ai/free/en/authors/tsy-yih/) and Zheyuan Dai · **Year:** 2023 · **Venue:** [Proceedings of the 37th Pacific Asia Conference on Language, Information and Computation](https://md.research.elamur.ai/free/en/venues/paclic/) ([PACLIC 2023](https://md.research.elamur.ai/free/en/venues/paclic/2023/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.paclic-1.5/) · [PDF](https://aclanthology.org/2023.paclic-1.5.pdf)

## Summary
Studies of UD guidelines usually examine one annotation layer at a time. This paper looks at how two layers interact: the universal part-of-speech tag (UPOS) and the dependency relation (DEPREL) on the same token. The authors cross-tabulate UPOS against DEPREL in the 20 PUD treebanks of UD v2.10. They find that the resulting matrices are sparse, that the frequencies of attested pairs follow a long-tail distribution, and that most parts of speech have one dominant relation. They explain this pattern with Croft's model, which crosses semantic classes with propositional acts.

From the dominant pairings and the attested minor pairings that reflect category shift (transcategorization), they define a Possible Combination Zone. Any pair outside the zone is flagged as one of three things:
- an annotation error,
- an inconsistency between annotators or treebanks,
- a construction that UD describes poorly.

This yields an error detector grounded in linguistic theory. Two case studies then propose guideline changes, one for multiword expressions and one for modifier relations. The work matters because it gives treebank maintainers a cheap, interpretable consistency check and a principled basis for revising tags.

## Key points
- **Data.** PUD (UD v2.10) covers 20 languages with 1,000 parallel news and Wikipedia sentences each. Treebank sizes range from 15,813 tokens (Finnish) to 28,788 (Japanese). DEPREL inventories range from 25 labels (Japanese) to 58 (Polish).
- **Utilization rate (UR).** UR is the share of attested pairs among all theoretically possible pairs. It is low everywhere, from 17.9% (Indonesian) to 31.6% (Icelandic). English uses 209 of 799 possible pairs (26.2%).
- **Two-way and one-way matches.**
  - Two-way matches include DET–det, ADJ–amod, ADV–advmod, NUM–nummod, CCONJ–cc, SCONJ–mark, ADP–case and PUNCT–punct.
  - Nominals align with argument relations once nsubj, obj, iobj and obl are merged.
  - One-way matches include VERB with acl, advcl, xcomp and ccomp, PROPN with flat, and PRON with expl.
- **Problem tags.**
  - PART has no consistent matching relation across languages, which suggests the category is a catch-all.
  - AUX splits between cop and aux, which together account for about 95% of its uses.
  - Nouns appear surprisingly often as modifiers.
- **Proposed guideline changes.**
  - For fixed and flat MWEs, give the first token the UPOS of the whole expression (e.g. *in addition* → ADV), mask internal UPOS, and concatenate the lemma, following Yih's CUD.
  - For modifiers, either merge them as SUD does with `mod`, or remap them to NP layers (deictics, numeratives, epithets, classifiers).

## Interesting ideas
- Use a UPOS×DEPREL contingency table with a whitelist of attested pairs as a rule-based checker, so that unusual pairs such as ADP & obj surface as likely errors without any trained model.
- Separate two-way matches from one-way matches, because relations outnumber tags and a pair can be dominant in only one direction.
- Read a tag that never settles on a dominant partner, like PART, as evidence that the category itself is incoherent rather than that annotators are careless.
- Treat a fixed MWE as one unit: the first token carries the category of the whole expression and the full lemma, while the inner tokens are masked.
- Encode finer semantic distinctions through deliberate, documented tag mismatches, such as associative adjectives as ADJ & nmod or fuzzy quantifiers as ADJ & nummod.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tree): Run the whitelist after the perceptron parser and gparse to flag implausible tag–relation pairs before trees reach frames.
- **I2** (learn): When witness parsers build silver data, give lower confidence to tokens whose pair falls outside the learned zone, and treat those tokens as candidates for self-induced repair rules.
- **I3** (tokens): Audit the tagger's own tag inventory this way to find catch-all tags worth splitting, such as AUX into copula and auxiliary.
- **I4** (frames): Expressions like *in addition* or *in theory* would reach frame building as single adverbials, so their inner nouns would not be mistaken for referents.
- **I5** (world): Mark *political*-type adjectives as classifying rather than describing, and treat *many*/*much* as quantity values rather than entity properties.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🌐 Universal Dependencies](https://md.research.elamur.ai/free/en/topics/universal-dependencies/) · 185 works

## Cite

Cite as: Tsy Yih and Zheyuan Dai. 2023. UPOS-DEPREL Mismatches: Detecting Annotation Errors and Improving UD Guidelines based on Linguistic Knowledge. Proceedings of the 37th Pacific Asia Conference on Language, Information and Computation.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{yih2023upos,
  title = {{UPOS-DEPREL Mismatches: Detecting Annotation Errors and Improving UD Guidelines based on Linguistic Knowledge}},
  author = {Tsy Yih and Zheyuan Dai},
  booktitle = {Proceedings of the 37th Pacific Asia Conference on Language, Information and Computation},
  year = {2023},
  url = {https://aclanthology.org/2023.paclic-1.5/}
}
```

- [← previous Unpacking Ambiguous Structure: A Dataset for Ambiguous Implicit Discourse Relations for English and Egyptian Arabic](https://md.research.elamur.ai/free/en/papers/ruby-2023-unpacking-ambiguous-structure-dataset-ambiguous/)
- [next → Using Word Embeddings for Identifying Emotions Relating to the Body in a Neo-Assyrian Corpus](https://md.research.elamur.ai/free/en/papers/bennett-2023-word-embeddings-identifying-emotions-relating/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/17/)

## References

2 references, 1 of them with a card of ours. ↗ leads to the work.

1. Lars Ahrenberg. 2019. [Towards an adequate account of parataxis in Universal Dependencies](https://md.research.elamur.ai/free/en/papers/ahrenberg-2019-adequate-account-parataxis-universal-dependencies/). In Proceedings of the Third Workshop on Universal Dependencies (UDW, SyntaxFest 2019), pages 94– 100, Paris, France. Association for Computational Linguistics.
2. Gosse Bouma, Jan Hajic, Dag Haug, Joakim Nivre, Per Erik Solberg, and Lilja Øvrelid. 2018. Expletives in Universal Dependency Treebanks. In Proceedings of the Second Workshop on Universal Dependencies (UDW 2018), pages 18–26, Brussels, Belgium. Association for Computational Linguistics. https://doi.org/1010.18653/v1/W18-6003. [↗](https://aclanthology.org/W18-6003/)

*Updated 2 November 2026*