---
title: "Biaffine Dependency and Semantic Graph — Attardi 2021"
description: "The paper describes the University of Pisa system for the IWPT 2021 Shared Task."
authors: ["Giuseppe Attardi", "Daniele Sartiano", "Maria Simi"]
year: 2021
venue: "Proceedings of the 17th International Conference on Parsing Technologies and the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies (IWPT 2021)"
source: "https://aclanthology.org/2021.iwpt-1.19/"
doi: "10.18653/v1/2021.iwpt-1.19"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/attardi-2021-biaffine-dependency-semantic-graph-parsing/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/attardi-2021-biaffine-dependency-semantic-graph-parsing/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2021](https://md.research.elamur.ai/free/papers/2021/) / Biaffine Dependency and Semantic Graph Parsing for EnhancedUniversal Dependencies

# Biaffine Dependency and Semantic Graph Parsing for EnhancedUniversal Dependencies

**Authors:** [Giuseppe Attardi](https://md.research.elamur.ai/free/authors/giuseppe-attardi/), [Daniele Sartiano](https://md.research.elamur.ai/free/authors/daniele-sartiano/), [Maria Simi](https://md.research.elamur.ai/free/authors/maria-simi/) · **Year:** 2021 · **Venue:** Proceedings of the 17th International Conference on Parsing Technologies and the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies (IWPT 2021) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.iwpt-1.19/) · [PDF](https://aclanthology.org/2021.iwpt-1.19.pdf) · [DOI 10.18653/v1/2021.iwpt-1.19](https://doi.org/10.18653/v1/2021.iwpt-1.19) · [OpenAlex](https://openalex.org/W3184051089)

**Citations:** 9 citations (openalex-snapshot · 2026-10-14) · [9](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
The paper describes the University of Pisa system for the IWPT 2021 Shared Task. The task was to build enhanced dependency graphs from raw text for 17 languages, with quality evaluated using the ELAS metric. The authors combined two parsers that share an architecture based on DiaParser, which derives from SuPar. The first is a biaffine parser for syntactic trees. It takes word representations only from a transformer model. It adds attention weights from one transformer layer, multiplied by a learnable coefficient, to the arc scores. The second is a semantic graph parser following the approach of Dozat and Manning (2018). It uses a sigmoid loss function instead of softmax, so a word can have several heads. The final graph is the union of the arcs from both parsers. A separate transformer model was used for each language; for Ukrainian it was ELECTRA from dbmdz. The system achieves high LAS on languages with large treebanks, while results on low-resource languages are weaker. After the deadline, the authors tested a shared multilingual model based on XLM-RoBERTa, which noticeably improved results for Lithuanian, Latvian and Tamil. The work provides a simple, reproducible baseline for enhanced UD, including for Ukrainian.

## Key points
- Using transformer attention weights (usually from the 6th layer) as a hint for the parser raised UAS/LAS on the English dev set with Electra from 91.32/89.33 to 92.21/90.31.
- Official test results (unipi-smax), averaged over languages: LAS 87.25, ELAS 83.64. Italian had the highest ELAS (91.81) and Tamil the lowest (51.73).
- Ukrainian with the dbmdz/electra-base-ukrainian-cased-discriminator model: UAS 93.68, LAS 91.92, EULAS 89.41, ELAS 87.51.
- Stanza was used for tokenization and multi-word token splitting. For Italian, the authors trained their own MWT model on ISST with additional sentences and contributed it to the official Stanza distribution.
- The multilingual xlm-roberta-large model was trained on all corpora together and then fine-tuned for each language. Delexicalization reduced the number of enhanced labels from 6125 to 1282. Lithuanian EULAS rose by 6.75 points, and ELAS rose from 71.31 to 76.62. At the same time, Slovak ELAS dropped from 86.05 to 81.17, probably because the lexical parts of the labels were not restored perfectly.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

Topics: [Dependency parsing](https://md.research.elamur.ai/free/topics/dependency-parsing/)

## Cited by

Cited by

9 papers of our library cite it (of the 140673 whose bibliographies we read); 3 of them have a page here, 6 not yet.

- [Explicit Morphological Knowledge Improves Pre-training of Language Models for Hebrew](https://md.research.elamur.ai/free/papers/langstream-2023-gueta-explicit-morphological-knowledge-improves-pre/)
- [A Second Wave of UD Hebrew Treebanking and Cross-Domain Parsing](https://md.research.elamur.ai/free/papers/ud-2022-zeldes-second-wave-ud-hebrew-treebanking/)
- [The DISRPT 2021 Shared Task on Elementary Discourse Unit Segmentation, Connective Detection, and Relation Classification](https://md.research.elamur.ai/free/papers/zeldes-2021-disrpt-2021-shared-task-elementary/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{attardi2021biaffine,
  title = {{Biaffine Dependency and Semantic Graph Parsing for EnhancedUniversal Dependencies}},
  author = {Attardi, Giuseppe and Sartiano, Daniele and Simi, Maria},
  booktitle = {Proceedings of the 17th International Conference on Parsing Technologies and the IWPT 2021 Shared Task on Parsing into Enhanced Universal Dependencies (IWPT 2021)},
  year = {2021},
  doi = {10.18653/v1/2021.iwpt-1.19},
  url = {https://doi.org/10.18653/v1/2021.iwpt-1.19}
}
```

- [← previous Benchmarking Transformer-based Language Models for Arabic Sentiment and Sarcasm Detection](https://md.research.elamur.ai/free/papers/farha-2021-benchmarking-transformer-based-language-models/)
- [next → BitFit: Simple Parameter-efficient Fine-tuning for Transformer-based Masked Language-models](https://md.research.elamur.ai/free/papers/ben-2021-zaken-bitfit-simple-parameter-efficient-fine/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/papers/2021/)

## References

14 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Gosse Bouma, Djamé Seddah, and Daniel Zeman. 2021. From raw text to enhanced universal dependencies: The parsing shared task at iwpt 2021. In Proceedings of the 17th International Conference on Parsing Technologies (IWPT 2021), pages 146–157, Bangkok, Thailand (online). Association for Computationa… [find](https://scholar.google.com/scholar?q=Gosse+Bouma%2C+Djame%CC%81+Seddah%2C+and+Daniel+Zeman%2E+2021%2E+From+raw+text+to+enhanced+universal+dependencies%3A+The+parsing+shared+task+at+iwpt+2021%2E+In+Proceedings+of+the+17th+International+Conference+on+Parsi) [our card](https://md.research.elamur.ai/free/papers/ud-2021-bouma-raw-text-enhanced-universal-dependencies/)
2. Y. Chu. 1965. On the shortest arborescence of a directed graph. Scientia Sinica, 14:1396–1400. [find](https://scholar.google.com/scholar?q=Y%2E+Chu%2E+1965%2E+On+the+shortest+arborescence+of+a+directed+graph%2E+Scientia+Sinica%2C+14%3A1396%E2%80%931400%2E)
3. Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. 2020. Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555. [find](https://scholar.google.com/scholar?q=Kevin+Clark%2C+Minh%2DThang+Luong%2C+Quoc+V+Le%2C+and+Christopher+D+Manning%2E+2020%2E+Electra%3A+Pre%2Dtraining+text+encoders+as+discriminators+rather+than+generators%2E+arXiv+preprint+arXiv%3A2003%2E10555%2E)
4. Timothy Dozat and Christopher D. Manning. 2017. Deep biaffine attention for neural dependency parsing. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. Open-Review.net. [find](https://scholar.google.com/scholar?q=Timothy+Dozat+and+Christopher+D%2E+Manning%2E+2017%2E+Deep+biaffine+attention+for+neural+dependency+parsing%2E+In+5th+International+Conference+on+Learning+Representations%2C+ICLR+2017%2C+Toulon%2C+France%2C+April+24%2D) [our card](https://md.research.elamur.ai/free/papers/missing-2016-dozat-deep-biaffine-attention-neural-dependency/)
5. Timothy Dozat and Christopher D. Manning. 2018. Simpler but more accurate semantic dependency parsing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 484–490, Melbourne, Australia. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Timothy+Dozat+and+Christopher+D%2E+Manning%2E+2018%2E+Simpler+but+more+accurate+semantic+dependency+parsing%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volum)
6. J. EDMONDS. 1967. Optimum branchings. Journal of Research of the National Bureau of Standards, B, 71:233–240. [find](https://scholar.google.com/scholar?q=J%2E+EDMONDS%2E+1967%2E+Optimum+branchings%2E+Journal+of+Research+of+the+National+Bureau+of+Standards%2C+B%2C+71%3A233%E2%80%93240%2E)
7. Stefan Grünewald and Annemarie Friedrich. 2020. RobertNLP at the IWPT 2020 shared task: Surprisingly simple enhanced UD parsing for English. In Proceedings of the 16th International Conference on Parsing Technologies and the IWPT 2020 Shared Task on Parsing into Enhanced Universal Dependencies, pag… [find](https://scholar.google.com/scholar?q=Stefan+Gru%CC%88newald+and+Annemarie+Friedrich%2E+2020%2E+RobertNLP+at+the+IWPT+2020+shared+task%3A+Surprisingly+simple+enhanced+UD+parsing+for+English%2E+In+Proceedings+of+the+16th+International+Conference+on+Par) [our card](https://md.research.elamur.ai/free/papers/ud-2020-grunewald-robertnlp-iwpt-2020-shared-task/)
8. John Hewitt and Christopher D. Manning. 2019. A structural probe for finding syntax in word representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4129… [find](https://scholar.google.com/scholar?q=John+Hewitt+and+Christopher+D%2E+Manning%2E+2019%2E+A+structural+probe+for+finding+syntax+in+word+representations%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+of+the+Association+for+) [our card](https://md.research.elamur.ai/free/papers/hewitt-2019-structural-probe-finding-syntax-word/)
9. Jenna Kanerva, Filip Ginter, Niko Miekka, Akseli Leino, and Tapio Salakoski. 2018. Turku neural parser pipeline: An end-to-end system for the CoNLL 2018 shared task. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 133–142, Brussels, B… [find](https://scholar.google.com/scholar?q=Jenna+Kanerva%2C+Filip+Ginter%2C+Niko+Miekka%2C+Akseli+Leino%2C+and+Tapio+Salakoski%2E+2018%2E+Turku+neural+parser+pipeline%3A+An+end%2Dto%2Dend+system+for+the+CoNLL+2018+shared+task%2E+In+Proceedings+of+the+CoNLL+2018+S)
10. Nelson F. Liu, Matt Gardner, Yonatan Belinkov, Matthew E. Peters, and Noah A. Smith. 2019a. Linguistic knowledge and transferability of contextual representations. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Tec… [find](https://scholar.google.com/scholar?q=Nelson+F%2E+Liu%2C+Matt+Gardner%2C+Yonatan+Belinkov%2C+Matthew+E%2E+Peters%2C+and+Noah+A%2E+Smith%2E+2019a%2E+Linguistic+knowledge+and+transferability+of+contextual+representations%2E+In+Proceedings+of+the+2019+Conferenc) [our card](https://md.research.elamur.ai/free/papers/liu-2019-linguistic-knowledge-transferability-contextual-representations/)
11. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. Roberta: A robustly optimized bert pretraining approach. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019b%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E)
12. Ilya Loshchilov and F. Hutter. 2019. Decoupled weight decay regularization. In ICLR. [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+F%2E+Hutter%2E+2019%2E+Decoupled+weight+decay+regularization%2E+In+ICLR%2E)
13. Daniel Zeman, Jan Hajič, Martin Popel, Martin Potthast, Milan Straka, Filip Ginter, Joakim Nivre, and Slav Petrov. 2018. [find](https://scholar.google.com/scholar?q=Daniel+Zeman%2C+Jan+Hajic%CC%8C%2C+Martin+Popel%2C+Martin+Potthast%2C+Milan+Straka%2C+Filip+Ginter%2C+Joakim+Nivre%2C+and+Slav+Petrov%2E+2018%2E)
14. CoNLL 2018 shared task: Multilingual parsing from raw text to universal dependencies. In Proceedings of the CoNLL 2018 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies, pages 1–21, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=CoNLL+2018+shared+task%3A+Multilingual+parsing+from+raw+text+to+universal+dependencies%2E+In+Proceedings+of+the+CoNLL+2018+Shared+Task%3A+Multilingual+Parsing+from+Raw+Text+to+Universal+Dependencies%2C+pages+) [our card](https://md.research.elamur.ai/free/papers/ud-2018-zeman-conll-2018-shared-task-multilingual/)