--- title: "An Annotation Scheme for Free Word Order Languages — Skut 1997" description: "The authors propose a different scheme that puts argument structure at the center. 97% accuracy. 13 references, cited by 107 papers here." authors: ["Wojciech Skut", "Brigitte Krenn", "Thorsten Brants", "Hans Uszkoreit"] year: 1997 venue: "Fifth Conference on Applied Natural Language Processing" source: "https://aclanthology.org/A97-1014/" doi: "10.3115/974557.974571" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/skut-1997-annotation-scheme-free-word-order/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/skut-1997-annotation-scheme-free-word-order/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [1997](https://md.research.elamur.ai/free/en/papers/1997/) / An Annotation Scheme for Free Word Order Languages # An Annotation Scheme for Free Word Order Languages **Authors:** Wojciech Skut, Brigitte Krenn, [Thorsten Brants](https://md.research.elamur.ai/free/en/authors/thorsten-brants/) and [Hans Uszkoreit](https://md.research.elamur.ai/free/en/authors/hans-uszkoreit/) · **Year:** 1997 · **Venue:** Fifth Conference on Applied Natural Language Processing · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/A97-1014/) · [PDF](https://aclanthology.org/A97-1014.pdf) · [DOI 10.3115/974557.974571](https://doi.org/10.3115/974557.974571) · [OpenAlex](https://openalex.org/W1965364888) **Citations:** 301 citations (OpenAlex, 2026-10-14) · [107](https://md.research.elamur.ai/free/en/papers/skut-1997-annotation-scheme-free-word-order/cited-by/) in our library, of the 140,884 papers whose bibliographies we read ## Summary Treebank formats built for English rely on a context-free phrase-structure skeleton, with traces linking moved material. For free word order languages such as German, this design struggles. Discontinuous constituents are frequent, so traces multiply, and the resulting trees drift away from predicate-argument structure. The authors propose a different scheme that puts argument structure at the center. It uses flat, unordered trees whose branches may cross, and it has no empty categories. Grammatical functions such as subject, modifier and head are stored as edge labels, while syntactic categories are node labels and POS tags. When one element serves several roles, as with equi verbs, extra secondary edges are added. The authors also built an annotation tool that suggests function labels with a statistical tagger and asks the annotator to check uncertain choices. About 1100 German sentences have been annotated so far. The tagger reaches 95% overall accuracy, and annotation time per sentence drops by about 25%. The work addresses the lack of syntactically annotated German corpora and yields data that is closer to semantics and less tied to any one syntactic theory. ## Key points - The scheme has three strata: - argument structure as unordered trees with crossing branches; - grammatical functions on edges (e.g. SB, MO, HD, OC, OA, DA, GL, GR, PM, SVP, NK); - categories on nodes plus POS tags on words. - The tagset is small: 16 node labels and 34 function tags. There are no bar levels or intermediate projections, and no empty categories. - Function labels are assigned by a trigram tagger, evaluated with 10 random 90/10 train/test splits. - 90% of assignments are judged reliable and applied automatically, with 97% accuracy. - The remaining 10% are flagged for confirmation and are correct 75% of the time. - Overall accuracy is 95%. Most errors involve telling subjects apart from the various object types. - Partial automation cuts average annotation time from roughly 4 to roughly 3 minutes per sentence. - A planned follow-up in the Verbmobil project targets 30,000 spoken German sentences, plus a similar amount of English and Japanese. ## Interesting ideas - **Headedness as a label, not a structure.** Marking the head with an HD edge label lets headless constructions (e.g. copula-less predicatives) be annotated without inventing empty heads. This is useful for any scheme that must handle "deficient" constructions. - **Flat trees with rich labels.** Keeping the structure shallow and pushing detail into function labels reduces disagreement between annotators over attachment levels. The same trade-off applies when designing UD-style or custom treebanks. - **Flagging uncertain predictions.** The tool compares the best label's probability with its strongest competitor and asks the human only when the ratio is close. This cheap uncertainty check fits pre-annotation pipelines for many tasks. - **Two-stage annotation.** A first pass covers structure and coarse functions. Later passes add finer details such as thematic roles, scope and anaphora, one level at a time. This keeps early annotation fast and consistent. - **Annotating the common minimum.** Where theories disagree, such as NP heads or the DP analysis, only the analysis shared across them is encoded. Theory-specific representations can then be derived later, which makes the resource reusable by different communities. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Institutions - [Saarland University](https://md.research.elamur.ai/free/en/universities/saarland-university/) — University, Saarbrücken, Germany · 86 works · 15,761 citations ## Cite Cite as: Wojciech Skut, Brigitte Krenn, Thorsten Brants and Hans Uszkoreit. 1997. An Annotation Scheme for Free Word Order Languages. Fifth Conference on Applied Natural Language Processing. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{skut1997annotation, title = {{An Annotation Scheme for Free Word Order Languages}}, author = {Wojciech Skut, Brigitte Krenn, Thorsten Brants and Hans Uszkoreit}, booktitle = {Fifth Conference on Applied Natural Language Processing}, year = {1997}, doi = {10.3115/974557.974571}, url = {https://doi.org/10.3115/974557.974571} } ``` - [← previous Stochastic Inversion Transduction Grammars and Bilingual Parsing of Parallel Corpora](https://md.research.elamur.ai/free/en/papers/wu-1997-stochastic-inversion-transduction-grammars-bilingual/) - [next → Automatic Extraction of Subcategorization from Corpora](https://md.research.elamur.ai/free/en/papers/briscoe-1997-automatic-extraction-subcategorization-corpora/) - [↑ back to papers of 1997](https://md.research.elamur.ai/free/en/papers/1997/) ## References 13 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Ann Bies et al. 1995. BTuck~t, ing Guidelin~:.~ for Treebank H Slyh' Penn Treebank Project. Technical report, University of Pennsylvania. [find](https://scholar.google.com/scholar?q=Ann+Bies+et+al%2E+1995%2E+BTuck%7Et%2C+ing+Guidelin%7E%3A%2E%7E+for+Treebank+H+Slyh%27+Penn+Treebank+Project%2E+Technical+report%2C+University+of+Pennsylvania%2E) 2. Ezra Black et al. 1996. Beyond Skeleton Parsing: Producing a Comprehensive Large-Scale General-English Treehank With Full Grammatical Analysis. In Th.~: 16th Int.~:rnational Conference on Computal, ional Linguistics, pages 107 - 113, Copenhagen, Denmark. [find](https://scholar.google.com/scholar?q=Ezra+Black+et+al%2E+1996%2E+Beyond+Skeleton+Parsing%3A+Producing+a+Comprehensive+Large%2DScale+General%2DEnglish+Treehank+With+Full+Grammatical+Analysis%2E+In+Th%2E%7E%3A+16th+Int%2E%7E%3Arnational+Conference+on+Computal%2C+io) 3. Doug Cutting, Julian Kupiec, Jan Pedersen, and Penelope Sibun. 1992. A practical part-of-speech tagger. In Procteding~ o.f th( 3rd Confer~nc, ou Applied Natural Language Proc¢.ssing (ACL), pages 133-140. [our card](https://md.research.elamur.ai/free/en/papers/cutting-1992-practical-part-speech-tagger/) 4. David Dowty. 1989. Towards a minimalist theory of syntactic structure. In Tilburg Conference on Discontinuous Constituency. [find](https://scholar.google.com/scholar?q=David+Dowty%2E+1989%2E+Towards+a+minimalist+theory+of+syntactic+structure%2E+In+Tilburg+Conference+on+Discontinuous+Constituency%2E) 5. Helmut Feldweg. 1995. Implementation and evaluation of a German HMM for POS disambiguation. In Proceedings of EACL-SIGDAT-95 Workshop, Dublin, Ireland. 1984. Word Grammar. [find](https://scholar.google.com/scholar?q=Helmut+Feldweg%2E+1995%2E+Implementation+and+evaluation+of+a+German+HMM+for+POS+disambiguation%2E+In+Proceedings+of+EACL%2DSIGDAT%2D95+Workshop%2C+Dublin%2C+Ireland%2E+1984%2E+Word+Grammar%2E) 6. Richard Hudson. Blackwell Ltd. Fred Karlsson, Atro Voutilainen, J uha Heikkila, and Arto Anttila. 1995. (,'onstrai,.~ G'rammar. A Language-Independent System for Parsing Unre,-slricted Text. Mouton de Gruyter, Berlin, New York. [find](https://scholar.google.com/scholar?q=Richard+Hudson%2E+Blackwell+Ltd%2E+Fred+Karlsson%2C+Atro+Voutilainen%2C+J+uha+Heikkila%2C+and+Arto+Anttila%2E+1995%2E+%28%2C%27onstrai%2C%2E%7E+G%27rammar%2E+A+Language%2DIndependent+System+for+Parsing+Unre%2C%2Dslricted+Text%2E+Mouton+de) 7. Kathol, Andreas and Carl Pollard. 1995. Extra.position via Complex Domain Formation. In P~v-ceedings of the 33 ''~ Annual M~.eting of the: ACL, pages 174-180, C, ambridge, MA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kathol%2C+Andreas+and+Carl+Pollard%2E+1995%2E+Extra%2Eposition+via+Complex+Domain+Formation%2E+In+P%7Ev%2Dceedings+of+the+33+%27%27%7E+Annual+M%7E%2Eeting+of+the%3A+ACL%2C+pages+174%2D180%2C+C%2C+ambridge%2C+MA%2E+Association+for+Computat) 8. Sabine Lehmann et al. 1996. TSNLP - Test Suites for Natural Language Processing. In Th~ 16th lnle't'national (:onf~renc~ on Computational Li~.-guistics, pages 711 - 717, Copenhagen, Denmark. [find](https://scholar.google.com/scholar?q=Sabine+Lehmann+et+al%2E+1996%2E+TSNLP+%2D+Test+Suites+for+Natural+Language+Processing%2E+In+Th%7E+16th+lnle%27t%27national+%28%3Aonf%7Erenc%7E+on+Computational+Li%7E%2E%2Dguistics%2C+pages+711+%2D+717%2C+Copenhagen%2C+Denmark%2E) 9. Mitchell Marcus et al. 1994. The Penn Treebank: Annotating Predicate Argument Structure. In Proceedings of lhe Haman Language Technolog:t I Workshop, San Francisco. Morgan Kaufmann. [our card](https://md.research.elamur.ai/free/en/papers/marcus-1994-penn-treebank-annotating-predicate-argument/) 10. James McCawley. 1987. Some additional evidence for discontimfity. In Huck and Ojeda (eds.), Discontinuous Const.iluency: Synl.a.v and Semanf.ies, pp 185-200. New York, Academic Press. [find](https://scholar.google.com/scholar?q=James+McCawley%2E+1987%2E+Some+additional+evidence+for+discontimfity%2E+In+Huck+and+Ojeda+%28eds%2E%29%2C+Discontinuous+Const%2Eiluency%3A+Synl%2Ea%2Ev+and+Semanf%2Eies%2C+pp+185%2D200%2E+New+York%2C+Academic+Press%2E) 11. Mike Reape. 1993. A Formal Theory o] Word Ord~:r: A Ca.s~ ,gtudy iTt W~st. G~.r'm.,nw. P h D . thesis, University of Edinburgh. [find](https://scholar.google.com/scholar?q=Mike+Reape%2E+1993%2E+A+Formal+Theory+o%5D+Word+Ord%7E%3Ar%3A+A+Ca%2Es%7E+%2Cgtudy+iTt+W%7Est%2E+G%7E%2Er%27m%2E%2Cnw%2E+P+h+D+%2E+thesis%2C+University+of+Edinburgh%2E) 12. Geoffrey Sampson. 1995. E,gli.~h ]'or th~ Computer. The SUSANNE Corp',..~ and Analytic 5'cheme. Clarendon Press, Oxford. [find](https://scholar.google.com/scholar?q=Geoffrey+Sampson%2E+1995%2E+E%2Cgli%2E%7Eh+%5D%27or+th%7E+Computer%2E+The+SUSANNE+Corp%27%2C%2E%2E%7E+and+Analytic+5%27cheme%2E+Clarendon+Press%2C+Oxford%2E) 13. Atro Voutilainen. 1994. Designing a Parsing Grammar. University of Helsinki, Dept. of General Linguistics. Publications No. 22. [find](https://scholar.google.com/scholar?q=Atro+Voutilainen%2E+1994%2E+Designing+a+Parsing+Grammar%2E+University+of+Helsinki%2C+Dept%2E+of+General+Linguistics%2E+Publications+No%2E+22%2E)