---
title: "Parsing — methods, history and key papers — Elamur Research"
description: "Parsing is the automatic analysis of a sentence into its grammatical structure, usually a constituency tree of nested phrases or a dependency…"
canonical: "https://research.elamur.ai/free/en/concepts/parsing/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/concepts/parsing/)

[home](https://md.research.elamur.ai/free/en/) / [concepts](https://md.research.elamur.ai/free/en/concepts/) / Parsing

# 🪜 Parsing

*Parsing in linguistics and natural language processing — finding the grammatical structure of a sentence. Not the parsing of code or of data formats.*

Field · 49 key papers · [all concepts](https://md.research.elamur.ai/free/en/concepts/)

Part of: [🗨️ Natural language processing](https://md.research.elamur.ai/free/en/concepts/natural-language-processing/)

Parsing is the automatic analysis of a sentence into its grammatical structure, usually a constituency tree of nested phrases or a dependency tree that links each word to its head. By extension, it also covers mapping text to formal meaning representations and other structures.

Also: syntactic parsing, syntactic analysis, natural-language parsing

About 16 minutes to read · every paper named here has its own page in this library

**On this page**

1. [What it is](#a-what-it-is)
2. [How it works](#a-how-it-works)
3. [History](#a-history)
4. [Methods](#a-methods)
5. [Datasets and benchmarks](#a-datasets-and-benchmarks)
6. [People and institutions](#a-people-and-institutions)
7. [Open questions](#a-open-questions)
8. [The library in figures](#a-in-figures)

## What it is

A sentence has a shape that the string of words does not show. In "The cat chased a mouse", a reader sees at once that "the cat" forms a unit, that the cat does the chasing, and that "a mouse" is what is chased. Parsing makes this hidden shape explicit. A parser takes a sentence and returns a structure that says how its words combine. It is one of the oldest problems in [natural language processing](https://md.research.elamur.ai/free/en/concepts/natural-language-processing/) and computational linguistics. The word is shared with compilers, which parse program text against the grammar of a programming language.

Two representations of syntax dominate. A constituency, or phrase-structure, tree groups words into nested phrases with category labels such as noun phrase (NP) and verb phrase (VP), in the way a [context-free grammar](https://md.research.elamur.ai/free/en/concepts/context-free-grammar/) licenses them. A dependency tree links each word to the one word it modifies, called its head, and labels each link with a grammatical relation. Building such trees is [dependency parsing](https://md.research.elamur.ai/free/en/concepts/dependency-parsing/). For the example sentence:

```
Constituency:   (S (NP (DT The) (NN cat))
                   (VP (VBD chased)
                       (NP (DT a) (NN mouse))))

Dependency:     ROOT   → chased
                chased → cat    (nsubj)
                cat    → The    (det)
                chased → mouse  (obj)
                mouse  → a      (det)
```

In the constituency tree, DT, NN and VBD are tags of the kind assigned in [part-of-speech tagging](https://md.research.elamur.ai/free/en/concepts/part-of-speech-tagging/), and S, NP and VP are phrase labels. In the dependency analysis, each line is an arc from a head to its dependent. The labels mark the subject (nsubj), the object (obj) and the determiners (det), and "chased" hangs from an artificial ROOT. Here the two analyses carry nearly the same information. One makes phrases explicit; the other makes word-to-word relations explicit.

The central difficulty is ambiguity. A wide-coverage grammar can license a large number of analyses even for a short sentence, so a practical parser has to rank candidates rather than merely list them ([Collins 2003](https://md.research.elamur.ai/free/en/papers/collins-2003-head-driven-statistical-models-natural/)). Parses matter because applications need predicate–argument structure: who did what to whom ([Collins 1997](https://md.research.elamur.ai/free/en/papers/collins-1997-three-generative-lexicalised-models-statistical/)). In [semantic role labeling](https://md.research.elamur.ai/free/en/concepts/semantic-role-labeling/), for instance, full syntactic parses help most at the stage that prunes argument candidates ([Punyakanok et al. 2008](https://md.research.elamur.ai/free/en/papers/punyakanok-2008-importance-syntactic-parsing-inference-semantic/)). The term has also widened. [Semantic parsing](https://md.research.elamur.ai/free/en/concepts/semantic-parsing/) maps sentences to executable meaning, and "parsing" now also names the recovery of discourse, argument and table structure.

The shelves its papers stand on Language and Linguistics: 612 of the 625 papers here.

**The numbers**

| Shelf | Papers |
| --- | --- |
| [Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) | 612 papers |
| [Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) | 607 papers |
| [Philosophy](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/philosophy/) | 35 papers |
| [Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) | 16 papers |
| [Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) | 9 papers |
| [Literature and Literary Theory](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/literature-and-literary-theory/) | 9 papers |
| [Computational Theory and Mathematics](https://md.research.elamur.ai/free/en/shelves/computer-science/computational-theory-and-mathematics/) | 5 papers |
| [Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) | 5 papers |
| [Molecular Biology](https://md.research.elamur.ai/free/en/shelves/biochemistry-genetics-and-molecular-biology/molecular-biology/) | 5 papers |
| [Computer Networks and Communications](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-networks-and-communications/) | 3 papers |
| [Cognitive Neuroscience](https://md.research.elamur.ai/free/en/shelves/neuroscience/cognitive-neuroscience/) | 2 papers |
| [Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) | 2 papers |

## How it works

### Dynamic programming over a chart

The classic algorithm for a context-free grammar in Chomsky normal form, where every rule rewrites a nonterminal either as two nonterminals or as a single word, is CKY. It is named after Cocke, Kasami and Younger (1960s). It fills a triangular chart from short spans to long ones. For a probabilistic context-free grammar (PCFG), the Viterbi version is:

```
β(i, i+1, A) = P(A → wᵢ₊₁)
β(i, j, A)   = max over rules A → B C and split points i < k < j of
               P(A → B C) · β(i, k, B) · β(k, j, C)
```

β(i, j, A) is the probability of the best subtree of category A that covers words i+1 through j. A, B and C are nonterminals, k is the point where the span splits into two children, P(A → B C) is a rule probability, and wᵢ₊₁ is a word of the sentence. Back-pointers stored with each cell recover the best tree. Replacing max with a sum gives the inside probability, which training by expectation–maximisation (EM) needs. The run time grows with the cube of sentence length times the grammar size.

Chart parsing generalises this idea. The chart stores partial constituents, so no sub-analysis is built twice. Earley's algorithm (Earley, 1970) works left to right with "dotted" rules through three operations: predict, scan and complete. It handles any context-free grammar without binarisation. Its probabilistic form yields prefix probabilities. Hale used Stolcke's probabilistic Earley parser to compute surprisal, the log-ratio of successive prefix probabilities, as a measure of word-by-word processing load. He showed that it predicts garden-path effects in reduced relatives and the extra difficulty of object relatives ([Hale 2001](https://md.research.elamur.ai/free/en/papers/hale-2001-probabilistic-earley-parser-as-psycholinguistic/)). This made parsing a tool of [psycholinguistics](https://md.research.elamur.ai/free/en/concepts/psycholinguistics/).

### Transition-based (shift-reduce) parsing

A transition-based parser does not search the space of trees. It reads the sentence once, keeps a stack of partly processed words and a buffer of unread words, and applies one action at a time. Nivre's deterministic algorithm, the arc-eager system, uses four actions (Left-Arc, Right-Arc, Reduce, Shift). It runs in time linear in sentence length and always produces a projective, acyclic graph ([Nivre 2003](https://md.research.elamur.ai/free/en/papers/nivre-2003-efficient-algorithm-projective-dependency-parsing/)). MaltParser turned this into a parser generator. It offers arc-eager and arc-standard variants, and a trained classifier picks the next action from features of the current state ([Nivre et al. 2006](https://md.research.elamur.ai/free/en/papers/nivre-2006-maltparser-data-driven-parser-generator/)). The arc-standard system has no Reduce action. With an artificial ROOT kept at the bottom of the stack, it builds the example tree in ten steps, two per word:

| Stack | Buffer | Action | Arc added |
| --- | --- | --- | --- |
| [ROOT] | The cat chased a mouse | SHIFT |  |
| [ROOT, The] | cat chased a mouse | SHIFT |  |
| [ROOT, The, cat] | chased a mouse | LEFT-ARC | cat → The (det) |
| [ROOT, cat] | chased a mouse | SHIFT |  |
| [ROOT, cat, chased] | a mouse | LEFT-ARC | chased → cat (nsubj) |
| [ROOT, chased] | a mouse | SHIFT |  |
| [ROOT, chased, a] | mouse | SHIFT |  |
| [ROOT, chased, a, mouse] | — | LEFT-ARC | mouse → a (det) |
| [ROOT, chased, mouse] | — | RIGHT-ARC | chased → mouse (obj) |
| [ROOT, chased] | — | RIGHT-ARC | ROOT → chased |

LEFT-ARC makes the top of the stack the head of the word beneath it and pops the dependent. RIGHT-ARC does the opposite. Greedy decisions are fast, but an early mistake cannot be undone. Two fixes have been studied. Non-monotonic systems let the parser repair earlier attachments ([Honnibal & Johnson 2015](https://md.research.elamur.ai/free/en/papers/honnibal-2015-improved-non-monotonic-transition-system/)). Beam search with globally normalised training scores whole action sequences rather than single steps ([Andor et al. 2016](https://md.research.elamur.ai/free/en/papers/andor-2016-globally-normalized-transition-based-neural/)).

### Graph-based parsing

Graph-based parsers score every possible head–dependent arc and then search for the best tree as a whole:

```
s(y) = Σ over arcs (h, m) in y of  w · f(h, m, x)
ŷ    = argmax over trees y of sentence x of  s(y)
```

Here x is the sentence, y a candidate tree, (h, m) an arc from head h to modifier m, f a feature vector describing that arc, and w the learned weights. Because the score splits into a sum over arcs, the best tree is a maximum spanning tree (MST) of the directed graph over the words. Eisner gave an O(n³) dynamic programme for projective trees, where n is sentence length ([Eisner 1996](https://md.research.elamur.ai/free/en/papers/eisner-1996-three-new-probabilistic-models-dependency/)). McDonald, Crammer and Pereira trained arc weights on top of it with the online large-margin method MIRA (Margin Infused Relaxed Algorithm) ([McDonald et al. 2005](https://md.research.elamur.ai/free/en/papers/mcdonald-2005-online-large-margin-training-dependency/)). McDonald, Pereira, Ribarov and Hajič then showed that non-projective trees, whose arcs cross, can be found exactly with the Chu–Liu–Edmonds algorithm in O(n²) ([McDonald et al. 2005](https://md.research.elamur.ai/free/en/papers/mcdonald-2005-non-projective-dependency-parsing-spanning/)). Richer scores break this exactness. With second-order features, projective parsing stays cubic, but non-projective parsing becomes NP-hard and needs approximate search ([McDonald & Pereira 2006](https://md.research.elamur.ai/free/en/papers/mcdonald-2006-online-learning-approximate-dependency-parsing/)).

## History

Early broad-coverage systems relied on hand-written grammars. Statistics entered through shallow analysis. Church's 1988 tagger chose tag sequences by dynamic programming over trigram statistics from the tagged Brown Corpus, and it also bracketed simple noun phrases ([Church 1988](https://md.research.elamur.ai/free/en/papers/church-1988-stochastic-parts-program-noun-phrase/)). The turning point was the Penn Treebank. It covers over 4.5 million words of American English. All of it is tagged for part of speech and more than half is bracketed for syntax. It soon became the standard training and test material for English ([Marcus et al. 1993](https://md.research.elamur.ai/free/en/papers/marcus-1993-building-large-annotated-corpus-english/)).

Landmark papers on this subject: the 12 most cited, in time The 12 most cited papers on this subject span 1993–2016; the dot grows with the citations.

**The numbers**

| Paper | Citations |
| --- | --- |
| [1993 — Building a Large Annotated Corpus of English: The Penn Treebank](https://md.research.elamur.ai/free/en/papers/marcus-1993-building-large-annotated-corpus-english/) | 3,024 |
| [1998 — Automatic Retrieval and Clustering of Similar Words](https://md.research.elamur.ai/free/en/papers/lin-1998-automatic-retrieval-clustering-similar-words/) | 2,062 |
| [2000 — A Maximum-Entropy-Inspired Parser](https://md.research.elamur.ai/free/en/papers/charniak-2000-maximum-entropy-inspired-parser/) | 2,050 |
| [2003 — Accurate Unlexicalized Parsing](https://md.research.elamur.ai/free/en/papers/klein-2003-accurate-unlexicalized-parsing/) | 3,942 |
| [2003 — Head-Driven Statistical Models for Natural Language Parsing](https://md.research.elamur.ai/free/en/papers/collins-2003-head-driven-statistical-models-natural/) | 2,001 |
| [2005 — The Proposition Bank: An Annotated Corpus of Semantic Roles](https://md.research.elamur.ai/free/en/papers/palmer-2005-proposition-bank-annotated-corpus-semantic/) | 3,212 |
| [2013 — Semantic Parsing on Freebase from Question-Answer Pairs](https://md.research.elamur.ai/free/en/papers/berant-2013-semantic-parsing-freebase-question-answer/) | 2,305 |
| [2013 — Abstract Meaning Representation for Sembanking](https://md.research.elamur.ai/free/en/papers/banarescu-2013-abstract-meaning-representation-sembanking/) | 1,919 |
| [2014 — A Fast and Accurate Dependency Parser using Neural Networks](https://md.research.elamur.ai/free/en/papers/chen-2014-fast-accurate-dependency-parser-neural/) | 2,309 |
| [2014 — Relation Classification via Convolutional Deep Neural Network](https://md.research.elamur.ai/free/en/papers/zeng-2014-relation-classification-convolutional-deep-neural/) | 1,670 |
| [2016 — Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification](https://md.research.elamur.ai/free/en/papers/zhou-2016-attention-based-bidirectional-long-short/) | 2,218 |
| [2016 — Universal Dependencies v1: A Multilingual Treebank Collection](https://md.research.elamur.ai/free/en/papers/nivre-2016-universal-dependencies-v1-multilingual-treebank/) | 1,741 |

Learned parsers followed. Magerman's SPATTER learned every parsing choice with decision trees and outperformed a long-developed grammar-based IBM parser ([Magerman 1995](https://md.research.elamur.ai/free/en/papers/magerman-1995-statistical-decision-tree-models-parsing/)). Collins' head-driven generative models ([Collins 1997](https://md.research.elamur.ai/free/en/papers/collins-1997-three-generative-lexicalised-models-statistical/)) and Charniak's maximum-entropy-inspired parser ([Charniak 2000](https://md.research.elamur.ai/free/en/papers/charniak-2000-maximum-entropy-inspired-parser/)) defined the lexicalised era. Klein and Manning then narrowed the gap with unlexicalised grammars ([Klein & Manning 2003](https://md.research.elamur.ai/free/en/papers/klein-2003-accurate-unlexicalized-parsing/)), and Petrov and colleagues automated their annotation ([Petrov et al. 2006](https://md.research.elamur.ai/free/en/papers/petrov-2006-learning-accurate-compact-interpretable-tree/)). Discriminative rerankers ([Collins & Koo 2005](https://md.research.elamur.ai/free/en/papers/collins-2000-discriminative-reranking-natural-language-parsing/); [Charniak & Johnson 2005](https://md.research.elamur.ai/free/en/papers/charniak-2005-coarse-fine-n-best-parsing/)) and self-training on unlabelled news ([McClosky et al. 2006](https://md.research.elamur.ai/free/en/papers/mcclosky-2006-effective-self-training-parsing/)) pushed treebank accuracy further.

Papers on this subject by year 625 dated papers from 1987 to 2026; the busiest year is 2020 with 48, and 36% of them appeared in 2018 or later.

**The numbers**

| Year | Papers |
| --- | --- |
| 2026 | 25 |
| 2025 | 10 |
| 2024 | 14 |
| 2023 | 17 |
| 2022 | 18 |
| 2021 | 32 |
| 2020 | 48 |
| 2019 | 27 |
| 2018 | 31 |
| 2017 | 43 |
| 2016 | 42 |
| 2015 | 44 |
| 2014 | 33 |
| 2013 | 21 |
| 2012 | 12 |
| 2011 | 19 |
| 2010 | 28 |
| 2009 | 21 |
| 2008 | 21 |
| 2007 | 15 |
| 2006 | 16 |
| 2005 | 10 |
| 2004 | 15 |
| 2003 | 5 |
| 2002 | 8 |
| 2001 | 7 |
| 2000 | 4 |
| 1999 | 6 |
| 1998 | 5 |
| 1997 | 6 |
| 1996 | 4 |
| 1995 | 3 |
| 1994 | 4 |
| 1993 | 3 |
| 1990 | 4 |
| 1988 | 2 |
| 1987 | 2 |

Dependency parsing grew alongside. After Eisner's algorithm ([Eisner 1996](https://md.research.elamur.ai/free/en/papers/eisner-1996-three-new-probabilistic-models-dependency/)) came Nivre's linear-time parser ([Nivre 2003](https://md.research.elamur.ai/free/en/papers/nivre-2003-efficient-algorithm-projective-dependency-parsing/)) and McDonald's spanning-tree parsers ([McDonald et al. 2005](https://md.research.elamur.ai/free/en/papers/mcdonald-2005-non-projective-dependency-parsing-spanning/)). The CoNLL-X shared task of 2006 and its 2007 successor gave dependency parsers common multilingual benchmarks ([Buchholz & Marsi 2006](https://md.research.elamur.ai/free/en/papers/buchholz-2006-conll-x-shared-task-multilingual/); [Nivre et al. 2007](https://md.research.elamur.ai/free/en/papers/nivre-2007-conll-2007-shared-task-dependency/)). A harmonised cross-lingual annotation followed in 2013 ([McDonald et al. 2013](https://md.research.elamur.ai/free/en/papers/mcdonald-2013-universal-dependency-annotation-multilingual-parsing/)) and grew into [Universal Dependencies](https://md.research.elamur.ai/free/en/concepts/universal-dependencies/).

The neural turn began when Chen and Manning replaced sparse feature templates with dense embeddings in a small feed-forward network ([Chen & Manning 2014](https://md.research.elamur.ai/free/en/papers/chen-2014-fast-accurate-dependency-parser-neural/)). Recurrent encoders ([Dyer et al. 2015](https://md.research.elamur.ai/free/en/papers/dyer-2015-transition-based-dependency-parsing-stack/); [Kiperwasser & Goldberg 2016](https://md.research.elamur.ai/free/en/papers/kiperwasser-2016-simple-accurate-dependency-parsing-bidirectional/)), biaffine scoring ([Dozat & Manning 2016](https://md.research.elamur.ai/free/en/papers/dozat-2016-deep-biaffine-attention-neural-dependency/)) and self-attention ([Kitaev & Klein 2018](https://md.research.elamur.ai/free/en/papers/kitaev-2018-constituency-parsing-self-attentive-encoder/)) came next. Semantic parsing, meanwhile, scaled training from question–answer pairs alone, with logical forms left latent, to a large knowledge base, Freebase ([Berant et al. 2013](https://md.research.elamur.ai/free/en/papers/berant-2013-semantic-parsing-freebase-question-answer/)).

Citations of the papers on this subject, by when they were published Papers published in 2015–2019 gather 28% of the 152,734 citations of the papers on this subject.

**The numbers**

| Published | Papers and their citations |
| --- | --- |
| 2025–2029 | 35 papers, 20 citations |
| 2020–2024 | 129 papers, 10,506 citations |
| 2015–2019 | 187 papers, 42,518 citations |
| 2010–2014 | 113 papers, 26,179 citations |
| 2005–2009 | 83 papers, 31,681 citations |
| 2000–2004 | 39 papers, 20,711 citations |
| 1995–1999 | 24 papers, 12,684 citations |
| 1990–1994 | 11 papers, 6,996 citations |
| 1985–1989 | 4 papers, 1,439 citations |

## Methods

### Probabilistic and lexicalised grammars

A plain PCFG read off a treebank parses poorly, because its independence assumptions ignore which words fill a phrase. Collins lexicalised each nonterminal with a head word and tag and generated trees top-down, outward from each head. Model 1 conditions modifiers on a distance measure. Model 2 adds complements and subcategorisation frames. Model 3 adds traces of wh-movement ([Collins 1997](https://md.research.elamur.ai/free/en/papers/collins-1997-three-generative-lexicalised-models-statistical/)). The journal version reported 88.3% precision and 88.0% recall for Models 2 and 3 on Wall Street Journal (WSJ) section 23, counting sentences of up to 100 words ([Collins 2003](https://md.research.elamur.ai/free/en/papers/collins-2003-head-driven-statistical-models-natural/)). Charniak's parser predicts each constituent's head tag, then its head word, then its expansion. It conditions these probabilities through products of ratios borrowed from log-linear modelling, and it cut errors by 13% relative to Collins ([Charniak 2000](https://md.research.elamur.ai/free/en/papers/charniak-2000-maximum-entropy-inspired-parser/)).

### Unlexicalised and latent-annotation grammars

Klein and Manning attributed much of the benefit of lexicalisation to structural context. Parent annotation, head-outward markovisation and hand-picked splits, such as separating finite from non-finite VPs, lifted an unsmoothed treebank grammar to about 86.3 F1 on short sentences ([Klein & Manning 2003](https://md.research.elamur.ai/free/en/papers/klein-2003-accurate-unlexicalized-parsing/)). Petrov, Barrett, Thibaux and Klein learned such splits automatically. Each symbol is split in two, the subsymbols are trained with EM over the known trees, and splits that add little likelihood are merged back. The resulting grammar of 1043 symbols outperformed fully lexicalised parsers, and its subcategories remained readable to linguists ([Petrov et al. 2006](https://md.research.elamur.ai/free/en/papers/petrov-2006-learning-accurate-compact-interpretable-tree/)). These grammars were slow, so Petrov and Klein pruned with the coarser grammars of earlier splitting rounds. Parsing ran up to 100 times faster with no loss of accuracy, and the method carried over to German and Chinese without tuning ([Petrov & Klein 2007](https://md.research.elamur.ai/free/en/papers/petrov-2007-improved-inference-unlexicalized-parsing/)).

### Discriminative training, reranking and semi-supervision

Generative models limit which features can be used. A reranker escapes this limit by rescoring a parser's n-best list with arbitrary global features ([Collins & Koo 2005](https://md.research.elamur.ai/free/en/papers/collins-2000-discriminative-reranking-natural-language-parsing/)). Charniak and Johnson paired coarse-to-fine n-best parsing with a maximum-entropy reranker ([Charniak & Johnson 2005](https://md.research.elamur.ai/free/en/papers/charniak-2005-coarse-fine-n-best-parsing/)). Collins and Roark trained an incremental parser with the perceptron ([Collins & Roark 2004](https://md.research.elamur.ai/free/en/papers/collins-2004-incremental-parsing-perceptron-algorithm/)). Self-training was long thought useless for parsers. McClosky, Charniak and Johnson made it work by retraining the first-stage parser on reranker-selected parses of unlabelled news ([McClosky et al. 2006](https://md.research.elamur.ai/free/en/papers/mcclosky-2006-effective-self-training-parsing/)). For dependencies, Brown word clusters learned from raw text roughly halved the annotated data needed to reach a given accuracy ([Koo et al. 2008](https://md.research.elamur.ai/free/en/papers/koo-2008-simple-semi-supervised-dependency-parsing/)). Bohnet showed that most of the run time of a feature-rich graph-based parser went to looking up feature weights. Hashing features straight to weight positions made his parser both faster and more accurate ([Bohnet 2010](https://md.research.elamur.ai/free/en/papers/bohnet-2010-top-accuracy-fast-dependency-parsing/)).

### Lexicalised grammar formalisms

Richer formalisms put most of the grammar in the lexicon. In Lexicalised Tree-Adjoining Grammar, each word selects an elementary tree, or supertag. Choosing supertags by statistical tagging leaves an "almost parse" that a light step completes, and disambiguation accuracy rose from 68% to 92% ([Bangalore & Joshi 1999](https://md.research.elamur.ai/free/en/papers/bangalore-1999-supertagging-approach-almost-parsing/)). Clark and Curran combined Combinatory Categorial Grammar (CCG) with log-linear models to build a wide-coverage and efficient statistical parser ([Clark & Curran 2007](https://md.research.elamur.ai/free/en/papers/clark-2007-wide-coverage-efficient-statistical-parsing/)).

### Neural parsers

Chen and Manning embedded the words, tags and arc labels found at selected stack and buffer positions. A single hidden layer with a cube activation, g(x) = x³, captures products of three inputs, and a softmax then chooses the next transition. The parser was more accurate than sparse-feature greedy baselines and processed over 1000 sentences per second ([Chen & Manning 2014](https://md.research.elamur.ai/free/en/papers/chen-2014-fast-accurate-dependency-parser-neural/)). Dyer and colleagues introduced the stack LSTM. It is a long short-term memory (LSTM) network that supports push and pop while keeping a vector summary of the whole stack. With it, the parser could encode its full state, including the buffer and the history of actions ([Dyer et al. 2015](https://md.research.elamur.ai/free/en/papers/dyer-2015-transition-based-dependency-parsing-stack/)).

Kiperwasser and Goldberg showed that a few bidirectional LSTM (BiLSTM) vectors per decision can replace large feature sets in both transition-based and graph-based parsers. They trained the encoder jointly with a structured loss ([Kiperwasser & Goldberg 2016](https://md.research.elamur.ai/free/en/papers/kiperwasser-2016-simple-accurate-dependency-parsing-bidirectional/)). Dozat and Manning added biaffine attention:

```
s(h, d) = r_hᵀ U r_d + r_hᵀ u
```

r_h and r_d are the outputs of two small feed-forward layers that view a BiLSTM state as a potential head and as a potential dependent. U is a weight matrix, and u is a weight vector. The first term scores how well a particular head–dependent pair fits. The second scores how readily word h takes dependents at all. A second biaffine classifier assigns labels, and an MST decoder guarantees a well-formed tree ([Dozat & Manning 2016](https://md.research.elamur.ai/free/en/papers/dozat-2016-deep-biaffine-attention-neural-dependency/)).

For constituency, Kitaev and Klein replaced the LSTM encoder of a chart-based span parser with self-attention adapted from the [Transformer](https://md.research.elamur.ai/free/en/concepts/transformer/). They found that keeping content and position in separate channels raised accuracy ([Kitaev & Klein 2018](https://md.research.elamur.ai/free/en/papers/kitaev-2018-constituency-parsing-self-attentive-encoder/)). The biaffine scorer has itself been reused outside syntax. It treats nested [named entity recognition](https://md.research.elamur.ai/free/en/concepts/named-entity-recognition/) as scoring start–end token pairs ([Yu et al. 2020](https://md.research.elamur.ai/free/en/papers/yu-2020-named-entity-recognition-as-dependency/)).

### Semantic parsing

A semantic parser maps an utterance to a logical form or query whose execution gives an answer. Berant and colleagues trained a Freebase parser from question–answer pairs and treated the logical forms as latent. A lexicon aligned from web text and a "bridging" operation connected phrases to predicates ([Berant et al. 2013](https://md.research.elamur.ai/free/en/papers/berant-2013-semantic-parsing-freebase-question-answer/)). Berant and Liang later recast the task as paraphrasing. Candidate logical forms are turned into canonical sentences, and a paraphrase model picks the closest one ([Berant & Liang 2014](https://md.research.elamur.ai/free/en/papers/berant-2014-semantic-parsing-paraphrasing/)). Yih and colleagues built query graphs in stages. They start from a topic entity and then add a relation chain, constraints and aggregation, so that the [knowledge graph](https://md.research.elamur.ai/free/en/concepts/knowledge-graph/) prunes the search early ([Yih et al. 2015](https://md.research.elamur.ai/free/en/papers/yih-2015-semantic-parsing-staged-query-graph/)).

Pasupat and Liang moved to Wikipedia tables never seen in training ([Pasupat & Liang 2015](https://md.research.elamur.ai/free/en/papers/pasupat-2015-compositional-semantic-parsing-semi-structured/)). Artzi and Zettlemoyer learned a weighted CCG parser for navigation instructions, using only the success of executing them as supervision ([Artzi & Zettlemoyer 2013](https://md.research.elamur.ai/free/en/papers/artzi-2013-weakly-supervised-learning-semantic-parsers/)). Text-to-SQL work, the branch that reaches into [code and programming](https://md.research.elamur.ai/free/en/topics/code-and-programming/), centres on unseen database schemas. RAT-SQL encodes schema relations and links between question words and columns inside relation-aware self-attention ([Wang et al. 2020](https://md.research.elamur.ai/free/en/papers/wang-2020-rat-sql-relation-aware-schema/)).

### Parsing in the age of language models

Pretrained encoders have changed the input representations of parsers, while the decoding algorithms survive. In the strongest systems, CKY, MST and transition decoders are fed contextual vectors from a [language model](https://md.research.elamur.ai/free/en/concepts/language-model/). ELMo (Embeddings from Language Models) inputs lifted the self-attentive constituency parser in this way ([Kitaev & Klein 2018](https://md.research.elamur.ai/free/en/papers/kitaev-2018-constituency-parsing-self-attentive-encoder/)), and BERT (Bidirectional Encoder Representations from Transformers) lifted RAT-SQL ([Wang et al. 2020](https://md.research.elamur.ai/free/en/papers/wang-2020-rat-sql-relation-aware-schema/)). Some tasks drop explicit structure entirely. TaPas answers table questions by selecting cells and an aggregation operator with a BERT-style encoder, with no logical form at all ([Herzig et al. 2020](https://md.research.elamur.ai/free/en/papers/herzig-2020-tapas-weakly-supervised-table-parsing/)). Pipelines such as UDPipe, meanwhile, made tokenisation, tagging, lemmatisation and dependency parsing a trainable commodity for dozens of languages ([Straka & Straková 2017](https://md.research.elamur.ai/free/en/papers/straka-2017-tokenizing-pos-tagging-lemmatizing-parsing/)). Structure prediction above the sentence follows the same pattern. Examples are discourse trees built from syntactic cues ([Soricut & Marcu 2003](https://md.research.elamur.ai/free/en/papers/soricut-2003-sentence-level-discourse-parsing-syntactic/)) and argument structure in essays, recovered with joint integer linear programming (ILP) inference ([Stab & Gurevych 2017](https://md.research.elamur.ai/free/en/papers/stab-2017-parsing-argumentation-structures-persuasive-essays/)). This work connects parsing to [discourse analysis](https://md.research.elamur.ai/free/en/concepts/discourse-analysis/).

## Datasets and benchmarks

### Evaluation measures

Constituency parsers are scored with the PARSEVAL bracket measures (Black et al., 1991): precision, recall and F1. These are usually computed with the EVALB tool, as in the latent-annotation experiments ([Petrov et al. 2006](https://md.research.elamur.ai/free/en/papers/petrov-2006-learning-accurate-compact-interpretable-tree/)). Dependency parsers are scored by the attachment measures UAS and LAS:

```
P   = correct labelled brackets / brackets in the parser's tree
R   = correct labelled brackets / brackets in the gold tree
F1  = 2 · P · R / (P + R)
UAS = tokens with the correct head / scored tokens
LAS = tokens with the correct head and relation label / scored tokens
```

A bracket is a constituent identified by its label and the span of words it covers. It counts as correct when the gold tree has the same bracket. P is precision, R recall, and F1 their harmonic mean. UAS and LAS are the unlabelled and labelled attachment scores. Conventions differ, and the differences matter. CoNLL-X made LAS its official metric and left punctuation out ([Buchholz & Marsi 2006](https://md.research.elamur.ai/free/en/papers/buchholz-2006-conll-x-shared-task-multilingual/)). CoNLL 2007 counted every token, punctuation included ([Nivre et al. 2007](https://md.research.elamur.ai/free/en/papers/nivre-2007-conll-2007-shared-task-dependency/)). Constituency results are often reported separately for sentences of at most 40 words and for longer limits ([Charniak 2000](https://md.research.elamur.ai/free/en/papers/charniak-2000-maximum-entropy-inspired-parser/)). Semantic parsers are scored by denotation accuracy, F1 over answers, or exact match of the produced query.

### Treebanks and shared tasks

The WSJ part of the Penn Treebank anchors English parsing ([Marcus et al. 1993](https://md.research.elamur.ai/free/en/papers/marcus-1993-building-large-annotated-corpus-english/)). Its usual split trains on sections 2–21 and tests on section 23 ([Klein & Manning 2003](https://md.research.elamur.ai/free/en/papers/klein-2003-accurate-unlexicalized-parsing/)). Some representative results on that test set follow. Their conditions differ, so they are not strictly comparable.

| System | Condition | Score |
| --- | --- | --- |
| Collins 1997 | constituent precision/recall, final system | 88.1 / 87.5 ([Collins 1997](https://md.research.elamur.ai/free/en/papers/collins-1997-three-generative-lexicalised-models-statistical/)) |
| Charniak 2000 | average of precision and recall, sentences ≤ 40 words | 90.1 ([Charniak 2000](https://md.research.elamur.ai/free/en/papers/charniak-2000-maximum-entropy-inspired-parser/)) |
| Petrov et al. 2006 | F1, sentences ≤ 40 words; LP / LR, all sentences | 90.2; 89.8 / 89.6 ([Petrov et al. 2006](https://md.research.elamur.ai/free/en/papers/petrov-2006-learning-accurate-compact-interpretable-tree/)) |
| McClosky et al. 2006 | f-score, self-trained | 92.1 ([McClosky et al. 2006](https://md.research.elamur.ai/free/en/papers/mcclosky-2006-effective-self-training-parsing/)) |
| Kitaev and Klein 2018 | F1, with ELMo | 95.13 ([Kitaev & Klein 2018](https://md.research.elamur.ai/free/en/papers/kitaev-2018-constituency-parsing-self-attentive-encoder/)) |

The same trees, converted to Stanford dependencies, serve as the main English dependency benchmark:

| System | Type | UAS / LAS |
| --- | --- | --- |
| Dyer et al. 2015 | stack-LSTM transition | 93.1 / 90.9 ([Dyer et al. 2015](https://md.research.elamur.ai/free/en/papers/dyer-2015-transition-based-dependency-parsing-stack/)) |
| Kiperwasser and Goldberg 2016 | greedy BiLSTM, pretrained vectors | 93.9 UAS ([Kiperwasser & Goldberg 2016](https://md.research.elamur.ai/free/en/papers/kiperwasser-2016-simple-accurate-dependency-parsing-bidirectional/)) |
| Andor et al. 2016 | globally normalised | 94.61 UAS ([Andor et al. 2016](https://md.research.elamur.ai/free/en/papers/andor-2016-globally-normalized-transition-based-neural/)) |
| Dozat and Manning 2016 | biaffine graph-based | 95.7 / 94.1 ([Dozat & Manning 2016](https://md.research.elamur.ai/free/en/papers/dozat-2016-deep-biaffine-attention-neural-dependency/)) |

The Prague Dependency Treebank is the classic test of non-projective parsing. On Czech, the spanning-tree parser beat its projective counterpart, with the largest gains on sentences that contain crossing arcs ([McDonald et al. 2005](https://md.research.elamur.ai/free/en/papers/mcdonald-2005-non-projective-dependency-parsing-spanning/)).

Shared tasks built the multilingual landscape. In CoNLL-X, 19 groups parsed 13 languages, and scores ranged from 37.8% LAS on Turkish to 91.7% on Japanese ([Buchholz & Marsi 2006](https://md.research.elamur.ai/free/en/papers/buchholz-2006-conll-x-shared-task-multilingual/)). CoNLL 2007 covered ten languages and a domain adaptation track ([Nivre et al. 2007](https://md.research.elamur.ai/free/en/papers/nivre-2007-conll-2007-shared-task-dependency/)). CoNLL 2008 joined syntactic dependencies with PropBank and NomBank roles ([Surdeanu et al. 2008](https://md.research.elamur.ai/free/en/papers/surdeanu-2008-conll-2008-shared-task-joint/)). The CoNLL 2017 task parsed raw text with Universal Dependencies annotation. Systems trained on 64 treebanks in 45 languages and were evaluated on 81 test sets, some of them for surprise languages with almost no training data. The winning system came from Stanford ([Zeman et al. 2017](https://md.research.elamur.ai/free/en/papers/zeman-2017-conll-2017-shared-task-multilingual/)). Shared tasks like these are the main reference points for [evaluation and benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) in parsing, alongside the [treebanks](https://md.research.elamur.ai/free/en/topics/treebanks/) behind them.

### Semantic parsing benchmarks

Free917 and WebQuestions test parsing over Freebase ([Berant et al. 2013](https://md.research.elamur.ai/free/en/papers/berant-2013-semantic-parsing-freebase-question-answer/)). WikiTableQuestions contains 22,033 questions over 2,108 Wikipedia tables ([Pasupat & Liang 2015](https://md.research.elamur.ai/free/en/papers/pasupat-2015-compositional-semantic-parsing-semi-structured/)). Spider pairs 10,181 questions with 5,693 SQL queries over 200 databases. When it appeared, the best model reached only 9.7% exact match on the split whose test databases are unseen ([Yu et al. 2018](https://md.research.elamur.ai/free/en/papers/yu-2018-spider-large-scale-human-labeled/)). Two years later, RAT-SQL reached 65.6% on the Spider test set with BERT ([Wang et al. 2020](https://md.research.elamur.ai/free/en/papers/wang-2020-rat-sql-relation-aware-schema/)).

Where the papers on this subject were published Annual Meeting of the Association for Computational Linguistics (ACL) carries the most, 137 of 625.

**The numbers**

| Venue | Papers |
| --- | --- |
| [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/) | 137 papers |
| [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/) | 93 papers |
| [Conference on Computational Natural Language Learning (CoNLL)](https://md.research.elamur.ai/free/en/venues/conll/) | 38 papers |
| [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/) | 38 papers |
| [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/) | 35 papers |
| [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/) | 34 papers |
| [International Conference on Computational Linguistics (COLING)](https://md.research.elamur.ai/free/en/venues/coling/) | 26 papers |
| [International Conference on Parsing Technologies and the IWPT Shared Task on Parsing into Enhanced Universal Dependencies](https://md.research.elamur.ai/free/en/venues/iwpt/) | 23 papers |
| [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/) | 15 papers |
| [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/) | 15 papers |

## People and institutions

[Joakim Nivre](https://md.research.elamur.ai/free/en/authors/joakim-nivre/), now at [Uppsala University](https://md.research.elamur.ai/free/en/universities/uppsala-university/) and RISE Research Institutes of Sweden, wrote the linear-time deterministic dependency parser ([Nivre 2003](https://md.research.elamur.ai/free/en/papers/nivre-2003-efficient-algorithm-projective-dependency-parsing/)) and co-built MaltParser ([Nivre et al. 2006](https://md.research.elamur.ai/free/en/papers/nivre-2006-maltparser-data-driven-parser-generator/)). He also co-organised the CoNLL shared tasks ([Nivre et al. 2007](https://md.research.elamur.ai/free/en/papers/nivre-2007-conll-2007-shared-task-dependency/)) and helped found universal dependency annotation ([McDonald et al. 2013](https://md.research.elamur.ai/free/en/papers/mcdonald-2013-universal-dependency-annotation-multilingual-parsing/)). [Christopher D. Manning](https://md.research.elamur.ai/free/en/authors/christopher-d-manning/) of [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) co-authored unlexicalised PCFG parsing ([Klein & Manning 2003](https://md.research.elamur.ai/free/en/papers/klein-2003-accurate-unlexicalized-parsing/)), an influential early neural dependency parser ([Chen & Manning 2014](https://md.research.elamur.ai/free/en/papers/chen-2014-fast-accurate-dependency-parser-neural/)) and biaffine parsing ([Dozat & Manning 2016](https://md.research.elamur.ai/free/en/papers/dozat-2016-deep-biaffine-attention-neural-dependency/)). [Dan Klein](https://md.research.elamur.ai/free/en/authors/dan-klein/), of the University of California, Berkeley, worked on latent-annotation grammars ([Petrov et al. 2006](https://md.research.elamur.ai/free/en/papers/petrov-2006-learning-accurate-compact-interpretable-tree/); [Petrov & Klein 2007](https://md.research.elamur.ai/free/en/papers/petrov-2007-improved-inference-unlexicalized-parsing/)) and self-attentive parsing ([Kitaev & Klein 2018](https://md.research.elamur.ai/free/en/papers/kitaev-2018-constituency-parsing-self-attentive-encoder/)).

Authors with the most papers on this subject Joakim Nivre leads with 27 papers; the 12 authors shown wrote 167 of the papers on this subject between them, counting each paper once per author.

**The numbers**

| Author | Papers here · their citations |
| --- | --- |
| [Joakim Nivre](https://md.research.elamur.ai/free/en/authors/joakim-nivre/) | 27 papers · 12,075 citations |
| [Christopher D. Manning](https://md.research.elamur.ai/free/en/authors/christopher-d-manning/) | 18 papers · 13,949 citations |
| [Noah A. Smith](https://md.research.elamur.ai/free/en/authors/noah-a-smith/) | 18 papers · 4,089 citations |
| [Luke Zettlemoyer](https://md.research.elamur.ai/free/en/authors/luke-zettlemoyer/) | 14 papers · 5,866 citations |
| [Michael Collins](https://md.research.elamur.ai/free/en/authors/michael-collins/) | 13 papers · 8,929 citations |
| [Ryan McDonald](https://md.research.elamur.ai/free/en/authors/ryan-mcdonald/) | 13 papers · 8,718 citations |
| [Yoav Goldberg](https://md.research.elamur.ai/free/en/authors/yoav-goldberg/) | 12 papers · 4,192 citations |
| [Nianwen Xue](https://md.research.elamur.ai/free/en/authors/nianwen-xue/) | 12 papers · 1,432 citations |
| [Mark Johnson](https://md.research.elamur.ai/free/en/authors/mark-johnson/) | 10 papers · 4,516 citations |
| [Nathan Schneider](https://md.research.elamur.ai/free/en/authors/nathan-schneider/) | 10 papers · 2,951 citations |
| [Reut Tsarfaty](https://md.research.elamur.ai/free/en/authors/reut-tsarfaty/) | 10 papers · 2,295 citations |
| [Giuseppe Carenini](https://md.research.elamur.ai/free/en/authors/giuseppe-carenini/) | 10 papers · 666 citations |

[Michael Collins](https://md.research.elamur.ai/free/en/authors/michael-collins/) defined head-driven lexicalised parsing ([Collins 1997](https://md.research.elamur.ai/free/en/papers/collins-1997-three-generative-lexicalised-models-statistical/)) and discriminative reranking ([Collins & Koo 2005](https://md.research.elamur.ai/free/en/papers/collins-2000-discriminative-reranking-natural-language-parsing/)). [Eugene Charniak](https://md.research.elamur.ai/free/en/authors/eugene-charniak/) and [Mark Johnson](https://md.research.elamur.ai/free/en/authors/mark-johnson/) built the Brown reranking parser and showed that self-training works ([Charniak & Johnson 2005](https://md.research.elamur.ai/free/en/papers/charniak-2005-coarse-fine-n-best-parsing/); [McClosky et al. 2006](https://md.research.elamur.ai/free/en/papers/mcclosky-2006-effective-self-training-parsing/)). Johnson also co-designed a non-monotonic transition system ([Honnibal & Johnson 2015](https://md.research.elamur.ai/free/en/papers/honnibal-2015-improved-non-monotonic-transition-system/)). [Ryan McDonald](https://md.research.elamur.ai/free/en/authors/ryan-mcdonald/) introduced spanning-tree dependency parsing ([McDonald et al. 2005](https://md.research.elamur.ai/free/en/papers/mcdonald-2005-non-projective-dependency-parsing-spanning/); [McDonald & Pereira 2006](https://md.research.elamur.ai/free/en/papers/mcdonald-2006-online-learning-approximate-dependency-parsing/)).

Where the authors on this subject work now Google is the current home of authors of 44 papers on this subject; counted are the authors with a page here, by their latest institution.

**The numbers**

| Institution | Papers |
| --- | --- |
| [Google](https://md.research.elamur.ai/free/en/companies/google/) | 44 papers |
| [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) | 32 papers |
| [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) | 30 papers |
| [Uppsala University](https://md.research.elamur.ai/free/en/universities/uppsala-university/) | 29 papers |
| [RISE Research Institutes of Sweden](https://ror.org/03nnxqz81) | 28 papers |
| [Centre National de la Recherche Scientifique](https://md.research.elamur.ai/free/en/labs/centre-national-de-la-recherche-scientifique/) | 25 papers |
| [Meta](https://md.research.elamur.ai/free/en/companies/meta/) | 23 papers |
| [University of Edinburgh](https://md.research.elamur.ai/free/en/universities/university-of-edinburgh/) | 21 papers |
| [Bar-Ilan University](https://md.research.elamur.ai/free/en/universities/bar-ilan-university/) | 20 papers |
| [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) | 19 papers |

[Yoav Goldberg](https://md.research.elamur.ai/free/en/authors/yoav-goldberg/), at [Bar-Ilan University](https://md.research.elamur.ai/free/en/universities/bar-ilan-university/) and the [Allen Institute for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/allen-institute/), co-authored BiLSTM feature parsing ([Kiperwasser & Goldberg 2016](https://md.research.elamur.ai/free/en/papers/kiperwasser-2016-simple-accurate-dependency-parsing-bidirectional/)). [Noah A. Smith](https://md.research.elamur.ai/free/en/authors/noah-a-smith/), also at the Allen Institute for Artificial Intelligence, co-developed the stack-LSTM parser ([Dyer et al. 2015](https://md.research.elamur.ai/free/en/papers/dyer-2015-transition-based-dependency-parsing-stack/)). [Luke Zettlemoyer](https://md.research.elamur.ai/free/en/authors/luke-zettlemoyer/), at the [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) and [Meta](https://md.research.elamur.ai/free/en/companies/meta/), led work on CCG semantic parsing from weak supervision ([Artzi & Zettlemoyer 2013](https://md.research.elamur.ai/free/en/papers/artzi-2013-weakly-supervised-learning-semantic-parsers/)). [Daniel Zeman](https://md.research.elamur.ai/free/en/authors/daniel-zeman/) of Charles University organised the raw-text UD shared task ([Zeman et al. 2017](https://md.research.elamur.ai/free/en/papers/zeman-2017-conll-2017-shared-task-multilingual/)). His co-organisers include [Nizar Habash](https://md.research.elamur.ai/free/en/authors/nizar-habash/), now at [New York University](https://md.research.elamur.ai/free/en/universities/new-york-university/) Abu Dhabi. [Stephen Clark](https://md.research.elamur.ai/free/en/authors/stephen-clark/) developed wide-coverage CCG parsing ([Clark & Curran 2007](https://md.research.elamur.ai/free/en/papers/clark-2007-wide-coverage-efficient-statistical-parsing/)). At [Google](https://md.research.elamur.ai/free/en/companies/google/), researchers built SyntaxNet ([Andor et al. 2016](https://md.research.elamur.ai/free/en/papers/andor-2016-globally-normalized-transition-based-neural/)) and TaPas ([Herzig et al. 2020](https://md.research.elamur.ai/free/en/papers/herzig-2020-tapas-weakly-supervised-table-parsing/)).

Countries of the authors on this subject United States comes first: authors working there now wrote 301 papers on this subject.

**The numbers**

| Country | Papers |
| --- | --- |
| [United States](https://md.research.elamur.ai/free/en/countries/united-states/) | 301 papers |
| [China](https://md.research.elamur.ai/free/en/countries/china/) | 115 papers |
| [United Kingdom](https://md.research.elamur.ai/free/en/countries/united-kingdom/) | 81 papers |
| [Germany](https://md.research.elamur.ai/free/en/countries/germany/) | 58 papers |
| [Sweden](https://md.research.elamur.ai/free/en/countries/sweden/) | 36 papers |
| [Israel](https://md.research.elamur.ai/free/en/countries/israel/) | 33 papers |
| [Japan](https://md.research.elamur.ai/free/en/countries/japan/) | 32 papers |
| [France](https://md.research.elamur.ai/free/en/countries/france/) | 31 papers |
| Hong Kong | 29 papers |
| [Singapore](https://md.research.elamur.ai/free/en/countries/singapore/) | 26 papers |

## Open questions

Is explicit syntax still needed? Pretrained encoders raise parser accuracy ([Kitaev & Klein 2018](https://md.research.elamur.ai/free/en/papers/kitaev-2018-constituency-parsing-self-attentive-encoder/)), and end-to-end models can answer structured questions without producing any structure ([Herzig et al. 2020](https://md.research.elamur.ai/free/en/papers/herzig-2020-tapas-weakly-supervised-table-parsing/)). It remains unsettled when an explicit tree helps a downstream system and when it only adds a source of errors. Some studies have asked where syntax pays off, such as the analysis of where full parses help in role labelling ([Punyakanok et al. 2008](https://md.research.elamur.ai/free/en/papers/punyakanok-2008-importance-syntactic-parsing-inference-semantic/)). Such studies have not yet been repeated systematically for current models. A related question is what attention-based parsers actually rely on. Kitaev and Klein found that their encoder leaned mostly on word position rather than word content ([Kitaev & Klein 2018](https://md.research.elamur.ai/free/en/papers/kitaev-2018-constituency-parsing-self-attentive-encoder/)), which links parsing to [interpretability](https://md.research.elamur.ai/free/en/topics/interpretability/).

Robustness beyond newswire is the second gap. In CoNLL 2007, the best LAS on chemical abstracts was 81.06, against 89.01 for the same team on in-domain English ([Nivre et al. 2007](https://md.research.elamur.ai/free/en/papers/nivre-2007-conll-2007-shared-task-dependency/)). In 2017, the top LAS reached only about 29 on Kazakh and about 32 on Buryat, languages with almost no training data ([Zeman et al. 2017](https://md.research.elamur.ai/free/en/papers/zeman-2017-conll-2017-shared-task-multilingual/)). Cross-lingual transfer from harmonised treebanks helps ([McDonald et al. 2013](https://md.research.elamur.ai/free/en/papers/mcdonald-2013-universal-dependency-annotation-multilingual-parsing/)). Even so, parsing [low-resource languages](https://md.research.elamur.ai/free/en/topics/low-resource-languages/) and languages with rich [morphology](https://md.research.elamur.ai/free/en/concepts/morphology/) is still far from English accuracy.

Third, exact search remains a constraint. Non-projective parsing with rich features is intractable to do exactly ([McDonald & Pereira 2006](https://md.research.elamur.ai/free/en/papers/mcdonald-2006-online-learning-approximate-dependency-parsing/)). Global normalisation narrowed the trade-off between local greedy decisions and global search ([Andor et al. 2016](https://md.research.elamur.ai/free/en/papers/andor-2016-globally-normalized-transition-based-neural/)), but it has not disappeared.

For semantic parsing, generalising to unseen schemas remains hard ([Yu et al. 2018](https://md.research.elamur.ai/free/en/papers/yu-2018-spider-large-scale-human-labeled/)). Exact-match metrics also misjudge queries that are equivalent but written differently ([Wang et al. 2020](https://md.research.elamur.ai/free/en/papers/wang-2020-rat-sql-relation-aware-schema/)). Finally, surprisal-based parsing once served as a model of human comprehension ([Hale 2001](https://md.research.elamur.ai/free/en/papers/hale-2001-probabilistic-earley-parser-as-psycholinguistic/)). It is open whether parsers can do so again, now that the strongest systems no longer compute explicit prefix probabilities over a grammar.

## The library in figures

Neighbouring subjects: papers shared Natural language processing shares the most: 560 of the 625 papers here (90%) stand on both.

**The numbers**

| Shelf | Papers shared |
| --- | --- |
| [Natural language processing](https://md.research.elamur.ai/free/en/concepts/natural-language-processing/) | 560 shared · 90% |
| [Semantics](https://md.research.elamur.ai/free/en/concepts/semantics/) | 33 shared · 5% |
| [Morphology](https://md.research.elamur.ai/free/en/concepts/morphology/) | 30 shared · 5% |
| [Information extraction](https://md.research.elamur.ai/free/en/concepts/information-extraction/) | 25 shared · 4% |
| [Natural language understanding](https://md.research.elamur.ai/free/en/concepts/natural-language-understanding/) | 16 shared · 3% |
| [Large language models](https://md.research.elamur.ai/free/en/concepts/large-language-models/) | 15 shared · 2% |
| [Machine learning](https://md.research.elamur.ai/free/en/concepts/machine-learning/) | 14 shared · 2% |
| [Question answering](https://md.research.elamur.ai/free/en/concepts/question-answering/) | 14 shared · 2% |
| [Few-shot learning](https://md.research.elamur.ai/free/en/concepts/few-shot-learning/) | 12 shared · 2% |
| [Knowledge graph](https://md.research.elamur.ai/free/en/concepts/knowledge-graph/) | 8 shared · 1% |

## Try it: a dependency tree, arc by arc

A sentence as a dependency tree: every word has exactly one head, and the arc from the head to the word carries a Universal Dependencies relation — `nsubj` for a subject, `obj` for an object, `det` for a determiner. Pick a sentence and a word to highlight its head and its dependents. The trees are the output of a real parser, so they show its choices, attachment ambiguities included («the man with the telescope»).

| word | part of speech | relation | head |
| --- | --- | --- | --- |
| The | DET | det | cat |
| cat | NOUN | nsubj | sat |
| sat | VERB | root | (root) |
| on | ADP | case | mat |
| the | DET | det | mat |
| mat | NOUN | obl | sat |
| because | SCONJ | mark | tired |
| it | PRON | nsubj | tired |
| was | AUX | cop | tired |
| tired | ADJ | advcl | sat |
| . | PUNCT | punct | sat |

| word | part of speech | relation | head |
| --- | --- | --- | --- |
| The | DET | det | parser |
| old | ADJ | amod | parser |
| parser | NOUN | nsubj | handle |
| could | AUX | aux | handle |
| not | PART | advmod | handle |
| handle | VERB | root | (root) |
| long | ADJ | amod | sentences |
| sentences | NOUN | obj | handle |
| . | PUNCT | punct | handle |

Parsed by [Mova](https://md.research.elamur.ai/free/en/projects/mova/)’s English parser, trained on Universal Dependencies treebanks; part-of-speech tags and relations as in UD.

## Key papers

The works this page cites, the most cited first; citations as counted by OpenAlex plus the papers of this library that cite the work, read 2026-10-24.

- [Accurate Unlexicalized Parsing](https://md.research.elamur.ai/free/en/papers/klein-2003-accurate-unlexicalized-parsing/) — Dan Klein, Christopher D. Manning · 2003 · 3,942 citations
- [Building a Large Annotated Corpus of English: The Penn Treebank](https://md.research.elamur.ai/free/en/papers/marcus-1993-building-large-annotated-corpus-english/) — Mitchell P. Marcus, Beatrice Santorini, Mary Ann Marcinkiewicz · 1993 · 3,024 citations
- [A Fast and Accurate Dependency Parser using Neural Networks](https://md.research.elamur.ai/free/en/papers/chen-2014-fast-accurate-dependency-parser-neural/) — Danqi Chen, Christopher Manning · 2014 · 2,309 citations
- [Semantic Parsing on Freebase from Question-Answer Pairs](https://md.research.elamur.ai/free/en/papers/berant-2013-semantic-parsing-freebase-question-answer/) — Jonathan Berant, Andrew Chou, Roy Frostig et al. · 2013 · 2,305 citations
- [A Maximum-Entropy-Inspired Parser](https://md.research.elamur.ai/free/en/papers/charniak-2000-maximum-entropy-inspired-parser/) — Eugene Charniak · 2000 · 2,050 citations
- [Head-Driven Statistical Models for Natural Language Parsing](https://md.research.elamur.ai/free/en/papers/collins-2003-head-driven-statistical-models-natural/) — Michael Collins · 2003 · 2,001 citations
- [Spider: A Large-Scale Human-Labeled Dataset for Complex and Cross-Domain Semantic Parsing and Text-to-SQL Task](https://md.research.elamur.ai/free/en/papers/yu-2018-spider-large-scale-human-labeled/) — Tao Yu, Rui Zhang, Kai Yang et al. · 2018 · 1,584 citations
- [A Probabilistic Earley Parser as a Psycholinguistic Model](https://md.research.elamur.ai/free/en/papers/hale-2001-probabilistic-earley-parser-as-psycholinguistic/) — John Hale · 2001 · 1,448 citations
- [CoNLL-X Shared Task on Multilingual Dependency Parsing](https://md.research.elamur.ai/free/en/papers/buchholz-2006-conll-x-shared-task-multilingual/) — Sabine Buchholz, Erwin Marsi · 2006 · 1,426 citations
- [Deep Biaffine Attention for Neural Dependency Parsing](https://md.research.elamur.ai/free/en/papers/dozat-2016-deep-biaffine-attention-neural-dependency/) — Timothy Dozat, Christopher D. Manning · 2016 · 1,323 citations
- [Coarse-to-Fine n-Best Parsing and MaxEnt Discriminative Reranking](https://md.research.elamur.ai/free/en/papers/charniak-2005-coarse-fine-n-best-parsing/) — Eugene Charniak, Mark Johnson · 2005 · 1,312 citations
- [Non-Projective Dependency Parsing using Spanning Tree Algorithms](https://md.research.elamur.ai/free/en/papers/mcdonald-2005-non-projective-dependency-parsing-spanning/) — Ryan McDonald, Fernando Pereira, Kiril Ribarov et al. · 2005 · 1,259 citations
- [Learning Accurate, Compact, and Interpretable Tree Annotation](https://md.research.elamur.ai/free/en/papers/petrov-2006-learning-accurate-compact-interpretable-tree/) — Slav Petrov, Leon Barrett, Romain Thibaux et al. · 2006 · 1,244 citations
- [Online Large-Margin Training of Dependency Parsers](https://md.research.elamur.ai/free/en/papers/mcdonald-2005-online-large-margin-training-dependency/) — Ryan McDonald, Koby Crammer, Fernando Pereira · 2005 · 1,212 citations
- [A Stochastic Parts Program and Noun Phrase Parser for Unrestricted Text](https://md.research.elamur.ai/free/en/papers/church-1988-stochastic-parts-program-noun-phrase/) — Kenneth Ward Church · 1988 · 1,183 citations
- [Three Generative, Lexicalised Models for Statistical Parsing](https://md.research.elamur.ai/free/en/papers/collins-1997-three-generative-lexicalised-models-statistical/) — Michael Collins · 1997 · 1,046 citations
- [The CoNLL 2007 Shared Task on Dependency Parsing](https://md.research.elamur.ai/free/en/papers/nivre-2007-conll-2007-shared-task-dependency/) — Joakim Nivre, Johan Hall, Sandra Kübler et al. · 2007 · 1,010 citations
- [Discriminative Reranking for Natural Language Parsing](https://md.research.elamur.ai/free/en/papers/collins-2000-discriminative-reranking-natural-language-parsing/) — Michael Collins, Terry Koo · 2005 · 990 citations
- [Semantic Parsing via Staged Query Graph Generation: Question Answering with Knowledge Base](https://md.research.elamur.ai/free/en/papers/yih-2015-semantic-parsing-staged-query-graph/) — Wen-tau Yih, Ming-Wei Chang, Xiaodong He et al. · 2015 · 962 citations
- [Improved Inference for Unlexicalized Parsing](https://md.research.elamur.ai/free/en/papers/petrov-2007-improved-inference-unlexicalized-parsing/) — Slav Petrov, Dan Klein · 2007 · 941 citations
- [Three New Probabilistic Models for Dependency Parsing: An Exploration](https://md.research.elamur.ai/free/en/papers/eisner-1996-three-new-probabilistic-models-dependency/) — Jason M. Eisner · 1996 · 908 citations
- [Compositional Semantic Parsing on Semi-Structured Tables](https://md.research.elamur.ai/free/en/papers/pasupat-2015-compositional-semantic-parsing-semi-structured/) — Panupong Pasupat, Percy Liang · 2015 · 907 citations
- [Simple and Accurate Dependency Parsing Using Bidirectional LSTM Feature Representations](https://md.research.elamur.ai/free/en/papers/kiperwasser-2016-simple-accurate-dependency-parsing-bidirectional/) — Eliyahu Kiperwasser, Yoav Goldberg · 2016 · 905 citations
- [Transition-Based Dependency Parsing with Stack Long Short-Term Memory](https://md.research.elamur.ai/free/en/papers/dyer-2015-transition-based-dependency-parsing-stack/) — Chris Dyer, Miguel Ballesteros, Wang Ling et al. · 2015 · 862 citations
- [Effective Self-Training for Parsing](https://md.research.elamur.ai/free/en/papers/mcclosky-2006-effective-self-training-parsing/) — David McClosky, Eugene Charniak, Mark Johnson · 2006 · 847 citations
- [TaPas: Weakly Supervised Table Parsing via Pre-training](https://md.research.elamur.ai/free/en/papers/herzig-2020-tapas-weakly-supervised-table-parsing/) — Jonathan Herzig, Pawel Krzysztof Nowak, Thomas Müller et al. · 2020 · 845 citations
- [Tokenizing, POS Tagging, Lemmatizing and Parsing UD 2.0 with UDPipe](https://md.research.elamur.ai/free/en/papers/straka-2017-tokenizing-pos-tagging-lemmatizing-parsing/) — Milan Straka, Jana Straková · 2017 · 811 citations
- [Constituency Parsing with a Self-Attentive Encoder](https://md.research.elamur.ai/free/en/papers/kitaev-2018-constituency-parsing-self-attentive-encoder/) — Nikita Kitaev, Dan Klein · 2018 · 762 citations
- [RAT-SQL: Relation-Aware Schema Encoding and Linking for Text-to-SQL Parsers](https://md.research.elamur.ai/free/en/papers/wang-2020-rat-sql-relation-aware-schema/) — Bailin Wang, Richard Shin, Xiaodong Liu et al. · 2020 · 752 citations
- [An Improved Non-monotonic Transition System for Dependency Parsing](https://md.research.elamur.ai/free/en/papers/honnibal-2015-improved-non-monotonic-transition-system/) — Matthew Honnibal, Mark Johnson · 2015 · 751 citations
- [Universal Dependency Annotation for Multilingual Parsing](https://md.research.elamur.ai/free/en/papers/mcdonald-2013-universal-dependency-annotation-multilingual-parsing/) — Ryan McDonald, Joakim Nivre, Yvonne Quirmbach-Brundage et al. · 2013 · 749 citations
- [An Efficient Algorithm for Projective Dependency Parsing](https://md.research.elamur.ai/free/en/papers/nivre-2003-efficient-algorithm-projective-dependency-parsing/) — Joakim Nivre · 2003 · 745 citations
- [Statistical Decision-Tree Models for Parsing](https://md.research.elamur.ai/free/en/papers/magerman-1995-statistical-decision-tree-models-parsing/) — David M. Magerman · 1995 · 733 citations
- [Online Learning of Approximate Dependency Parsing Algorithms](https://md.research.elamur.ai/free/en/papers/mcdonald-2006-online-learning-approximate-dependency-parsing/) — Ryan McDonald, Fernando Pereira · 2006 · 729 citations
- [The CoNLL 2008 Shared Task on Joint Parsing of Syntactic and Semantic Dependencies](https://md.research.elamur.ai/free/en/papers/surdeanu-2008-conll-2008-shared-task-joint/) — Mihai Surdeanu, Richard Johansson, Adam Meyers et al. · 2008 · 728 citations
- [Semantic Parsing via Paraphrasing](https://md.research.elamur.ai/free/en/papers/berant-2014-semantic-parsing-paraphrasing/) — Jonathan Berant, Percy Liang · 2014 · 682 citations
- [Globally Normalized Transition-Based Neural Networks](https://md.research.elamur.ai/free/en/papers/andor-2016-globally-normalized-transition-based-neural/) — Daniel Andor, Chris Alberti, David Weiss et al. · 2016 · 679 citations
- [MaltParser: A Data-Driven Parser-Generator for Dependency Parsing](https://md.research.elamur.ai/free/en/papers/nivre-2006-maltparser-data-driven-parser-generator/) — Joakim Nivre, Johan Hall, Jens Nilsson · 2006 · 669 citations
- [Simple Semi-supervised Dependency Parsing](https://md.research.elamur.ai/free/en/papers/koo-2008-simple-semi-supervised-dependency-parsing/) — Terry Koo, Xavier Carreras, Michael Collins · 2008 · 657 citations
- [Wide-Coverage Efficient Statistical Parsing with CCG and Log-Linear Models](https://md.research.elamur.ai/free/en/papers/clark-2007-wide-coverage-efficient-statistical-parsing/) — Stephen Clark, James R. Curran · 2007 · 626 citations
- [Parsing Argumentation Structures in Persuasive Essays](https://md.research.elamur.ai/free/en/papers/stab-2017-parsing-argumentation-structures-persuasive-essays/) — Christian Stab, Iryna Gurevych · 2017 · 613 citations
- [CoNLL 2017 Shared Task: Multilingual Parsing from Raw Text to Universal Dependencies](https://md.research.elamur.ai/free/en/papers/zeman-2017-conll-2017-shared-task-multilingual/) — Daniel Zeman, Martin Popel, Milan Straka et al. · 2017 · 604 citations
- [Named Entity Recognition as Dependency Parsing](https://md.research.elamur.ai/free/en/papers/yu-2020-named-entity-recognition-as-dependency/) — Juntao Yu, Bernd Bohnet, Massimo Poesio · 2020 · 594 citations
- [Weakly Supervised Learning of Semantic Parsers for Mapping Instructions to Actions](https://md.research.elamur.ai/free/en/papers/artzi-2013-weakly-supervised-learning-semantic-parsers/) — Yoav Artzi, Luke Zettlemoyer · 2013 · 593 citations
- [Top Accuracy and Fast Dependency Parsing is not a Contradiction](https://md.research.elamur.ai/free/en/papers/bohnet-2010-top-accuracy-fast-dependency-parsing/) — Bernd Bohnet · 2010 · 583 citations
- [Incremental Parsing with the Perceptron Algorithm](https://md.research.elamur.ai/free/en/papers/collins-2004-incremental-parsing-perceptron-algorithm/) — Michael Collins, Brian Roark · 2004 · 568 citations
- [Supertagging: An Approach to Almost Parsing](https://md.research.elamur.ai/free/en/papers/bangalore-1999-supertagging-approach-almost-parsing/) — Srinivas Bangalore, Aravind K. Joshi · 1999 · 567 citations
- [Sentence Level Discourse Parsing using Syntactic and Lexical Information](https://md.research.elamur.ai/free/en/papers/soricut-2003-sentence-level-discourse-parsing-syntactic/) — Radu Soricut, Daniel Marcu · 2003 · 560 citations
- [The Importance of Syntactic Parsing and Inference in Semantic Role Labeling](https://md.research.elamur.ai/free/en/papers/punyakanok-2008-importance-syntactic-parsing-inference-semantic/) — Vasin Punyakanok, Dan Roth, Wen-tau Yih · 2008 · 538 citations

## Related concepts

- [Dependency parsing](https://md.research.elamur.ai/free/en/concepts/dependency-parsing/) — Dependency parsing is the task of recovering a sentence's syntactic structure as directed, labelled links between its words. Each word is…
- [Context-free grammar](https://md.research.elamur.ai/free/en/concepts/context-free-grammar/) — A context-free grammar is a formal system of rewriting rules in which each rule replaces a single nonterminal symbol with a string of…
- [Semantic parsing](https://md.research.elamur.ai/free/en/concepts/semantic-parsing/) — Semantic parsing is the task of mapping a natural-language utterance to a formal, machine-interpretable representation of its meaning, such…

## More papers

Topics: [🌳 Dependency parsing](https://md.research.elamur.ai/free/en/topics/dependency-parsing/) · [📐 Syntax & grammar formalisms](https://md.research.elamur.ai/free/en/topics/syntax-grammar/) · [🧾 Semantic parsing & meaning representation](https://md.research.elamur.ai/free/en/topics/semantic-parsing/) · [🗂️ Treebanks](https://md.research.elamur.ai/free/en/topics/treebanks/)

Shelves: [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/)

Updated 24 October 2026

- [← previous concept parataxis: parataxis](https://md.research.elamur.ai/free/en/concepts/ud-parataxis/)
- [next concept → Part-of-speech tagging](https://md.research.elamur.ai/free/en/concepts/part-of-speech-tagging/)
- [↑ back to concepts](https://md.research.elamur.ai/free/en/concepts/)