---
title: "Artificial Intelligence — papers"
description: "Artificial Intelligence: 7,081 papers of the Elamur Research library · page 1 of 72 · 1–25, the most cited first — our summary of each, its ideas…"
canonical: "https://research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/)

[home](https://md.research.elamur.ai/free/en/) / [shelves](https://md.research.elamur.ai/free/en/shelves/) / [Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) / Artificial Intelligence

# 🤖 Artificial Intelligence

7,081 papers · page 1 of 72 · 1–25 · the most cited first · [all shelves](https://md.research.elamur.ai/free/en/shelves/)

Artificial Intelligence is a subfield of Computer Science in the OpenAlex classification of science. The topics most of its papers here are on: «Natural Language Processing Techniques», «Topic Modeling», «Speech and dialogue systems».

A survey · about 17 minutes to read · every paper named here has its own page in this library

**In this article**

1. [What it is](#a-what-it-is)
2. [History and milestones](#a-history-and-milestones)
3. [Main problems and tasks](#a-main-problems-and-tasks)
4. [Methods and schools](#a-methods-and-schools)
5. [Datasets, resources and benchmarks](#a-datasets-resources-and-benchmarks)
6. [Landmark papers in this library](#a-landmark-papers-in-this-library)
7. [People](#a-people)
8. [Institutions and places](#a-institutions-and-places)
9. [Open questions](#a-open-questions)
10. [Neighbouring topics](#a-neighbouring-topics)
11. [Key works in this library](#a-key-works-in-this-library)

Natural language processing (NLP) is the branch of computer science and artificial intelligence that builds programs able to analyse, understand, generate and translate human language. It sits where linguistics, statistics and machine learning meet, and for most of its history it has also been called computational linguistics: the first name stresses the engineering of working systems, the second the use of computation to study language itself. In practice the two communities share one set of conferences, one journal tradition and, increasingly, one toolbox.

In this library the Artificial Intelligence shelf, a subfield of computer science in OpenAlex, is above all natural language processing: it gathers the core literature of the field, mostly from the ACL Anthology and arXiv. It runs from early work on discourse structure and rule-based tagging, through the statistical turn of the 1990s and 2000s, to word embeddings, neural sequence models, the Transformer and the large pretrained language models that now dominate research. Along the way it covers the classic tasks: parsing, tagging, information extraction, sentiment analysis, question answering, dialogue, summarization, and the benchmarks and evaluation methods that made progress measurable.

The article below surveys the field as a whole. For depth on a single area, follow the topic pages: [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/), [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/), [Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/), [Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/), [Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) and the others linked throughout.

## What it is

NLP treats text and speech as data to be processed by algorithms. Its central difficulty is that language is ambiguous at every level. A word can carry several senses, a sentence can admit several syntactic analyses, a pronoun can point to more than one antecedent, and the intended meaning of an utterance depends on context, world knowledge and the speaker's goals. Much of the field's history is a search for representations and learning methods that resolve this ambiguity reliably and at scale.

The field is traditionally divided by linguistic level:

- **Morphology and tokenization** — splitting text into words and subword units and analysing their internal structure; see [Morphology](https://md.research.elamur.ai/free/en/topics/morphology/) and [Tokenization & segmentation](https://md.research.elamur.ai/free/en/topics/tokenization/).
- **Syntax** — assigning part-of-speech tags and building phrase-structure or dependency trees; see [Part-of-speech tagging](https://md.research.elamur.ai/free/en/topics/part-of-speech-tagging/), [Dependency parsing](https://md.research.elamur.ai/free/en/topics/dependency-parsing/) and [Syntax & grammar formalisms](https://md.research.elamur.ai/free/en/topics/syntax-grammar/).
- **Semantics** — word meaning, semantic roles and formal meaning representations; see [Lexical semantics & word embeddings](https://md.research.elamur.ai/free/en/topics/lexical-semantics/), [Semantic role labeling](https://md.research.elamur.ai/free/en/topics/semantic-role-labeling/) and [Semantic parsing & meaning representation](https://md.research.elamur.ai/free/en/topics/semantic-parsing/).
- **Discourse and pragmatics** — how sentences connect into coherent texts and conversations; see [Discourse analysis](https://md.research.elamur.ai/free/en/topics/discourse-analysis/) and [Coreference resolution](https://md.research.elamur.ai/free/en/topics/coreference-resolution/).

Alongside the levels stand applications that cut across them: translation, extraction of facts from text, classification of opinions, answering questions, holding conversations and writing summaries.

NLP differs from its neighbours mainly in emphasis. Theoretical [linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) asks how language works; NLP asks how to make a machine handle it, and will accept a method that works even if it explains little. General Artificial Intelligence supplies the learning algorithms, but NLP contributes its own problems (structured prediction over trees and sequences, open vocabularies, discrete symbols) and its own evaluation culture. Speech processing deals with the acoustic signal and is usually treated as a sister field, though the line has blurred since end-to-end neural models.

## History and milestones

Landmark papers on this shelf: the 12 most cited, in time The 12 most cited papers on this shelf span 1993–2021; the dot grows with the citations.

**The numbers**

| Paper | Citations |
| --- | --- |
| [1993 — C4.5: Programs for Machine Learning](https://md.research.elamur.ai/free/en/papers/quinlan-1993-c45-programs-machine-learning/) | 23,781 |
| [2002 — SMOTE: Synthetic Minority Over-sampling Technique](https://md.research.elamur.ai/free/en/papers/chawla-2002-smote-synthetic-minority-over-sampling/) | 32,661 |
| [2002 — Bleu: a Method for Automatic Evaluation of Machine Translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/) | 32,065 |
| [2003 — Latent Dirichlet Allocation](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/) | 28,297 |
| [2013 — Rich feature hierarchies for accurate object detection and semantic segmentation](https://md.research.elamur.ai/free/en/papers/girshick-2013-rich-feature-hierarchies-accurate-object/) | 31,715 |
| [2014 — Very Deep Convolutional Networks for Large-Scale Image Recognition](https://md.research.elamur.ai/free/en/papers/simonyan-2014-very-deep-convolutional-networks-large/) | 72,803 |
| [2014 — Neural Machine Translation by Jointly Learning to Align and Translate](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/) | 18,908 |
| [2019 — SciPy 1.0 — Fundamental Algorithms for Scientific Computing in Python](https://md.research.elamur.ai/free/en/papers/virtanen-2019-scipy-1-0-fundamental-algorithms/) | 38,317 |
| [2019 — BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/) | 23,422 |
| [2019 — PyTorch: An Imperative Style, High-Performance Deep Learning Library](https://md.research.elamur.ai/free/en/papers/paszke-2019-pytorch-imperative-style-high-performance/) | 18,392 |
| [2020 — Array Programming with NumPy](https://md.research.elamur.ai/free/en/papers/harris-2020-array-programming-numpy/) | 23,591 |
| [2021 — Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://md.research.elamur.ai/free/en/papers/liu-2021-swin-transformer-hierarchical-vision-transformer/) | 33,826 |

### Symbolic beginnings

The earliest NLP systems, from the 1950s to the 1980s, were built by hand: grammars, lexicons and rules written by linguists and programmers. This era produced influential theories of how texts hang together. Barbara Grosz and Candace Sidner's [Attention, Intentions, and the Structure of Discourse](https://md.research.elamur.ai/free/en/papers/grosz-1986-attention-intentions-structure-discourse/) (1986) split discourse structure into three interacting components — the linguistic segments, the speaker's purposes, and a focus of attention — and the later [Centering, a framework for local coherence](https://md.research.elamur.ai/free/en/papers/grosz-1995-centering-framework-modeling-local-coherence/) described how the entity in focus shifts from one utterance to the next. Finite-state methods, surveyed in [Finite-State Transducers in Language and Speech Processing](https://md.research.elamur.ai/free/en/papers/mohri-1997-finite-state-transducers-language-speech/), gave an efficient formal basis for morphology and speech.

### The statistical revolution

From the late 1980s, the availability of large machine-readable corpora and annotated resources such as the Penn Treebank (released in the early 1990s) moved the field towards learning from data. Kenneth Church's [stochastic parts program](https://md.research.elamur.ai/free/en/papers/church-1988-stochastic-parts-program-noun-phrase/) (1988) showed that a simple probabilistic model could tag unrestricted text with high accuracy. Eric Brill answered with [a rule-based tagger](https://md.research.elamur.ai/free/en/papers/brill-1992-simple-rule-based-part-speech/) whose rules were themselves learned from errors, later generalised as [transformation-based error-driven learning](https://md.research.elamur.ai/free/en/papers/brill-1995-transformation-based-error-driven-learning/). Gale and Church's [sentence aligner for bilingual corpora](https://md.research.elamur.ai/free/en/papers/gale-1993-program-aligning-sentences-bilingual-corpora/) and the IBM translation models opened the way to statistical machine translation.

The 1990s also brought a general-purpose learning framework: [A maximum entropy approach to NLP](https://md.research.elamur.ai/free/en/papers/berger-1996-maximum-entropy-approach-natural-language/) (1996) argued for log-linear models that combine many overlapping features, an approach soon applied to tagging in [Ratnaparkhi's maximum entropy tagger](https://md.research.elamur.ai/free/en/papers/ratnaparkhi-1996-maximum-entropy-model-part-speech/). Language modelling matured with careful comparisons of [smoothing techniques](https://md.research.elamur.ai/free/en/papers/chen-1996-empirical-study-smoothing-techniques-language/). Lexicalised probabilistic parsers, notably Michael Collins's [Three Generative, Lexicalised Models for Statistical Parsing](https://md.research.elamur.ai/free/en/papers/collins-1997-three-generative-lexicalised-models-statistical/) and its journal version [Head-Driven Statistical Models for Natural Language Parsing](https://md.research.elamur.ai/free/en/papers/collins-2003-head-driven-statistical-models-natural/), set the standard for English parsing for a decade.

### Feature engineering and structured learning (2000s)

The 2000s were the age of discriminative models with rich, hand-designed features: maximum entropy taggers, conditional random fields, support vector machines and kernel methods. Accurate parsers became widely usable — [Accurate Unlexicalized Parsing](https://md.research.elamur.ai/free/en/papers/klein-2003-accurate-unlexicalized-parsing/) and the split-merge grammars of [Learning Accurate, Compact, and Interpretable Tree Annotation](https://md.research.elamur.ai/free/en/papers/petrov-2006-learning-accurate-compact-interpretable-tree/) are representative. Statistical machine translation moved from word alignment ([Improved Statistical Alignment Models](https://md.research.elamur.ai/free/en/papers/och-2000-improved-statistical-alignment-models/)) to discriminative log-linear systems ([Och and Ney, 2002](https://md.research.elamur.ai/free/en/papers/och-2002-discriminative-training-maximum-entropy-models/)) and to [hierarchical phrase-based translation](https://md.research.elamur.ai/free/en/papers/chiang-2007-hierarchical-phrase-based-translation/). Sentiment analysis was born as a research area in 2002, and shared tasks such as [CoNLL 2007 on dependency parsing](https://md.research.elamur.ai/free/en/papers/nivre-2007-conll-2007-shared-task-dependency/) made multilingual comparison routine.

### Neural networks and embeddings (2010s)

Papers on this shelf by year 7,075 dated papers from 1813 to 2026 (7 before 1966 are not drawn); the busiest year is 2026 with 849, and 65% of them appeared in 2018 or later.

**The numbers**

| Year | Papers |
| --- | --- |
| 2026 | 849 |
| 2025 | 293 |
| 2024 | 490 |
| 2023 | 708 |
| 2022 | 415 |
| 2021 | 453 |
| 2020 | 566 |
| 2019 | 449 |
| 2018 | 389 |
| 2017 | 308 |
| 2016 | 303 |
| 2015 | 179 |
| 2014 | 186 |
| 2013 | 127 |
| 2012 | 143 |
| 2011 | 105 |
| 2010 | 131 |
| 2009 | 95 |
| 2008 | 93 |
| 2007 | 84 |
| 2006 | 92 |
| 2005 | 69 |
| 2004 | 95 |
| 2003 | 58 |
| 2002 | 52 |
| 2001 | 23 |
| 2000 | 34 |
| 1999 | 32 |
| 1998 | 39 |
| 1997 | 29 |
| 1996 | 21 |
| 1995 | 24 |
| 1994 | 32 |
| 1993 | 14 |
| 1992 | 13 |
| 1991 | 15 |
| 1990 | 12 |
| 1989 | 6 |
| 1988 | 11 |
| 1987 | 8 |
| 1986 | 7 |
| 1985 | 2 |
| 1983 | 1 |
| 1982 | 3 |
| 1981 | 3 |
| 1978 | 1 |
| 1976 | 3 |
| 1975 | 1 |
| 1973 | 1 |
| 1969 | 1 |
| 1961 | 1 |
| 1923 | 1 |
| 1922 | 1 |
| 1884 | 1 |
| 1861 | 1 |
| 1816 | 1 |
| 1813 | 1 |

Around 2011–2013 neural methods arrived in force. Dense word vectors, popularised by work such as [Linguistic Regularities in Continuous Space Word Representations](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/), replaced sparse one-hot features. Recursive and recurrent networks were applied to sentiment ([Socher et al., 2011](https://md.research.elamur.ai/free/en/papers/socher-2011-semi-supervised-recursive-autoencoders-predicting/)), to compositional semantics ([Socher et al., 2012](https://md.research.elamur.ai/free/en/papers/socher-2012-semantic-compositionality-through-recursive-matrix/)) and to translation ([Recurrent Continuous Translation Models](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2013-recurrent-continuous-translation-models/)). Attention mechanisms, introduced for neural machine translation in 2014–2015, spread to classification ([Hierarchical Attention Networks for Document Classification](https://md.research.elamur.ai/free/en/papers/yang-2016-hierarchical-attention-networks-document-classification/)), relation extraction and reading comprehension. Large crowdsourced datasets — SQuAD above all — gave these data-hungry models something to learn from.

### Transformers and pretrained language models (2017 onwards)

The Transformer architecture (2017) replaced recurrence with self-attention and proved both faster to train and more accurate; refinements such as [relative position representations](https://md.research.elamur.ai/free/en/papers/shaw-2018-self-attention-relative-position-representations/) followed quickly. From 2018, pretraining a large network on raw text and then fine-tuning it became the default recipe: ELMo, BERT, GPT and their successors, including [BART](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/), [ERNIE 2.0](https://md.research.elamur.ai/free/en/papers/sun-2019-ernie-2-0-continual/) and [GLM](https://md.research.elamur.ai/free/en/papers/du-2022-glm-general-language-model-pretraining/). Shared software such as the [Hugging Face Transformers library](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/) and [fairseq](https://md.research.elamur.ai/free/en/papers/ott-2019-fairseq-fast-extensible-toolkit-sequence/) made these models available to everyone. With GPT-3 (2020) attention shifted to prompting and in-context learning, and since 2022 instruction-tuned conversational models have reshaped both the research agenda and public perception of the field. Surveys such as [A Comprehensive Overview of Large Language Models](https://md.research.elamur.ai/free/en/papers/naveed-2023-comprehensive-overview-large-language-models/) chart this latest phase.

## Main problems and tasks

NLP research is organised largely around tasks: well-defined input–output problems with shared data and metrics. The most important families are these.

**Sequence labelling.** Assigning a label to every token: part-of-speech tags, named-entity labels, chunk boundaries. Classic systems include the [cyclic dependency network tagger](https://md.research.elamur.ai/free/en/papers/toutanova-2003-feature-rich-part-speech-tagging/) and the CRF-based entity recogniser of [Finkel, Grenager and Manning](https://md.research.elamur.ai/free/en/papers/finkel-2005-incorporating-non-local-information-into/); neural successors include [contextual string embeddings](https://md.research.elamur.ai/free/en/papers/akbik-2018-contextual-string-embeddings-sequence-labeling/) and [lattice LSTMs for Chinese](https://md.research.elamur.ai/free/en/papers/zhang-2018-chinese-ner-lattice-lstm/). See [Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/).

**Parsing.** Recovering syntactic structure, either as constituency trees or as head–dependent relations. Parsing is the subject of [Dependency parsing](https://md.research.elamur.ai/free/en/topics/dependency-parsing/), [Treebanks](https://md.research.elamur.ai/free/en/topics/treebanks/) and [Universal Dependencies](https://md.research.elamur.ai/free/en/topics/universal-dependencies/).

**Semantic analysis.** Labelling who did what to whom ([Automatic Labeling of Semantic Roles](https://md.research.elamur.ai/free/en/papers/gildea-2002-automatic-labeling-semantic-roles/)), disambiguating word senses ([Word sense disambiguation](https://md.research.elamur.ai/free/en/topics/word-sense-disambiguation/)), measuring word similarity, and mapping sentences to logical forms.

**Information extraction.** Turning free text into structured records: entities, relations, events. Work here moved from kernels over dependency paths ([Bunescu and Mooney](https://md.research.elamur.ai/free/en/papers/bunescu-2005-shortest-path-dependency-kernel-relation/)) to [convolutional](https://md.research.elamur.ai/free/en/papers/zeng-2014-relation-classification-convolutional-deep-neural/) and [recurrent](https://md.research.elamur.ai/free/en/papers/miwa-2016-end-end-relation-extraction-lstms/) networks and [graph convolutions over trees](https://md.research.elamur.ai/free/en/papers/zhang-2018-graph-convolution-over-pruned-dependency/). See [Information extraction](https://md.research.elamur.ai/free/en/topics/information-extraction/) and [Knowledge graphs & ontologies](https://md.research.elamur.ai/free/en/topics/knowledge-graphs/).

**Text classification and sentiment.** Assigning labels to whole documents or sentences — topic, polarity, emotion, stance, toxicity. Sentiment analysis is one of the largest strands on this shelf; see [Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/).

**Question answering and reading comprehension.** Finding or generating answers from a passage, a document collection or a model's own parameters; see [Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/) and [Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/).

**Generation.** Producing fluent text: [summarization](https://md.research.elamur.ai/free/en/topics/summarization/), [dialogue](https://md.research.elamur.ai/free/en/topics/dialogue-systems/), data-to-text and open-ended [text generation](https://md.research.elamur.ai/free/en/topics/text-generation/).

**Translation.** Mapping between languages, historically the founding problem of the field; see [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) and the separate [Machine translation topic](https://md.research.elamur.ai/free/en/topics/machine-translation/).

## Methods and schools

### Rule-based and knowledge-based approaches

Early systems encoded linguistic knowledge explicitly: grammars, morphological analysers, semantic networks and dictionaries such as WordNet. Hand-built resources remain valuable where data is scarce, and lexicon-based sentiment scoring — see [Lexicon-Based Methods for Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/taboada-2011-lexicon-based-methods-sentiment-analysis/) — still offers transparent, domain-robust baselines. Measures of word similarity over WordNet, compared in [Budanitsky and Hirst](https://md.research.elamur.ai/free/en/papers/budanitsky-2006-evaluating-wordnet-based-measures-lexical/) and built on formulas such as [Wu and Palmer's](https://md.research.elamur.ai/free/en/papers/wu-1994-verb-semantics-lexical-selection/), illustrate how knowledge bases fed quantitative methods.

### Statistical and distributional approaches

The statistical school holds that the right analysis is the most probable one given data. Its core tools were n-gram language models, hidden Markov models, probabilistic context-free grammars, log-linear models and, for scale, efficient toolkits such as [KenLM](https://md.research.elamur.ai/free/en/papers/heafield-2011-kenlm-faster-smaller-language-model/). In parallel, distributional semantics — the idea that a word is characterised by the contexts it occurs in — produced [distributional clustering](https://md.research.elamur.ai/free/en/papers/pereira-1993-distributional-clustering-english-words/), [automatic thesaurus construction](https://md.research.elamur.ai/free/en/papers/lin-1998-automatic-retrieval-clustering-similar-words/) and [word sense discrimination by context vectors](https://md.research.elamur.ai/free/en/papers/schutze-1998-automatic-word-sense-discrimination/), the direct ancestors of word embeddings.

A recurring concern of this school was annotation quality: if systems are trained and judged against human labels, those labels must be reliable. Jean Carletta's [argument for the kappa statistic](https://md.research.elamur.ai/free/en/papers/carletta-1996-assessing-agreement-classification-tasks-kappa/) and the later survey on [inter-coder agreement](https://md.research.elamur.ai/free/en/papers/artstein-2008-survey-article-inter-coder-agreement/) set the methodological standard.

### Neural approaches

Neural NLP replaced manual feature design with learned representations. The progression ran from fixed word vectors to convolutional and recurrent encoders, attention, and finally Transformers pretrained on very large corpora. Pretrained models are now adapted by fine-tuning, by lightweight adapters, or by prompting — the last explored in [AutoPrompt](https://md.research.elamur.ai/free/en/papers/shin-2020-autoprompt-eliciting-knowledge-language-models/), [cloze-style few-shot learning](https://md.research.elamur.ai/free/en/papers/schick-2021-exploiting-cloze-questions-few-shot/), [better few-shot fine-tuning](https://md.research.elamur.ai/free/en/papers/gao-2021-making-pre-trained-language-models/) and [P-Tuning](https://md.research.elamur.ai/free/en/papers/liu-2022-p-tuning-prompt-tuning-can/).

### The main debates

Several arguments run through the field. One is **structure versus scale**: whether explicit linguistic structure (trees, roles, logical forms) still helps once models are large enough. Probing studies such as [BERT Rediscovers the Classical NLP Pipeline](https://md.research.elamur.ai/free/en/papers/tenney-2019-bert-rediscovers-classical-nlp-pipeline/), [a structural probe for syntax](https://md.research.elamur.ai/free/en/papers/hewitt-2019-structural-probe-finding-syntax-word/) and [tests of syntactic agreement in LSTMs](https://md.research.elamur.ai/free/en/papers/linzen-2016-assessing-ability-lstms-learn-syntax/) found that much classical structure is learned implicitly, though unevenly. A second is **form versus meaning**: [Climbing towards NLU](https://md.research.elamur.ai/free/en/papers/bender-2020-climbing-nlu-meaning-form-understanding/) argues that a system trained only on text form cannot in principle acquire meaning. A third concerns **explanation**: whether attention weights reveal why a model decided, debated in [Attention is not Explanation](https://md.research.elamur.ai/free/en/papers/jain-2019-attention-is-not-explanation/) and [Attention is not not Explanation](https://md.research.elamur.ai/free/en/papers/wiegreffe-2019-attention-is-not-not-explanation/); see [Interpretability](https://md.research.elamur.ai/free/en/topics/interpretability/). A fourth concerns **cost and harm**, from the energy use of training ([Energy and policy considerations for deep NLP](https://md.research.elamur.ai/free/en/papers/strubell-2019-energy-policy-considerations-deep-learning/)) to social bias ([Language (Technology) is Power](https://md.research.elamur.ai/free/en/papers/blodgett-2020-language-technology-is-power-critical/)).

## Datasets, resources and benchmarks

Shared data has driven NLP as much as algorithms. Key resources include:

- **Treebanks and annotated corpora.** The Penn Treebank for English syntax; [PropBank](https://md.research.elamur.ai/free/en/papers/palmer-2005-proposition-bank-annotated-corpus-semantic/), which adds predicate–argument labels on top of it; and the cross-lingual Universal Dependencies treebanks.
- **Lexical resources.** WordNet, the [Paraphrase Database](https://md.research.elamur.ai/free/en/papers/ganitkevitch-2013-ppdb-paraphrase-database/), and crowd-built [emotion lexicons](https://md.research.elamur.ai/free/en/papers/mohammad-2010-emotions-evoked-common-words-phrases/).
- **Parallel corpora** such as [OpenSubtitles2016](https://md.research.elamur.ai/free/en/papers/lison-2016-opensubtitles2016-extracting-large-parallel-corpora/), and web-scale text more generally, discussed early in [the Web-as-corpus special issue](https://md.research.elamur.ai/free/en/papers/kilgarriff-2003-introduction-special-issue-web-as/).
- **Reading comprehension and QA**: [SQuAD](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/) and [SQuAD 2.0](https://md.research.elamur.ai/free/en/papers/rajpurkar-2018-know-what-you-don-t/), [TriviaQA](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/), [Natural Questions](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/), [HotpotQA](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/), [CoQA](https://md.research.elamur.ai/free/en/papers/reddy-2019-coqa-conversational-question-answering-challenge/), [QuAC](https://md.research.elamur.ai/free/en/papers/choi-2018-quac-question-answering-context/) and [BoolQ](https://md.research.elamur.ai/free/en/papers/clark-2019-boolq-exploring-surprising-difficulty-natural/).
- **Understanding benchmarks**: [GLUE](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/) bundled sentence-level tasks into one leaderboard.
- **Fact checking and truthfulness**: [FEVER](https://md.research.elamur.ai/free/en/papers/thorne-2018-fever-large-scale-dataset-fact/), [LIAR](https://md.research.elamur.ai/free/en/papers/wang-2017-liar-liar-pants-fire-new/) and [TruthfulQA](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/).
- **Sentiment and social media**: movie-review corpora, the SemEval tasks on [aspect-based sentiment](https://md.research.elamur.ai/free/en/papers/pontiki-2014-semeval-2014-task-4-aspect/) and [sentiment in Twitter](https://md.research.elamur.ai/free/en/papers/rosenthal-2017-semeval-2017-task-4-sentiment/), and [offensive language](https://md.research.elamur.ai/free/en/papers/zampieri-2019-predicting-type-target-offensive-posts/).
- **Tools**: [NLTK](https://md.research.elamur.ai/free/en/papers/bird-2006-nltk-natural-language-toolkit/) for teaching, [brat](https://md.research.elamur.ai/free/en/papers/stenetorp-2012-brat-web-based-tool-nlp/) for annotation, [OpenNMT](https://md.research.elamur.ai/free/en/papers/klein-2017-opennmt-open-source-toolkit-neural/) and fairseq for sequence modelling.

Evaluation metrics form their own literature: BLEU for translation, and ROUGE for summaries, whose n-gram overlap scoring was validated in [Lin and Hovy (2003)](https://md.research.elamur.ai/free/en/papers/lin-2003-automatic-evaluation-summaries-n-gram/). Newer metrics try to escape reference texts altogether, such as [CLIPScore](https://md.research.elamur.ai/free/en/papers/hessel-2021-clipscore-reference-free-evaluation-metric/) for captions and [FActScore](https://md.research.elamur.ai/free/en/papers/min-2023-factscore-fine-grained-atomic-evaluation/) for factual precision. See [Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/).

## Landmark papers in this library

Citations of the papers on this shelf, by when they were published Papers published in 2015–2019 gather 35% of the 2,805,391 citations of the papers on this shelf.

**The numbers**

| Published | Papers and their citations |
| --- | --- |
| 2025–2029 | 1,142 papers, 21,722 citations |
| 2020–2024 | 2,632 papers, 839,473 citations |
| 2015–2019 | 1,628 papers, 979,218 citations |
| 2010–2014 | 692 papers, 404,055 citations |
| 2005–2009 | 433 papers, 163,658 citations |
| 2000–2004 | 262 papers, 216,013 citations |
| 1995–1999 | 145 papers, 106,508 citations |
| 1990–1994 | 86 papers, 65,550 citations |
| 1985–1989 | 34 papers, 6,697 citations |
| 1980–1984 | 7 papers, 227 citations |
| 1975–1979 | 5 papers, 744 citations |
| 1970–1974 | 1 papers, 1,486 citations |
| 1965–1969 | 1 papers, 0 citations |
| 1960–1964 | 1 papers, 4 citations |
| 1920–1924 | 2 papers, 0 citations |
| 1880–1884 | 1 papers, 0 citations |
| 1860–1864 | 1 papers, 0 citations |
| 1815–1819 | 1 papers, 0 citations |
| 1810–1814 | 1 papers, 0 citations |

### Transformers library (2020)

[Transformers: State-of-the-Art Natural Language Processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/) is not a new model but shared infrastructure. It wraps dozens of pretrained architectures behind one interface — tokenizer, encoder body, task head — and ties them to a community hub of downloadable weights. Its influence is visible in how quickly new models now spread from paper to practice.

### BART (2020)

[BART](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/) trains an encoder–decoder Transformer to reconstruct text that has been deliberately corrupted, for instance by shuffling sentences and replacing spans with a single mask. Because it combines a bidirectional encoder with a left-to-right decoder, the same model performs well on both classification and generation, especially abstractive summarization.

### SQuAD (2016)

[SQuAD](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/) asked crowdworkers to pose questions about Wikipedia paragraphs and mark the answer span. It was large enough to train neural readers and easy to score, so it became the reference benchmark for reading comprehension; the follow-up [SQuAD 2.0](https://md.research.elamur.ai/free/en/papers/rajpurkar-2018-know-what-you-don-t/) added questions with no answer in the text.

### Thumbs up? (2002)

[Thumbs up?](https://md.research.elamur.ai/free/en/papers/pang-2002-thumbs-up-sentiment-classification-machine/) framed the polarity of a movie review — favourable or not — as a supervised learning problem, with labels taken from star ratings. Learned classifiers outperformed word lists written by people, yet fell short of topic-classification accuracy — evidence that opinion is harder to read than subject matter. With [Turney's unsupervised method](https://md.research.elamur.ai/free/en/papers/turney-2002-thumbs-up-or-thumbs-down/) the same year, it launched sentiment analysis as a field.

### GLUE (2018)

[GLUE](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/) collected nine sentence-understanding tasks — inference, paraphrase, similarity, acceptability, sentiment — into a single benchmark with a diagnostic set. It became the scoreboard on which BERT-era models were compared, and its rapid saturation prompted harder successors.

### Maximum entropy models (1996)

[A maximum entropy approach to NLP](https://md.research.elamur.ai/free/en/papers/berger-1996-maximum-entropy-approach-natural-language/) showed how to build a probability model from arbitrary features while assuming nothing beyond what the data supports. Its feature-selection procedure and applications to translation subproblems made log-linear modelling the workhorse of the 2000s.

### Accurate Unlexicalized Parsing (2003)

[Accurate Unlexicalized Parsing](https://md.research.elamur.ai/free/en/papers/klein-2003-accurate-unlexicalized-parsing/) demonstrated that a plain treebank grammar, improved by systematically splitting its categories, could approach the accuracy of heavily lexicalised parsers. The result shaped the Stanford parser and led to the automatic state-splitting methods that followed.

### Learning Word Vectors for Sentiment Analysis (2011)

[Learning Word Vectors for Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/maas-2011-learning-word-vectors-sentiment-analysis/) learned word representations that capture both meaning and sentiment, and introduced the large IMDB movie-review dataset that remains a standard test for document classification.

### Hierarchical Attention Networks (2016)

[Hierarchical Attention Networks for Document Classification](https://md.research.elamur.ai/free/en/papers/yang-2016-hierarchical-attention-networks-document-classification/) mirrored the structure of documents: words build sentences, sentences build the document, and attention at each level picks out the parts that matter. It was among the most widely adopted neural text classifiers before pretraining took over.

### Energy and Policy Considerations (2019)

[Energy and policy considerations for deep NLP](https://md.research.elamur.ai/free/en/papers/strubell-2019-energy-policy-considerations-deep-learning/) estimated the electricity, carbon emissions and cloud cost of training large NLP models, including architecture search. It made the environmental and economic cost of research a mainstream topic and called for reporting training cost alongside accuracy.

### The Proposition Bank (2005)

[The Proposition Bank](https://md.research.elamur.ai/free/en/papers/palmer-2005-proposition-bank-annotated-corpus-semantic/) added verb-specific semantic roles to the Penn Treebank, giving shallow semantic analysis a common training resource. Together with [Gildea and Jurafsky's role labeller](https://md.research.elamur.ai/free/en/papers/gildea-2002-automatic-labeling-semantic-roles/) it defined semantic role labelling as a task.

### Recognizing Contextual Polarity (2005)

[Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/wilson-2005-recognizing-contextual-polarity-phrase-level/) observed that a word's prior polarity often flips or vanishes in context — through negation, intensifiers or neutral use. Its two-step classifier and subjectivity lexicon became standard resources for fine-grained opinion analysis.

### Natural Questions (2019)

[Natural Questions](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/) used real anonymised search-engine queries paired with Wikipedia pages, annotated with long and short answers. Because the questions were asked by people who did not know the answer, it offered a more realistic test of open-domain QA. It became a standard evaluation for the retrieve-then-read line of systems that began with [Reading Wikipedia to Answer Open-Domain Questions](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/) and continued with [retrieval-augmented generative readers](https://md.research.elamur.ai/free/en/papers/izacard-2021-leveraging-passage-retrieval-generative-models/).

### A Diversity-Promoting Objective (2016)

[A diversity-promoting objective for conversation models](https://md.research.elamur.ai/free/en/papers/li-2016-diversity-promoting-objective-function-neural/) tackled the tendency of neural chatbots to answer everything with bland, safe replies. Ranking candidate responses by mutual information with the input yielded more varied and relevant output, a theme later developed in [persona-grounded dialogue](https://md.research.elamur.ai/free/en/papers/zhang-2018-personalizing-dialogue-agents-i-have/) and [open-domain chatbot recipes](https://md.research.elamur.ai/free/en/papers/roller-2021-recipes-building-open-domain-chatbot/).

## People

Authors with the most papers on this shelf Christopher Manning leads with 76 papers; the 12 authors shown wrote 607 of the papers on this shelf between them, counting each paper once per author.

**The numbers**

| Author | Papers here · their citations |
| --- | --- |
| [Christopher Manning](https://md.research.elamur.ai/free/en/authors/christopher-d-manning/) | 76 papers · 68,189 citations |
| [Nizar Habash](https://md.research.elamur.ai/free/en/authors/nizar-habash/) | 75 papers · 7,295 citations |
| [Luke Zettlemoyer](https://md.research.elamur.ai/free/en/authors/luke-zettlemoyer/) | 69 papers · 46,992 citations |
| [Noah A. Smith](https://md.research.elamur.ai/free/en/authors/noah-a-smith/) | 52 papers · 18,178 citations |
| [Yoav Goldberg](https://md.research.elamur.ai/free/en/authors/yoav-goldberg/) | 47 papers · 14,938 citations |
| [Yejin Choi](https://md.research.elamur.ai/free/en/authors/yejin-choi/) | 45 papers · 18,370 citations |
| [Massimo Poesio](https://md.research.elamur.ai/free/en/authors/massimo-poesio/) | 45 papers · 5,023 citations |
| [Graham Neubig](https://md.research.elamur.ai/free/en/authors/graham-neubig/) | 44 papers · 19,138 citations |
| [Philipp Koehn](https://md.research.elamur.ai/free/en/authors/philipp-koehn/) | 42 papers · 18,798 citations |
| [Iryna Gurevych](https://md.research.elamur.ai/free/en/authors/iryna-gurevych/) | 42 papers · 12,789 citations |
| [Hannaneh Hajishirzi](https://md.research.elamur.ai/free/en/authors/hannaneh-hajishirzi/) | 35 papers · 15,018 citations |
| [Vincent Ng](https://md.research.elamur.ai/free/en/authors/vincent-ng/) | 35 papers · 4,358 citations |

- **[Christopher D. Manning](https://md.research.elamur.ai/free/en/authors/christopher-d-manning/)** (Stanford University) is the most prolific author on this shelf. His papers here span unlexicalised parsing, feature-rich tagging, CRF-based entity recognition with long-distance constraints and, later, multi-hop question answering with HotpotQA.
- **[Dan Klein](https://md.research.elamur.ai/free/en/authors/dan-klein/)** (University of California, Berkeley) co-authored the unlexicalised parser and developed latent-annotation grammars with Slav Petrov, together defining the high-water mark of statistical constituency parsing.
- **[Daniel Jurafsky](https://md.research.elamur.ai/free/en/authors/dan-jurafsky/)** (Stanford University) is represented by the founding paper on automatic semantic role labelling, by diachronic word embeddings measuring semantic change, and by neural dialogue generation.
- **[Eduard Hovy](https://md.research.elamur.ai/free/en/authors/eduard-hovy/)** (The University of Melbourne) appears through the ROUGE evaluation study, early opinion mining, pattern-based question answering and hierarchical attention for documents.
- **[Regina Barzilay](https://md.research.elamur.ai/free/en/authors/regina-barzilay/)** (MIT) contributed lexical-chain summarization, paraphrase extraction, the entity-grid model of coherence with Mirella Lapata, and rationale-based neural explanation.
- **[Mirella Lapata](https://md.research.elamur.ai/free/en/authors/mirella-lapata/)** (University of Edinburgh) works on summarization, coherence and semantic space models, including early neural extractive summarizers.
- **[Martha Palmer](https://md.research.elamur.ai/free/en/authors/martha-palmer/)** led the PropBank annotation effort and co-authored the Wu–Palmer similarity measure.
- **[Luke Zettlemoyer](https://md.research.elamur.ai/free/en/authors/luke-zettlemoyer/)** (University of Washington, Meta) is behind BART, TriviaQA, QuAC and a study of why demonstrations help in-context learning.
- **[Noah A. Smith](https://md.research.elamur.ai/free/en/authors/noah-a-smith/)** (Allen Institute) has papers here on toxicity in generated text, racial bias in hate-speech detection, probing contextual representations and the interpretability of attention.
- **[Yejin Choi](https://md.research.elamur.ai/free/en/authors/yejin-choi/)** (Stanford University) co-authored RealToxicityPrompts, CLIPScore and studies of misinformation language.
- **[Percy Liang](https://md.research.elamur.ai/free/en/authors/percy-liang/)** (Stanford University) created SQuAD and its unanswerable-question extension, and has more recently benchmarked large models on summarization.
- **[Nizar Habash](https://md.research.elamur.ai/free/en/authors/nizar-habash/)** (New York University Abu Dhabi) is the leading figure in Arabic NLP here: morphological tagging and tokenisation, preprocessing for translation, and shared tasks on dialect identification.
- **[Reut Tsarfaty](https://md.research.elamur.ai/free/en/authors/reut-tsarfaty/)** (Bar-Ilan University) works on parsing morphologically rich languages, especially Hebrew, from joint morpho-syntactic models to the AlephBERT language model; [Avi Shmidman](https://md.research.elamur.ai/free/en/authors/avi-shmidman/) (Bar-Ilan) builds Hebrew diacritisation tools and Hebrew BERT and LLM models.
- **[Iryna Gurevych](https://md.research.elamur.ai/free/en/authors/iryna-gurevych/)** (Technische Universität Darmstadt) co-authored the adapter frameworks AdapterHub and MAD-X and work on argument mining.
- **[Graham Neubig](https://md.research.elamur.ai/free/en/authors/graham-neubig/)** (Carnegie Mellon University) has papers on probing what language models know, joint text–table pretraining and entity recognition for African languages.
- **[Kyunghyun Cho](https://md.research.elamur.ai/free/en/authors/kyunghyun-cho/)** (New York University) worked on large-vocabulary and multilingual neural translation and on sentence representations.
- **[Philipp Koehn](https://md.research.elamur.ai/free/en/authors/philipp-koehn/)** (Johns Hopkins University) shaped statistical machine translation with factored models, clause reordering and compound splitting.
- **[Yoav Goldberg](https://md.research.elamur.ai/free/en/authors/yoav-goldberg/)** (Bar-Ilan University) contributed to syntactic probing of LSTMs, faithful interpretability and removing protected attributes from representations.
- **[Preslav Nakov](https://md.research.elamur.ai/free/en/authors/preslav-nakov/)** (Mohamed bin Zayed University of Artificial Intelligence) co-organised a long series of SemEval tasks on sentiment in Twitter and on offensive language.

Other names that recur among the most cited works include [Bo Pang](https://md.research.elamur.ai/free/en/authors/bo-pang/) and [Lillian Lee](https://md.research.elamur.ai/free/en/authors/lillian-lee/) for sentiment analysis, [Michael Collins](https://md.research.elamur.ai/free/en/authors/michael-collins/) for statistical parsing, [Barbara J. Grosz](https://md.research.elamur.ai/free/en/authors/barbara-j-grosz/) for discourse theory, [Samuel R. Bowman](https://md.research.elamur.ai/free/en/authors/samuel-r-bowman/) for benchmarks and sentence generation, [Danqi Chen](https://md.research.elamur.ai/free/en/authors/danqi-chen/) for open-domain QA and few-shot fine-tuning, and [Jason Weston](https://md.research.elamur.ai/free/en/authors/jason-weston/) for dialogue and question answering.

## Institutions and places

Where the authors on this shelf work now Google is the current home of authors of 427 papers on this shelf; counted are the authors with a page here, by their latest institution.

**The numbers**

| Institution | Papers |
| --- | --- |
| [Google](https://md.research.elamur.ai/free/en/companies/google/) | 427 papers |
| [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) | 422 papers |
| [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) | 415 papers |
| [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) | 330 papers |
| [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) | 302 papers |
| [New York University](https://md.research.elamur.ai/free/en/universities/new-york-university/) | 235 papers |
| [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) | 234 papers |
| [Google DeepMind](https://md.research.elamur.ai/free/en/companies/google-deepmind/) | 225 papers |
| [University of Edinburgh](https://md.research.elamur.ai/free/en/universities/university-of-edinburgh/) | 207 papers |
| [Allen Institute for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/allen-institute-for-artificial-intelligence/) | 205 papers |

The field's centre of gravity is in the United States. [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/), [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/), the [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) and [New York University](https://md.research.elamur.ai/free/en/universities/new-york-university/) are the leading academic homes of authors on this shelf, joined by [MIT](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/), [Johns Hopkins University](https://md.research.elamur.ai/free/en/universities/johns-hopkins-university/), [Princeton University](https://md.research.elamur.ai/free/en/universities/princeton-university/) and the [University of Illinois Urbana-Champaign](https://md.research.elamur.ai/free/en/universities/university-of-illinois-urbana-champaign/). Industry laboratories are unusually prominent: [Google](https://md.research.elamur.ai/free/en/companies/google/), [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/), [Google DeepMind](https://md.research.elamur.ai/free/en/companies/google-deepmind/), [Amazon](https://md.research.elamur.ai/free/en/companies/amazon/), [Meta](https://md.research.elamur.ai/free/en/companies/meta/) and the nonprofit [Allen Institute for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/allen-institute-for-artificial-intelligence/) all employ many leading authors.

In Europe the [University of Edinburgh](https://md.research.elamur.ai/free/en/universities/university-of-edinburgh/) has long been a stronghold, especially for translation, summarization and semantics. In China, [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/), [Peking University](https://md.research.elamur.ai/free/en/universities/peking-university/) and the [Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/government/chinese-academy-of-sciences/) are major centres. [Bar-Ilan University](https://md.research.elamur.ai/free/en/universities/bar-ilan-university/) in Israel and [Mohamed bin Zayed University of Artificial Intelligence](https://md.research.elamur.ai/free/en/universities/mohamed-bin-zayed-university-of-artificial-intelligence/) in the United Arab Emirates stand out for Hebrew and Arabic NLP respectively, reflecting this library's strong collection on Semitic languages.

Countries of the authors on this shelf United States comes first: authors working there now wrote 3,339 papers on this shelf.

**The numbers**

| Country | Papers |
| --- | --- |
| [United States](https://md.research.elamur.ai/free/en/countries/united-states/) | 3,339 papers |
| [China](https://md.research.elamur.ai/free/en/countries/china/) | 1,387 papers |
| [United Kingdom](https://md.research.elamur.ai/free/en/countries/united-kingdom/) | 1,025 papers |
| [Germany](https://md.research.elamur.ai/free/en/countries/germany/) | 621 papers |
| [Canada](https://md.research.elamur.ai/free/en/countries/canada/) | 359 papers |
| [Israel](https://md.research.elamur.ai/free/en/countries/israel/) | 308 papers |
| Hong Kong | 291 papers |
| [France](https://md.research.elamur.ai/free/en/countries/france/) | 284 papers |
| [United Arab Emirates](https://md.research.elamur.ai/free/en/countries/united-arab-emirates/) | 272 papers |
| [Netherlands](https://md.research.elamur.ai/free/en/countries/netherlands/) | 252 papers |

The main venues are those of the Association for Computational Linguistics: its annual meeting, EMNLP and NAACL, the journal *Computational Linguistics* and *Transactions of the ACL*, plus many workshops and SemEval shared tasks. Most papers are open access through the ACL Anthology, which is one reason the field's literature is so complete here.

Where the papers on this shelf were published Annual Meeting of the Association for Computational Linguistics (ACL) carries the most, 996 of 7081.

**The numbers**

| Venue | Papers |
| --- | --- |
| [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/) | 996 papers |
| [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/) | 693 papers |
| [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/) | 680 papers |
| [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/) | 318 papers |
| [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/) | 231 papers |
| [International Conference on Computational Linguistics (COLING)](https://md.research.elamur.ai/free/en/venues/coling/) | 225 papers |
| [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/) | 170 papers |
| [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/) | 129 papers |
| [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/) | 128 papers |
| [Conference on Computational Natural Language Learning (CoNLL)](https://md.research.elamur.ai/free/en/venues/conll/) | 110 papers |

## Open questions

- **Meaning and grounding.** Do language models that learn only from text truly understand it, and how much does grounding in perception or action add? Work in [Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) — from [LXMERT](https://md.research.elamur.ai/free/en/papers/tan-2019-lxmert-learning-cross-modality-encoder/) to studies of [object hallucination in vision-language models](https://md.research.elamur.ai/free/en/papers/li-2023-evaluating-object-hallucination-large-vision/) — partly addresses this.
- **Factuality.** Generators produce fluent text that may be false. Measuring and reducing hallucination, as in TruthfulQA and FActScore, remains unsolved.
- **Reasoning.** Whether models reason or pattern-match, and how to make multi-step inference reliable; see [Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/).
- **Evaluation.** Benchmarks saturate quickly and may leak into training data; reference-based metrics correlate poorly with human judgement for open generation.
- **Bias, fairness and safety.** Models absorb social biases from data, as shown in [StereoSet](https://md.research.elamur.ai/free/en/papers/nadeem-2021-stereoset-measuring-stereotypical-bias-pretrained/), [RealToxicityPrompts](https://md.research.elamur.ai/free/en/papers/gehman-2020-realtoxicityprompts-evaluating-neural-toxic-degeneration/) and [racial bias in hate-speech data](https://md.research.elamur.ai/free/en/papers/sap-2019-risk-racial-bias-hate-speech/); see [Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/).
- **Languages beyond English.** Most resources still favour a handful of languages. [How Multilingual is Multilingual BERT?](https://md.research.elamur.ai/free/en/papers/pires-2019-how-multilingual-is-multilingual-bert/) and [cross-lingual transfer of monolingual models](https://md.research.elamur.ai/free/en/papers/artetxe-2020-cross-lingual-transferability-monolingual-representations/) show how far transfer goes; see [Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) and [Low-resource languages](https://md.research.elamur.ai/free/en/topics/low-resource-languages/).
- **Efficiency.** Training and serving large models is costly in energy and money; see [Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/).
- **Interpretability.** Explaining model decisions faithfully, beyond attention maps; see [Interpretability](https://md.research.elamur.ai/free/en/topics/interpretability/).

## Neighbouring topics

NLP overlaps most with [Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/), which supplies its theories and many of its resources, and with the [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) shelf, historically the field's first application. It draws its learning methods from Artificial Intelligence and its hardware from [Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/). Applied strands reach into [Law](https://md.research.elamur.ai/free/en/shelves/social-sciences/law/) through [Legal AI](https://md.research.elamur.ai/free/en/topics/legal-ai/), into [History](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/history/) through work on ancient and historical languages, into [Medicine](https://md.research.elamur.ai/free/en/shelves/medicine/) through biomedical text mining such as [SciBERT](https://md.research.elamur.ai/free/en/papers/beltagy-2019-scibert-pretrained-language-model-scientific/) and [PubMedQA](https://md.research.elamur.ai/free/en/papers/jin-2019-pubmedqa-dataset-biomedical-research-question/), and into [Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) and [Education](https://md.research.elamur.ai/free/en/shelves/social-sciences/education/).

Neighbouring shelves: papers shared Language and Linguistics shares the most: 6478 of the 7,081 papers here (91%) stand on both.

**The numbers**

| Shelf | Papers shared |
| --- | --- |
| [Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) | 6478 shared · 91% |
| [Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) | 921 shared · 13% |
| [Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) | 361 shared · 5% |
| [Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) | 252 shared · 4% |
| [Literature and Literary Theory](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/literature-and-literary-theory/) | 168 shared · 2% |
| [Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) | 159 shared · 2% |
| [Signal Processing](https://md.research.elamur.ai/free/en/shelves/computer-science/signal-processing/) | 144 shared · 2% |
| [Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) | 137 shared · 2% |
| [Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) | 106 shared · 1% |
| [Philosophy](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/philosophy/) | 106 shared · 1% |

Within the shelf, the largest topics are machine translation and language models, followed by a distinctive cluster on [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) and [Hebrew & Aramaic](https://md.research.elamur.ai/free/en/topics/hebrew-aramaic/), then evaluation, multilingual NLP, sentiment and morphology. Smaller but active topics include [Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/), [Psycholinguistics & language acquisition](https://md.research.elamur.ai/free/en/topics/psycholinguistics/), [Historical linguistics & language change](https://md.research.elamur.ai/free/en/topics/language-change/), [Grammatical error correction & learner language](https://md.research.elamur.ai/free/en/topics/grammatical-error-correction/), [Idioms & multiword expressions](https://md.research.elamur.ai/free/en/topics/multiword-expressions/) and [Code & programming languages](https://md.research.elamur.ai/free/en/topics/code-and-programming/).

The topics on this shelf Language models & pretraining: 1014 of the 7,081 papers here.

**The numbers**

| Topic | Papers |
| --- | --- |
| [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) | 1,014 papers |
| [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) | 906 papers |
| [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) | 864 papers |
| [English](https://md.research.elamur.ai/free/en/topics/english/) | 772 papers |
| [Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) | 565 papers |
| [Coreference resolution](https://md.research.elamur.ai/free/en/topics/coreference-resolution/) | 513 papers |
| [Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) | 459 papers |
| [Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) | 368 papers |
| [Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) | 342 papers |
| [Discourse analysis](https://md.research.elamur.ai/free/en/topics/discourse-analysis/) | 321 papers |
| [Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) | 241 papers |
| [Morphology](https://md.research.elamur.ai/free/en/topics/morphology/) | 210 papers |

## Key works in this library

- [A stochastic parts program](https://md.research.elamur.ai/free/en/papers/church-1988-stochastic-parts-program-noun-phrase/) — early proof that probabilistic tagging works on real text.
- [Attention, Intentions, and the Structure of Discourse](https://md.research.elamur.ai/free/en/papers/grosz-1986-attention-intentions-structure-discourse/) — a three-part theory of discourse structure.
- [Assessing agreement: the kappa statistic](https://md.research.elamur.ai/free/en/papers/carletta-1996-assessing-agreement-classification-tasks-kappa/) — the case for chance-corrected agreement in annotation.
- [A maximum entropy approach to NLP](https://md.research.elamur.ai/free/en/papers/berger-1996-maximum-entropy-approach-natural-language/) — log-linear models for language.
- [Automatic Retrieval and Clustering of Similar Words](https://md.research.elamur.ai/free/en/papers/lin-1998-automatic-retrieval-clustering-similar-words/) — a thesaurus built from parsed text.
- [Thumbs up?](https://md.research.elamur.ai/free/en/papers/pang-2002-thumbs-up-sentiment-classification-machine/) — supervised sentiment classification of reviews.
- [Automatic Labeling of Semantic Roles](https://md.research.elamur.ai/free/en/papers/gildea-2002-automatic-labeling-semantic-roles/) — the first statistical role labeller.
- [Accurate Unlexicalized Parsing](https://md.research.elamur.ai/free/en/papers/klein-2003-accurate-unlexicalized-parsing/) — careful category splitting rivals lexicalised parsers.
- [Head-Driven Statistical Models for Natural Language Parsing](https://md.research.elamur.ai/free/en/papers/collins-2003-head-driven-statistical-models-natural/) — the classic lexicalised parser.
- [Non-local information in extraction via Gibbs sampling](https://md.research.elamur.ai/free/en/papers/finkel-2005-incorporating-non-local-information-into/) — long-distance constraints in entity recognition.
- [The Proposition Bank](https://md.research.elamur.ai/free/en/papers/palmer-2005-proposition-bank-annotated-corpus-semantic/) — semantic roles over the Penn Treebank.
- [Domain Adaptation with Structural Correspondence Learning](https://md.research.elamur.ai/free/en/papers/blitzer-2006-domain-adaptation-structural-correspondence-learning/) — transferring models to new domains via pivot features.
- [Hierarchical Phrase-Based Translation](https://md.research.elamur.ai/free/en/papers/chiang-2007-hierarchical-phrase-based-translation/) — synchronous grammars for statistical translation.
- [Linguistic Regularities in Continuous Space Word Representations](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/) — word vectors capture analogies as offsets.
- [SQuAD](https://md.research.elamur.ai/free/en/papers/rajpurkar-2016-squad-100-000-questions-machine/) — the reading comprehension benchmark.
- [Diachronic word embeddings and laws of semantic change](https://md.research.elamur.ai/free/en/papers/hamilton-2016-diachronic-word-embeddings-reveal-statistical/) — embeddings over time trace meaning shift.
- [GLUE, a multi-task understanding benchmark](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/) — one leaderboard for sentence understanding.
- [Energy and policy considerations for deep NLP](https://md.research.elamur.ai/free/en/papers/strubell-2019-energy-policy-considerations-deep-learning/) — the carbon and money cost of training.
- [A Primer in BERTology](https://md.research.elamur.ai/free/en/papers/rogers-2020-primer-bertology-what-we-know/) — a survey of what probing found inside BERT.
- [BART: denoising sequence-to-sequence pretraining](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/) — denoising pretraining for generation and understanding.
- [Transformers: State-of-the-Art Natural Language Processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/) — the shared library of pretrained models.
- [Rethinking the role of demonstrations](https://md.research.elamur.ai/free/en/papers/min-2022-rethinking-role-demonstrations-what-makes/) — what in-context examples actually contribute.
- [TruthfulQA: Measuring How Models Mimic Human Falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/) — testing whether models repeat popular misconceptions.
- [FActScore](https://md.research.elamur.ai/free/en/papers/min-2023-factscore-fine-grained-atomic-evaluation/) — scoring long generations fact by fact.

## Papers

**1** [2](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/page/2/) … [72](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/page/72/) [next →](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/page/2/)

1. [**Very Deep Convolutional Networks for Large-Scale Image Recognition**Karen Simonyan, Andrew Zisserman*2014 · 72,803 citations*](https://md.research.elamur.ai/free/en/papers/simonyan-2014-very-deep-convolutional-networks-large/)
2. [**SciPy 1.0 — Fundamental Algorithms for Scientific Computing in Python**Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, et al.*2019 · 38,317 citations*](https://md.research.elamur.ai/free/en/papers/virtanen-2019-scipy-1-0-fundamental-algorithms/)
3. [**Swin Transformer: Hierarchical Vision Transformer using Shifted Windows**Ze Liu, Yutong Lin, Yue Cao, et al.*2021 · 33,826 citations*](https://md.research.elamur.ai/free/en/papers/liu-2021-swin-transformer-hierarchical-vision-transformer/)
4. [**SMOTE: Synthetic Minority Over-sampling Technique**Nitesh V. Chawla, Kevin W. Bowyer, Lawrence Hall, et al.*2002 · 32,661 citations*](https://md.research.elamur.ai/free/en/papers/chawla-2002-smote-synthetic-minority-over-sampling/)
5. [**Bleu: a Method for Automatic Evaluation of Machine Translation**Kishore Papineni, Salim Roukos, Todd Ward, et al.*2002 · 32,065 citations*](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
6. [**Rich feature hierarchies for accurate object detection and semantic segmentation**Ross Girshick, Jeff Donahue, Trevor Darrell, et al.*2013 · 31,715 citations*](https://md.research.elamur.ai/free/en/papers/girshick-2013-rich-feature-hierarchies-accurate-object/)
7. [**Latent Dirichlet Allocation**David M. Blei, Andrew Y. Ng, Michael I. Jordan*2003 · 28,297 citations*](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/)
8. [**C4.5: Programs for Machine Learning**J. Ross Quinlan*1993 · 23,781 citations*](https://md.research.elamur.ai/free/en/papers/quinlan-1993-c45-programs-machine-learning/)
9. [**Array Programming with NumPy**Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, et al.*2020 · 23,591 citations*](https://md.research.elamur.ai/free/en/papers/harris-2020-array-programming-numpy/)
10. [**BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding**Jacob Devlin, Ming-Wei Chang, Kenton Lee, et al.*2019 · 23,422 citations*](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/)
11. [**Neural Machine Translation by Jointly Learning to Align and Translate**Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio*2014 · 18,908 citations*](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
12. [**PyTorch: An Imperative Style, High-Performance Deep Learning Library**Adam Paszke, Sam Gross, Francisco Massa, et al.*2019 · 18,392 citations*](https://md.research.elamur.ai/free/en/papers/paszke-2019-pytorch-imperative-style-high-performance/)
13. [**End-to-End Object Detection with Transformers**Nicolas Carion, Francisco Massa, Gabriel Synnaeve, et al.*2020 · 16,761 citations*](https://md.research.elamur.ai/free/en/papers/carion-2020-end-end-object-detection-transformers/)
14. [**Bagging Predictors**Leo Breiman*1996 · 16,493 citations*](https://md.research.elamur.ai/free/en/papers/breiman-1996-bagging-predictors/)
15. [**Attention Is All You Need**Ashish Vaswani, Noam Shazeer, Niki Parmar, et al.*2017 · 16,413 citations*](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/)
16. [**The Anatomy of a Large-Scale Hypertextual Web Search Engine**Sergey Brin, Lawrence Page*1998 · 16,227 citations*](https://md.research.elamur.ai/free/en/papers/brin-1998-anatomy-large-scale-hypertextual-web/)
17. [**Convolutional Neural Networks for Sentence Classification**Yoon Kim*2014 · 16,218 citations*](https://md.research.elamur.ai/free/en/papers/kim-2014-convolutional-neural-networks-sentence-classification/)
18. [**Transformers: State-of-the-Art Natural Language Processing**Thomas Wolf, Lysandre Debut, Victor Sanh, et al.*2020 · 15,591 citations*](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
19. [**Distilling the Knowledge in a Neural Network**Geoffrey Hinton, Oriol Vinyals, Jeff Dean*2015 · 15,026 citations*](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/)
20. [**Auto-Encoding Variational Bayes**Diederik P Kingma, Max Welling*2013 · 14,991 citations*](https://md.research.elamur.ai/free/en/papers/kingma-2013-auto-encoding-variational-bayes/)
21. [**Momentum Contrast for Unsupervised Visual Representation Learning**Kaiming He, Haoqi Fan, Yuxin Wu, et al.*2019 · 13,143 citations*](https://md.research.elamur.ai/free/en/papers/he-2019-momentum-contrast-unsupervised-visual-representation/)
22. [**BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension**Mike Lewis, Yinhan Liu, Naman Goyal, et al.*2020 · 12,752 citations*](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/)
23. [**GPT-4 Technical Report**OpenAI, Josh Achiam, Steven Adler, et al.*2023 · 12,694 citations*](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
24. [**Chain-of-Thought Prompting Elicits Reasoning in Large Language Models**Jason Wei, Xuezhi Wang, Dale Schuurmans, et al.*2022 · 11,278 citations*](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
25. [**Language Models are Few-Shot Learners**Tom B. Brown, Benjamin Mann, Nick Ryder, et al.*2020 · 10,782 citations*](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)

**1** [2](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/page/2/) … [72](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/page/72/) [next →](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/page/2/)

Updated 23 October 2026

- [next subfield → Computational Theory and Mathematics](https://md.research.elamur.ai/free/en/shelves/computer-science/computational-theory-and-mathematics/)
- [↑ back to Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/)