---
title: "Computer Science — papers — Elamur Research"
description: "Computer Science: 9,085 papers of the Elamur Research library · page 1 of 92 · 1–25, the most cited first — our summary of each, its ideas…"
canonical: "https://research.elamur.ai/free/en/shelves/computer-science/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/shelves/computer-science/)

[home](https://md.research.elamur.ai/free/en/) / [shelves](https://md.research.elamur.ai/free/en/shelves/) / Computer Science

# 🖥️ Computer Science

9,085 papers · page 1 of 92 · 1–25 · the most cited first · [all shelves](https://md.research.elamur.ai/free/en/shelves/)

Artificial intelligence, language technology, vision, systems, networks, software and the theory of computing.

## Subfields

The sub-shelves of Computer Science, the most papers first, with the two OpenAlex topics most of their papers here are on.

1. [**Artificial Intelligence**Natural Language Processing Techniques · Topic Modeling*8,885 papers*](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/)
2. [**Computer Vision and Pattern Recognition**Multimodal Machine Learning Applications · Advanced Neural Network Applications*1,282 papers*](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/)
3. [**Information Systems**Information Retrieval and Search Behavior · Software Engineering Research*523 papers*](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/)
4. [**Signal Processing**Speech and Audio Processing · Music and Audio Processing*213 papers*](https://md.research.elamur.ai/free/en/shelves/computer-science/signal-processing/)
5. [**Computational Theory and Mathematics**Mathematics, Computing, and Information Processing · semigroups and automata theory*116 papers*](https://md.research.elamur.ai/free/en/shelves/computer-science/computational-theory-and-mathematics/)
6. [**Human-Computer Interaction**Digital Communication and Language · Hand Gesture Recognition Systems*61 papers*](https://md.research.elamur.ai/free/en/shelves/computer-science/human-computer-interaction/)
7. [**Computer Science Applications**Mobile Crowdsensing and Crowdsourcing · Teaching and Learning Programming*42 papers*](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-science-applications/)
8. [**Hardware and Architecture**Parallel Computing and Optimization Techniques · Energy Efficiency in Computing*38 papers*](https://md.research.elamur.ai/free/en/shelves/computer-science/hardware-and-architecture/)
9. [**Software**Software Testing and Debugging Techniques · Model-Driven Software Engineering Techniques*25 papers*](https://md.research.elamur.ai/free/en/shelves/computer-science/software/)
10. [**Computer Networks and Communications**Advanced Database Systems and Queries · Constraint Satisfaction and Optimization*19 papers*](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-networks-and-communications/)
11. [**Computer Graphics and Computer-Aided Design**Digital Media and Visual Art*1 paper*](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-graphics-and-computer-aided-design/)

10 papers of the field stand on none of its subfields.

A survey · about 17 minutes to read · every paper named here has its own page in this library

**In this article**

1. [What the field covers](#a-what-the-field-covers)
2. [History and milestones](#a-history-and-milestones)
3. [Main problems and tasks](#a-main-problems-and-tasks)
4. [Methods and schools](#a-methods-and-schools)
5. [Datasets, resources and benchmarks](#a-datasets-resources-and-benchmarks)
6. [Landmark papers in this library](#a-landmark-papers-in-this-library)
7. [People](#a-people)
8. [Institutions and places](#a-institutions-and-places)
9. [Open questions](#a-open-questions)
10. [Neighbouring topics](#a-neighbouring-topics)
11. [Key works in this library](#a-key-works-in-this-library)

The Computer Science shelf holds every paper of this library that OpenAlex places in computer science. Most of them are language technology, surveyed on the [Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) sub-shelf; this survey is about the rest, the papers whose main contribution lies in computer science proper rather than in the study of language: algorithms for search and ranking on the web, systems that train and serve large neural networks on GPUs, compilers for tensor programs, tools for organising and clustering large text collections, and models that read and write source code. Many of them were written by language researchers, and most of them handle text in some form, but their questions are about efficiency, scale, data structures and software.

The shelf is unusually spread out in time. Some of its most cited works date from the late 1990s, when the web was young and the problem of the day was how to rank millions of pages; others are from the last few years, when the problem became how to fit a model with billions of parameters into GPU memory and answer thousands of requests a second. Between them lie two decades of probabilistic topic models, learning-to-rank, dense retrieval and code generation.

What links these strands is infrastructure. The ranking methods here decide what search engines return, the retrieval libraries and embedding models feed retrieval-augmented systems, the serving and quantisation papers determine what it costs to run a language model, and the code benchmarks define how progress in programming assistants is measured. The shelf is a map of the machinery beneath modern language technology.

## What the field covers

The shelf groups several areas of computer science that meet language processing.

**Web search and information retrieval.** Link analysis ([PageRank](https://md.research.elamur.ai/free/en/papers/page-1999-pagerank-citation-ranking-bringing/), hubs and authorities), probabilistic and language-model ranking functions, learning to rank, and, since 2019, neural retrievers that encode queries and documents as vectors. This strand overlaps heavily with the topic [Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/).

**Text mining, topic models and clustering.** Latent-variable models that discover themes in document collections, metrics for judging whether those themes make sense to people, classic text classification, and general-purpose tools for clustering and dimension reduction.

**Systems for large models.** Distributed training, memory management for inference, quantisation, long-context attention kernels, speculative and parallel decoding, and compilers that turn tensor programs into fast GPU code. The topics [GPU & parallel computing](https://md.research.elamur.ai/free/en/topics/gpu-parallel-computing/) and [Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) cover much of this.

**Code and programming languages.** Pretrained models of source code, program synthesis from natural language, text-to-SQL, and benchmarks that execute generated programs against tests. This is the core of [Code & programming languages](https://md.research.elamur.ai/free/en/topics/code-and-programming/).

**Language-processing infrastructure.** Tokenisers, language identifiers, efficient parsing algorithms and grammar formalisms written with complexity in mind, and tree transducers for syntax-based translation.

It differs from [Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/), its largest neighbour, in that the papers here usually treat language as data to be indexed, compressed or served rather than as a phenomenon to be explained. It differs from [Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) in its focus on engineering: throughput, memory, latency, and the software that ships the algorithms.

## History and milestones

Landmark papers on this shelf: the 12 most cited, in time The 12 most cited papers on this shelf span 1993–2021; the dot grows with the citations.

**The numbers**

| Paper | Citations |
| --- | --- |
| [1993 — C4.5: Programs for Machine Learning](https://md.research.elamur.ai/free/en/papers/quinlan-1993-c45-programs-machine-learning/) | 23,783 |
| [2002 — SMOTE: Synthetic Minority Over-sampling Technique](https://md.research.elamur.ai/free/en/papers/chawla-2002-smote-synthetic-minority-over-sampling/) | 32,663 |
| [2002 — Bleu: a Method for Automatic Evaluation of Machine Translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/) | 32,134 |
| [2003 — Latent Dirichlet Allocation](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/) | 28,336 |
| [2013 — Rich feature hierarchies for accurate object detection and semantic segmentation](https://md.research.elamur.ai/free/en/papers/girshick-2013-rich-feature-hierarchies-accurate-object/) | 31,724 |
| [2013 — Efficient Estimation of Word Representations in Vector Space](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/) | 28,676 |
| [2014 — Very Deep Convolutional Networks for Large-Scale Image Recognition](https://md.research.elamur.ai/free/en/papers/simonyan-2014-very-deep-convolutional-networks-large/) | 72,829 |
| [2015 — Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks](https://md.research.elamur.ai/free/en/papers/ren-2015-faster-r-cnn-towards-real/) | 64,758 |
| [2019 — BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/) | 55,571 |
| [2019 — SciPy 1.0 — Fundamental Algorithms for Scientific Computing in Python](https://md.research.elamur.ai/free/en/papers/virtanen-2019-scipy-1-0-fundamental-algorithms/) | 38,317 |
| [2020 — Array Programming with NumPy](https://md.research.elamur.ai/free/en/papers/harris-2020-array-programming-numpy/) | 23,591 |
| [2021 — Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://md.research.elamur.ai/free/en/papers/liu-2021-swin-transformer-hierarchical-vision-transformer/) | 33,834 |

### Probabilistic retrieval and efficient parsing (1994–2001)

The oldest paper on the shelf, [Okapi at TREC-3](https://md.research.elamur.ai/free/en/papers/robertson-1994-okapi-trec-3/), reports the system in which the BM25 weighting function took its familiar form; BM25 remained the default first-stage ranker for a quarter of a century. [A Language Modeling Approach to Information Retrieval](https://md.research.elamur.ai/free/en/papers/ponte-1998-language-modeling-approach-information/) proposed ranking documents by the probability that their language model generated the query, a framework that shaped academic IR for years. In parsing, [Three New Probabilistic Models for Dependency Parsing](https://md.research.elamur.ai/free/en/papers/eisner-1996-three-new-probabilistic-models-dependency/) and [Efficient Parsing for Bilexical Context-Free Grammars and Head Automaton Grammars](https://md.research.elamur.ai/free/en/papers/eisner-1999-efficient-parsing-bilexical-context-free/) introduced the cubic-time algorithms still used for projective dependency parsing, while [An Efficient Probabilistic Context-Free Parsing Algorithm that Computes Prefix Probabilities](https://md.research.elamur.ai/free/en/papers/stolcke-1995-efficient-probabilistic-context-free-parsing/) and [Fast Context-Free Grammar Parsing Requires Fast Boolean Matrix Multiplication](https://md.research.elamur.ai/free/en/papers/lee-2001-fast-context-free-grammar-parsing/) tied parsing to deeper questions of complexity.

### The link-analysis moment (1998–1999)

In 1998 and 1999 three papers changed web search. [The Anatomy of a Large-Scale Hypertextual Web Search Engine](https://md.research.elamur.ai/free/en/papers/brin-1998-anatomy-large-scale-hypertextual-web/) described the Google prototype built at Stanford; the companion technical report on PageRank defined page importance through the link graph and a random surfer; and [Authoritative Sources in a Hyperlinked Environment](https://md.research.elamur.ai/free/en/papers/kleinberg-1999-authoritative-sources-hyperlinked-environment/) introduced hubs and authorities. Around the same time, machine learning for text matured: [Text Categorization with Support Vector Machines](https://md.research.elamur.ai/free/en/papers/joachims-1998-text-categorization-support-vector-machines/) established SVMs for document classification, and [Combining Labeled and Unlabeled Data with Co-Training](https://md.research.elamur.ai/free/en/papers/blum-1998-combining-labeled-unlabeled-data-co-training/) founded multi-view semi-supervised learning using web pages as its example. The papers of this period gather a large share of all citations on the shelf.

### Topic models and learning to rank (1999–2015)

[Probabilistic Latent Semantic Analysis](https://md.research.elamur.ai/free/en/papers/hofmann-1999-probabilistic-latent-semantic-analysis/) recast latent semantic indexing as a mixture model. [Latent Dirichlet Allocation](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/), the most cited paper on the shelf, made it fully generative with Dirichlet priors and launched a decade of topic modelling, including [Dynamic Topic Models](https://md.research.elamur.ai/free/en/papers/blei-2006-dynamic-topic-models/). Researchers then asked whether topics are interpretable: [Reading Tea Leaves](https://md.research.elamur.ai/free/en/papers/chang-2009-reading-tea-leaves-humans-interpret-topic/) measured it with people, and [Automatic Evaluation of Topic Coherence](https://md.research.elamur.ai/free/en/papers/newman-2010-automatic-evaluation-topic-coherence/), [Optimizing Semantic Coherence in Topic Models](https://md.research.elamur.ai/free/en/papers/mimno-2011-optimizing-semantic-coherence-topic-models/), [Machine Reading Tea Leaves](https://md.research.elamur.ai/free/en/papers/lau-2014-machine-reading-tea-leaves-automatically/) and [Exploring the Space of Topic Coherence Measures](https://md.research.elamur.ai/free/en/papers/roder-2015-exploring-space-topic-coherence-measures/) automated it. In ranking, [Learning to Rank using Gradient Descent](https://md.research.elamur.ai/free/en/papers/burges-2005-learning-rank-using-gradient/) (RankNet) brought a pairwise probabilistic loss to commercial search, and [Reciprocal Rank Fusion](https://md.research.elamur.ai/free/en/papers/cormack-2009-reciprocal-rank-fusion-outperforms/) showed that a very simple formula merges ranked lists better than elaborate methods. Benchmarks such as [RCV1](https://md.research.elamur.ai/free/en/papers/lewis-2004-rcv1-benchmark-collection-text-categorization/) supported reproducible text classification.

### Neural text, GPUs and tooling (2015–2019)

Deep learning moved the shelf towards GPUs. [Character-level Convolutional Networks for Text Classification](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/) treated text as a raw signal; [Deep Speech 2](https://md.research.elamur.ai/free/en/papers/amodei-2015-deep-speech-2-end-end/) showed what a high-performance training stack could do for speech recognition; [Billion-scale similarity search with GPUs](https://md.research.elamur.ai/free/en/papers/johnson-2017-billion-scale-similarity-search-gpus/) produced the Faiss library. Compilers appeared for tensor code, with [TVM](https://md.research.elamur.ai/free/en/papers/chen-2018-tvm-automated-end-end-optimizing/) and [Triton](https://md.research.elamur.ai/free/en/papers/tillet-2019-triton-intermediate-language-compiler-tiled/), and model parallelism with [Mesh-TensorFlow](https://md.research.elamur.ai/free/en/papers/shazeer-2018-mesh-tensorflow-deep-learning-supercomputers/). Widely used tools from this time include [SentencePiece](https://md.research.elamur.ai/free/en/papers/kudo-2018-sentencepiece-simple-language-independent-subword/), [UMAP](https://md.research.elamur.ai/free/en/papers/mcinnes-2018-umap-uniform-manifold-approximation-projection/), [hdbscan](https://md.research.elamur.ai/free/en/papers/mcinnes-2017-hdbscan-hierarchical-density-clustering/) and [Sentence-BERT](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/).

Papers on this shelf by year 9,079 dated papers from 1813 to 2026 (6 before 1966 are not drawn); the busiest year is 2026 with 949, and 66% of them appeared in 2018 or later.

**The numbers**

| Year | Papers |
| --- | --- |
| 2026 | 949 |
| 2025 | 369 |
| 2024 | 673 |
| 2023 | 890 |
| 2022 | 575 |
| 2021 | 668 |
| 2020 | 766 |
| 2019 | 598 |
| 2018 | 547 |
| 2017 | 384 |
| 2016 | 380 |
| 2015 | 232 |
| 2014 | 240 |
| 2013 | 166 |
| 2012 | 183 |
| 2011 | 135 |
| 2010 | 185 |
| 2009 | 111 |
| 2008 | 115 |
| 2007 | 94 |
| 2006 | 109 |
| 2005 | 79 |
| 2004 | 109 |
| 2003 | 69 |
| 2002 | 65 |
| 2001 | 28 |
| 2000 | 42 |
| 1999 | 41 |
| 1998 | 39 |
| 1997 | 33 |
| 1996 | 22 |
| 1995 | 25 |
| 1994 | 37 |
| 1993 | 19 |
| 1992 | 13 |
| 1991 | 16 |
| 1990 | 16 |
| 1989 | 6 |
| 1988 | 13 |
| 1987 | 8 |
| 1986 | 7 |
| 1985 | 2 |
| 1983 | 2 |
| 1982 | 3 |
| 1981 | 3 |
| 1978 | 1 |
| 1976 | 3 |
| 1975 | 1 |
| 1973 | 1 |
| 1969 | 1 |
| 1961 | 1 |
| 1923 | 1 |
| 1884 | 1 |
| 1861 | 1 |
| 1816 | 1 |
| 1813 | 1 |

### Dense retrieval, code models and LLM systems (2020–2025)

Most of the shelf's papers are recent. [Dense Passage Retrieval for Open-Domain Question Answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/) and [ColBERT](https://md.research.elamur.ai/free/en/papers/khattab-2020-colbert-efficient-effective-passage/) made learned retrieval practical; [CodeBERT](https://md.research.elamur.ai/free/en/papers/feng-2020-codebert-pre-trained-model-programming/) and [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/) opened the era of code models; and with large language models in production, systems work turned to serving, as in [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). By 2024–2025 the emphasis had shifted to agents that edit real repositories ([SWE-agent](https://md.research.elamur.ai/free/en/papers/yang-2024-swe-agent-agent-computer-interfaces/)), contamination-free code evaluation, and systems that discover algorithms ([AlphaEvolve](https://md.research.elamur.ai/free/en/papers/novikov-2025-alphaevolve-coding-agent-scientific-algorithmic/)).

## Main problems and tasks

- **Ranking.** Given a query, order a very large collection so that the most useful documents come first. Signals range from term statistics (BM25) and link structure (PageRank) to learned scoring functions and neural encoders.
- **First-stage retrieval at scale.** Finding candidates among millions or billions of items fast enough for interactive use, via inverted indexes, approximate nearest-neighbour search, or learned sparse representations such as [SPLADE](https://md.research.elamur.ai/free/en/papers/formal-2021-splade-sparse-lexical-expansion/).
- **Text classification and clustering.** Assigning documents to categories, with or without labels, and grouping them without supervision; evaluation measures such as [V-Measure](https://md.research.elamur.ai/free/en/papers/rosenberg-2007-vmeasure-conditional-entropy-cluster-evaluation/) belong here.
- **Topic discovery.** Finding interpretable themes in a corpus and tracking how they change.
- **Code understanding and generation.** Searching code with natural language, writing functions from docstrings, translating between programming languages, generating SQL from questions, and resolving real software issues.
- **Efficient training.** Fitting models that exceed one GPU's memory through data, tensor, pipeline and sequence parallelism, optimizer-state offloading and activation recomputation.
- **Efficient inference.** Reducing the memory, latency and energy cost of serving language models through batching, caching, quantisation, sparsity and faster decoding.
- **Language identification.** Telling which language, or which closely related variety, a text is in, a prerequisite for building multilingual web corpora.

## Methods and schools

### Probabilistic and link-based ranking

The first school treats retrieval as estimation. BM25 derives term weights from a probabilistic relevance model; the language-modelling approach scores documents by query likelihood; PageRank and HITS compute principal eigenvectors of link matrices. These methods are unsupervised, cheap and hard to beat on unfamiliar data, which explains BM25's survival as a baseline in nearly every neural retrieval paper here.

### Learning to rank

From the mid-2000s, ranking functions were trained on relevance judgements. RankNet's pairwise logistic loss became the basis of a family of later methods. [Reciprocal Rank Fusion](https://md.research.elamur.ai/free/en/papers/cormack-2009-reciprocal-rank-fusion-outperforms/) is the counterpoint: an untrained combination of ranked lists that often beats trained ones, and that is again widely used in hybrid search.

### Dense, late-interaction and learned sparse retrieval

The neural era brought three designs. Bi-encoders such as DPR and Sentence-BERT compress each text into one vector and rely on fast vector search such as Faiss. Late-interaction models such as ColBERT and [ColBERTv2](https://md.research.elamur.ai/free/en/papers/santhanam-2022-colbertv2-effective-efficient-retrieval-lightweight/) keep a vector per token and compare them at query time. Learned sparse models such as SPLADE, [Document Expansion by Query Prediction](https://md.research.elamur.ai/free/en/papers/nogueira-2019-document-expansion-query-prediction/) and [Query2doc](https://md.research.elamur.ai/free/en/papers/wang-2023-query2doc-query-expansion-large-language/) keep the inverted index but let a neural model choose the terms. The debate between these designs is largely one about the trade-off between accuracy, index size and latency, and about how well each generalises beyond its training data, which [BEIR](https://md.research.elamur.ai/free/en/papers/thakur-2021-beir-heterogenous-benchmark-zero/) was built to measure.

### Probabilistic topic models and their successors

pLSA and LDA model each document as a mixture of word distributions and are fitted by EM, variational inference or sampling. Later work added time, correlation and embeddings: [Topic Modeling in Embedding Spaces](https://md.research.elamur.ai/free/en/papers/dieng-2020-topic-modeling-embedding-spaces/) places words and topics in one vector space, and [BERTopic](https://md.research.elamur.ai/free/en/papers/grootendorst-2022-bertopic-neural-topic-modeling-class/) abandons the generative model altogether, clustering sentence embeddings with UMAP and HDBSCAN and labelling clusters with a class-based TF-IDF.

### Systems for large models

Training at scale relies on partitioning. [ZeRO-Offload](https://md.research.elamur.ai/free/en/papers/ren-2021-zero-offload-democratizing-billion-scale/) moves optimizer state to CPU memory; [Reducing Activation Recomputation in Large Transformer Models](https://md.research.elamur.ai/free/en/papers/korthikanti-2022-reducing-activation-recomputation-large-transformer/) and [DeepSpeed Ulysses](https://md.research.elamur.ai/free/en/papers/jacobs-2023-deepspeed-ulysses-system-optimizations-enabling/) split activations and sequences across devices; [Ring Attention](https://md.research.elamur.ai/free/en/papers/liu-2023-ring-attention-blockwise-transformers-near/) passes blocks of keys and values around a ring of GPUs to handle very long contexts; [DeepSpeed-MoE](https://md.research.elamur.ai/free/en/papers/rajbhandari-2022-deepspeed-moe-advancing-mixture-experts/) addresses mixture-of-experts models; [PyTorch Distributed](https://md.research.elamur.ai/free/en/papers/li-2020-pytorch-distributed-experiences-accelerating-data/) documents the data-parallel machinery most practitioners use.

For inference, vLLM's PagedAttention borrows virtual-memory paging from operating systems to remove waste in the key–value cache, and [SGLang](https://md.research.elamur.ai/free/en/papers/zheng-2023-sglang-efficient-execution-structured-language/) reuses cached prefixes across structured programs of calls. [FlexGen](https://md.research.elamur.ai/free/en/papers/sheng-2023-flexgen-high-throughput-generative-inference/) and [LLM in a flash](https://md.research.elamur.ai/free/en/papers/alizadeh-2024-llm-flash-efficient-large-language/) run large models with limited GPU or device memory. Quantisation methods compress weights, activations or the cache to a few bits: [ZeroQuant](https://md.research.elamur.ai/free/en/papers/yao-2022-zeroquant-efficient-affordable-post-training/), [I-BERT](https://md.research.elamur.ai/free/en/papers/kim-2021-i-bert-integer-only-bert/), [SqueezeLLM](https://md.research.elamur.ai/free/en/papers/kim-2023-squeezellm-dense-sparse-quantization/), [QuaRot](https://md.research.elamur.ai/free/en/papers/ashkboos-2024-quarot-outlier-free-4-bit/), [QuIP#](https://md.research.elamur.ai/free/en/papers/tseng-2024-quip-even-better-llm-quantization/), [KVQuant](https://md.research.elamur.ai/free/en/papers/hooper-2024-kvquant-towards-10-million-context/), [KIVI](https://md.research.elamur.ai/free/en/papers/liu-2024-kivi-tuning-free-asymmetric-2bit/) and [8-bit Optimizers via Block-wise Quantization](https://md.research.elamur.ai/free/en/papers/dettmers-2021-8-bit-optimizers-block-wise/). Decoding is accelerated by drafting several tokens at once, as in [Medusa](https://md.research.elamur.ai/free/en/papers/cai-2024-medusa-simple-llm-inference-acceleration/), [EAGLE](https://md.research.elamur.ai/free/en/papers/li-2024-eagle-speculative-sampling-requires-rethinking/) and [Lookahead Decoding](https://md.research.elamur.ai/free/en/papers/fu-2024-break-sequential-dependency-llm-inference/), while [DuoAttention](https://md.research.elamur.ai/free/en/papers/xiao-2024-duoattention-efficient-long-context-llm/) and [Native Sparse Attention](https://md.research.elamur.ai/free/en/papers/yuan-2025-native-sparse-attention-hardware/) cut the cost of long contexts. [From Words to Watts](https://md.research.elamur.ai/free/en/papers/samsi-2023-words-watts-benchmarking-energy-costs/) measures the energy these systems consume.

### Compilers for tensor programs

TVM searches over schedules to generate fast kernels for many hardware back ends; Triton gives programmers a tile-based language that compiles to efficient GPU code without hand-written CUDA. [KernelBench](https://md.research.elamur.ai/free/en/papers/ouyang-2025-kernelbench-llms-write-efficient-gpu/) closes the loop by asking whether language models can write such kernels themselves.

### Learning from code

Code models evolved from syntax-aware generators such as [A Syntactic Neural Model for General-Purpose Code Generation](https://md.research.elamur.ai/free/en/papers/yin-2017-syntactic-neural-model-general-purpose/), through encoder pretraining (CodeBERT, [GraphCodeBERT](https://md.research.elamur.ai/free/en/papers/guo-2021-graphcodebert-pre-training-code-representations/), [UniXcoder](https://md.research.elamur.ai/free/en/papers/guo-2022-unixcoder-unified-cross-modal-pre/), [PLBART](https://md.research.elamur.ai/free/en/papers/ahmad-2021-unified-pre-training-program-understanding/)), to large generative models ([Codex](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/), [AlphaCode](https://md.research.elamur.ai/free/en/papers/li-2022-competition-level-code-generation-alphacode/), [Code Llama](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/)). Training tricks such as [fill-in-the-middle](https://md.research.elamur.ai/free/en/papers/bavarian-2022-efficient-training-language-models-fill/) and reinforcement from unit tests ([CodeRL](https://md.research.elamur.ai/free/en/papers/le-2022-coderl-mastering-code-generation-through/)) and agent loops ([Reflexion](https://md.research.elamur.ai/free/en/papers/shinn-2023-reflexion-language-agents-verbal-reinforcement/), SWE-agent) followed. [Unsupervised Translation of Programming Languages](https://md.research.elamur.ai/free/en/papers/lachaux-2020-unsupervised-translation-programming-languages/) carried back-translation from machine translation to code.

## Datasets, resources and benchmarks

| Resource | Purpose | Paper here |
| --- | --- | --- |
| RCV1 | Large documented news collection for text categorisation | [RCV1](https://md.research.elamur.ai/free/en/papers/lewis-2004-rcv1-benchmark-collection-text-categorization/) |
| MS MARCO | Real search queries with passages; the main training set for neural rankers | [MS MARCO](https://md.research.elamur.ai/free/en/papers/bajaj-2016-ms-marco-human-generated/) |
| BEIR | Zero-shot retrieval across heterogeneous tasks | [BEIR](https://md.research.elamur.ai/free/en/papers/thakur-2021-beir-heterogenous-benchmark-zero/) |
| MTEB | Text embeddings across many task types | [MTEB](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-mteb-massive-text-embedding-benchmark/) |
| MIRACL | Multilingual retrieval in 18 languages | [MIRACL](https://md.research.elamur.ai/free/en/papers/zhang-2023-miracl-multilingual-retrieval-dataset-covering/) |
| HumanEval | Python functions graded by unit tests | [Codex paper](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/) |
| MBPP | Short Python programming problems | [Program Synthesis with Large Language Models](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/) |
| APPS | Coding-challenge problems | [APPS](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-coding-challenge-competence-apps/) |
| EvalPlus | Stronger tests for HumanEval and MBPP | [Is Your Code Generated by ChatGPT Really Correct?](https://md.research.elamur.ai/free/en/papers/liu-2023-your-code-generated-chatgpt-really/) |
| CodeSearchNet | Natural-language code search | [CodeSearchNet](https://md.research.elamur.ai/free/en/papers/husain-2020-codesearchnet-challenge-evaluating-state-semantic/) |
| CoNaLa | Aligned Stack Overflow code and intents | [Learning to Mine Aligned Code…](https://md.research.elamur.ai/free/en/papers/yin-2018-learning-mine-aligned-code-natural/) |
| Spider | Cross-domain text-to-SQL | [Spider](https://md.research.elamur.ai/free/en/papers/yu-2018-spider-large-scale-human-labeled/) |
| SWE-bench | Resolving real GitHub issues | [SWE-bench](https://md.research.elamur.ai/free/en/papers/jimenez-2024-swe-bench-can-language-models/) |
| LiveCodeBench, CRUXEval, xCodeEval | Fresh, execution-based and multilingual code evaluation | [LiveCodeBench](https://md.research.elamur.ai/free/en/papers/jain-2024-livecodebench-holistic-contamination-free-evaluation/), [CRUXEval](https://md.research.elamur.ai/free/en/papers/gu-2024-cruxeval-benchmark-code-reasoning-understanding/), [xCodeEval](https://md.research.elamur.ai/free/en/papers/khan-2024-xcodeeval-execution-based-large-scale/) |
| DSL / VarDial shared tasks | Discriminating similar languages and dialects | [DSL 2014](https://md.research.elamur.ai/free/en/papers/zampieri-2014-report-dsl-shared-task/), [VarDial 2017](https://md.research.elamur.ai/free/en/papers/zampieri-2017-findings-vardial-evaluation-campaign/) |

Key software described by papers here includes Faiss, SentencePiece, sentence-transformers, vLLM, TVM, Triton, [langid.py](https://md.research.elamur.ai/free/en/papers/lui-2012-langid-py-off-shelf-language/), UMAP, hdbscan and the [TAGME](https://md.research.elamur.ai/free/en/papers/ferragina-2010-tagme-fly-annotation-short/) entity annotator. Web-scale corpora such as [The FineWeb Datasets](https://md.research.elamur.ai/free/en/papers/penedo-2024-fineweb-datasets-decanting-web-finest/) and [DataComp-LM](https://md.research.elamur.ai/free/en/papers/li-2024-datacomp-lm-search-next-generation/) link data engineering to pretraining.

## Landmark papers in this library

Citations of the papers on this shelf, by when they were published Papers published in 2015–2019 gather 37% of the 3,390,907 citations of the papers on this shelf.

**The numbers**

| Published | Papers and their citations |
| --- | --- |
| 2025–2029 | 1,318 papers, 26,974 citations |
| 2020–2024 | 3,572 papers, 1,004,519 citations |
| 2015–2019 | 2,141 papers, 1,257,489 citations |
| 2010–2014 | 909 papers, 480,878 citations |
| 2005–2009 | 508 papers, 179,498 citations |
| 2000–2004 | 313 papers, 228,245 citations |
| 1995–1999 | 160 papers, 133,036 citations |
| 1990–1994 | 101 papers, 70,263 citations |
| 1985–1989 | 36 papers, 7,232 citations |
| 1980–1984 | 8 papers, 502 citations |
| 1975–1979 | 5 papers, 745 citations |
| 1970–1974 | 1 papers, 1,486 citations |
| 1965–1969 | 1 papers, 0 citations |
| 1960–1964 | 1 papers, 4 citations |
| 1920–1924 | 1 papers, 0 citations |
| 1880–1884 | 1 papers, 0 citations |
| 1860–1864 | 1 papers, 0 citations |
| 1815–1819 | 1 papers, 0 citations |
| 1810–1814 | 1 papers, 0 citations |

**[Latent Dirichlet Allocation](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/).** Blei, Ng and Jordan gave each document a Dirichlet-distributed mixture of topics and fitted the model with variational inference, fixing pLSA's lack of a document-level generative story. LDA became one of the most widely used models in all of text analysis and spawned dynamic, correlated and supervised variants.

**[The Anatomy of a Large-Scale Hypertextual Web Search Engine](https://md.research.elamur.ai/free/en/papers/brin-1998-anatomy-large-scale-hypertextual-web/).** A rare full description of a search engine's crawler, indexer and ranker, built around link analysis and anchor text. It describes the prototype that grew into Google.

**[The PageRank Citation Ranking: Bringing Order to the Web](https://md.research.elamur.ai/free/en/papers/page-1999-pagerank-citation-ranking-bringing/).** Defines a page's importance recursively by the importance of pages that link to it, computed by power iteration with a random-jump term. Personalised variants of PageRank are now used far beyond the web, in graph-based ranking of all kinds.

**[Authoritative Sources in a Hyperlinked Environment](https://md.research.elamur.ai/free/en/papers/kleinberg-1999-authoritative-sources-hyperlinked-environment/).** Kleinberg's HITS algorithm separates hubs, which point to good pages, from authorities, which good hubs point to, and obtains both as eigenvectors of matrices built from a query-focused subgraph. It is a foundation of web-graph analysis.

**[Text Categorization with Support Vector Machines](https://md.research.elamur.ai/free/en/papers/joachims-1998-text-categorization-support-vector-machines/).** Joachims argued from the properties of text, with its many relevant but sparse features, that large-margin classifiers fit the task, and showed them beating the standard alternatives on Reuters and Ohsumed. Linear SVMs remained the default text classifier until neural methods took over.

**[UMAP](https://md.research.elamur.ai/free/en/papers/mcinnes-2018-umap-uniform-manifold-approximation-projection/).** A manifold-learning method grounded in topology that produces layouts comparable to t-SNE while running faster and preserving more global structure. It is now a standard step for visualising embeddings, including in topic-modelling pipelines.

**[Dense Passage Retrieval for Open-Domain Question Answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/).** Two BERT encoders trained with in-batch negatives outperformed BM25 on several question-answering datasets without special pretraining. DPR made dense retrieval the usual first stage of retrieval-augmented generation.

**[Sentence-BERT](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/).** Fine-tuning BERT in a siamese network yields sentence vectors comparable by cosine similarity, reducing large similarity searches from hours to seconds. The accompanying library became a standard toolkit for embeddings.

**[Billion-scale similarity search with GPUs](https://md.research.elamur.ai/free/en/papers/johnson-2017-billion-scale-similarity-search-gpus/).** A fast GPU k-selection routine and product-quantised indexes made nearest-neighbour search over a billion vectors feasible on a single machine. The resulting Faiss library underlies much of today's vector search.

**[SentencePiece](https://md.research.elamur.ai/free/en/papers/kudo-2018-sentencepiece-simple-language-independent-subword/).** A language-independent subword tokeniser that learns directly from raw text and treats whitespace as an ordinary symbol, making tokenisation reversible. Many multilingual and large language models use it; it connects the shelf to [Tokenization & segmentation](https://md.research.elamur.ai/free/en/topics/tokenization/).

**[Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/).** Introduced Codex and the HumanEval benchmark, and made functional correctness, measured by running unit tests with the pass@k metric, the standard for code generation.

**[Efficient Memory Management for Large Language Model Serving with PagedAttention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/).** By storing the key–value cache in non-contiguous fixed-size blocks, vLLM nearly eliminates memory fragmentation and allows much larger batches. It became one of the most widely deployed open serving engines.

**[CodeBERT](https://md.research.elamur.ai/free/en/papers/feng-2020-codebert-pre-trained-model-programming/).** An early bimodal model pretrained on natural language and code from several programming languages, strong on code search and documentation generation. It led to a series of code encoders from the same group.

**[Learning to Rank using Gradient Descent](https://md.research.elamur.ai/free/en/papers/burges-2005-learning-rank-using-gradient/).** RankNet models the probability that one document should outrank another with a logistic function of their score difference and trains neural networks on that loss, an approach that underpinned later commercial ranking systems.

**[SWE-bench](https://md.research.elamur.ai/free/en/papers/jimenez-2024-swe-bench-can-language-models/).** Collected real issues and pull requests from popular Python repositories and asked models to produce patches that pass the project's tests. It moved code evaluation from isolated functions to whole codebases and became the main yardstick for coding agents.

## People

Authors with the most papers on this shelf Nizar Habash leads with 88 papers; the 12 authors shown wrote 743 of the papers on this shelf between them, counting each paper once per author.

**The numbers**

| Author | Papers here · their citations |
| --- | --- |
| [Nizar Habash](https://md.research.elamur.ai/free/en/authors/nizar-habash/) | 88 papers · 8,041 citations |
| [Christopher Manning](https://md.research.elamur.ai/free/en/authors/christopher-d-manning/) | 82 papers · 71,147 citations |
| [Luke Zettlemoyer](https://md.research.elamur.ai/free/en/authors/luke-zettlemoyer/) | 76 papers · 60,645 citations |
| [Noah A. Smith](https://md.research.elamur.ai/free/en/authors/noah-a-smith/) | 70 papers · 20,919 citations |
| [Graham Neubig](https://md.research.elamur.ai/free/en/authors/graham-neubig/) | 66 papers · 22,400 citations |
| [Yejin Choi](https://md.research.elamur.ai/free/en/authors/yejin-choi/) | 63 papers · 22,543 citations |
| [Iryna Gurevych](https://md.research.elamur.ai/free/en/authors/iryna-gurevych/) | 59 papers · 31,789 citations |
| [Yoav Goldberg](https://md.research.elamur.ai/free/en/authors/yoav-goldberg/) | 52 papers · 15,685 citations |
| [Massimo Poesio](https://md.research.elamur.ai/free/en/authors/massimo-poesio/) | 51 papers · 5,846 citations |
| [Hannaneh Hajishirzi](https://md.research.elamur.ai/free/en/authors/hannaneh-hajishirzi/) | 46 papers · 16,270 citations |
| [Dan Roth](https://md.research.elamur.ai/free/en/authors/dan-roth/) | 46 papers · 12,890 citations |
| [Philipp Koehn](https://md.research.elamur.ai/free/en/authors/philipp-koehn/) | 44 papers · 19,115 citations |

- **[Marcos Zampieri](https://md.research.elamur.ai/free/en/authors/marcos-zampieri/)** (George Mason University) is the most prolific author on the shelf, through the shared tasks on discriminating similar languages and dialects and the [survey of automatic language identification](https://md.research.elamur.ai/free/en/papers/jauhiainen-2019-automatic-language-identification-texts/). His co-organisers include **[Jörg Tiedemann](https://md.research.elamur.ai/free/en/authors/jorg-tiedemann/)** (University of Helsinki), **[Nikola Ljubešić](https://md.research.elamur.ai/free/en/authors/nikola-ljubesic/)** (Jožef Stefan Institute and University of Ljubljana) and **[Preslav Nakov](https://md.research.elamur.ai/free/en/authors/preslav-nakov/)** (Mohamed bin Zayed University of Artificial Intelligence).
- **[David M. Blei](https://md.research.elamur.ai/free/en/authors/david-m-blei/)** (Columbia University) is behind the shelf's most cited line of work: LDA, dynamic topic models, the human study of topic interpretability and embedded topic models.
- **[Lawrence Page](https://md.research.elamur.ai/free/en/authors/lawrence-page/)** and **[Sergey Brin](https://md.research.elamur.ai/free/en/authors/sergey-brin/)** wrote the two foundational papers on PageRank and the Google prototype at Stanford.
- **[Thorsten Joachims](https://md.research.elamur.ai/free/en/authors/thorsten-joachims/)** established SVMs for text classification and later shaped learning to rank from user behaviour.
- **[Thomas Hofmann](https://md.research.elamur.ai/free/en/authors/thomas-hofmann/)** introduced probabilistic latent semantic analysis, the direct precursor of LDA.
- **[Timothy Baldwin](https://md.research.elamur.ai/free/en/authors/timothy-baldwin/)** (University of Melbourne and MBZUAI) contributed both to topic-coherence evaluation and to language identification with langid.py, alongside **[Jey Han Lau](https://md.research.elamur.ai/free/en/authors/jey-han-lau/)** and **[David Newman](https://md.research.elamur.ai/free/en/authors/david-newman/)**.
- **[Lianmin Zheng](https://md.research.elamur.ai/free/en/authors/lianmin-zheng/)**, **[Ying Sheng](https://md.research.elamur.ai/free/en/authors/ying-sheng/)**, **[Joseph E. Gonzalez](https://md.research.elamur.ai/free/en/authors/joseph-e-gonzalez/)** and **[Ion Stoica](https://md.research.elamur.ai/free/en/authors/ion-stoica/)** (University of California, Berkeley) built vLLM, SGLang and FlexGen; Zheng also co-authored TVM with **[Tianqi Chen](https://md.research.elamur.ai/free/en/authors/tianqi-chen/)**.
- **[Minjia Zhang](https://md.research.elamur.ai/free/en/authors/minjia-zhang/)**, **[Yuxiong He](https://md.research.elamur.ai/free/en/authors/yuxiong-he/)**, **[Samyam Rajbhandari](https://md.research.elamur.ai/free/en/authors/samyam-rajbhandari/)**, **[Zhewei Yao](https://md.research.elamur.ai/free/en/authors/zhewei-yao/)** and **[Reza Yazdani Aminabadi](https://md.research.elamur.ai/free/en/authors/reza-yazdani-aminabadi/)** form the DeepSpeed group behind ZeRO-Offload, ZeroQuant, DeepSpeed-MoE and Ulysses.
- **[Kurt Keutzer](https://md.research.elamur.ai/free/en/authors/kurt-keutzer/)**, **[Michael W. Mahoney](https://md.research.elamur.ai/free/en/authors/michael-w-mahoney/)**, **[Amir Gholami](https://md.research.elamur.ai/free/en/authors/amir-gholami/)** and **[Sehoon Kim](https://md.research.elamur.ai/free/en/authors/sehoon-kim/)** (Berkeley) developed the integer-only and low-bit quantisation papers I-BERT, SqueezeLLM and KVQuant.
- **[Nils Reimers](https://md.research.elamur.ai/free/en/authors/nils-reimers/)** (now at Cohere) created Sentence-BERT with **[Iryna Gurevych](https://md.research.elamur.ai/free/en/authors/iryna-gurevych/)** and co-authored BEIR and MTEB, defining how text embeddings are built and evaluated.
- **[Matei Zaharia](https://md.research.elamur.ai/free/en/authors/matei-zaharia/)** co-authored ColBERT and ColBERTv2 with **[Omar Khattab](https://md.research.elamur.ai/free/en/authors/omar-khattab/)**, as well as Ring Attention.
- **[Daya Guo](https://md.research.elamur.ai/free/en/authors/daya-guo/)** and **[Nan Duan](https://md.research.elamur.ai/free/en/authors/nan-duan/)** (Microsoft Research Asia) led the CodeBERT, GraphCodeBERT and UniXcoder series.
- **[Karthik Narasimhan](https://md.research.elamur.ai/free/en/authors/karthik-narasimhan/)** (Princeton University) and **[Shunyu Yao](https://md.research.elamur.ai/free/en/authors/shunyu-yao/)** co-authored Reflexion, SWE-bench and SWE-agent, central works on language agents for software.
- **[Baptiste Rozière](https://md.research.elamur.ai/free/en/authors/baptiste-roziere/)** worked on unsupervised translation between programming languages, Code Llama and CRUXEval.
- **[Kevin Knight](https://md.research.elamur.ai/free/en/authors/kevin-knight/)** and **[Jason M. Eisner](https://md.research.elamur.ai/free/en/authors/jason-eisner/)** represent the formal side: syntax-based translation models, tree transducers and efficient parsing algorithms.
- **[Francisco J. R. Ruiz](https://md.research.elamur.ai/free/en/authors/francisco-j-r-ruiz/)** (Google DeepMind) links embedded topic models to program search for mathematical discovery in [FunSearch](https://md.research.elamur.ai/free/en/papers/paredes-2023-mathematical-discoveries-program-search-large/) and AlphaEvolve.

## Institutions and places

Where the authors on this shelf work now Carnegie Mellon University is the current home of authors of 583 papers on this shelf; counted are the authors with a page here, by their latest institution.

**The numbers**

| Institution | Papers |
| --- | --- |
| [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) | 583 papers |
| [Google](https://md.research.elamur.ai/free/en/companies/google/) | 563 papers |
| [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) | 543 papers |
| [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) | 420 papers |
| [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) | 396 papers |
| [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) | 350 papers |
| [Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/government/chinese-academy-of-sciences/) | 302 papers |
| [New York University](https://md.research.elamur.ai/free/en/universities/new-york-university/) | 300 papers |
| [Google DeepMind](https://md.research.elamur.ai/free/en/companies/google-deepmind/) | 279 papers |
| [University of Edinburgh](https://md.research.elamur.ai/free/en/universities/university-of-edinburgh/) | 269 papers |

[Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/), the birthplace of PageRank, is also the most frequent current home of authors on the shelf, together with [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/). The [University of California, Berkeley](https://md.research.elamur.ai/free/en/universities/university-of-california-berkeley/) anchors the serving and quantisation work, and [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) the DeepSpeed and code-model work. Other strong centres include [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/), the [University of Illinois Urbana-Champaign](https://md.research.elamur.ai/free/en/universities/university-of-illinois-urbana-champaign/), [Columbia University](https://md.research.elamur.ai/free/en/universities/columbia-university/), [Princeton University](https://md.research.elamur.ai/free/en/universities/princeton-university/), the [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/), [Cornell University](https://md.research.elamur.ai/free/en/universities/cornell-university/), the [University of California, Santa Cruz](https://md.research.elamur.ai/free/en/universities/university-of-california-santa-cruz/), [Amazon](https://md.research.elamur.ai/free/en/companies/amazon/), [Google](https://md.research.elamur.ai/free/en/companies/google/) and the [Allen Institute for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/allen-institute-for-artificial-intelligence/).

Outside the United States, Chinese institutions such as [Peking University](https://md.research.elamur.ai/free/en/universities/peking-university/), the [Harbin Institute of Technology](https://md.research.elamur.ai/free/en/universities/harbin-institute-of-technology/), the [Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/government/chinese-academy-of-sciences/) and the [University of Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/universities/university-of-chinese-academy-of-sciences/) are prominent. [Mohamed bin Zayed University of Artificial Intelligence](https://md.research.elamur.ai/free/en/universities/mohamed-bin-zayed-university-of-artificial-intelligence/) in the United Arab Emirates and the [University of Helsinki](https://md.research.elamur.ai/free/en/universities/university-of-helsinki/) in Finland stand out through the language-identification work.

Countries of the authors on this shelf United States comes first: authors working there now wrote 4,440 papers on this shelf.

**The numbers**

| Country | Papers |
| --- | --- |
| [United States](https://md.research.elamur.ai/free/en/countries/united-states/) | 4,440 papers |
| [China](https://md.research.elamur.ai/free/en/countries/china/) | 1,992 papers |
| [United Kingdom](https://md.research.elamur.ai/free/en/countries/united-kingdom/) | 1,324 papers |
| [Germany](https://md.research.elamur.ai/free/en/countries/germany/) | 792 papers |
| [Canada](https://md.research.elamur.ai/free/en/countries/canada/) | 467 papers |
| Hong Kong | 433 papers |
| [Israel](https://md.research.elamur.ai/free/en/countries/israel/) | 382 papers |
| [United Arab Emirates](https://md.research.elamur.ai/free/en/countries/united-arab-emirates/) | 360 papers |
| [France](https://md.research.elamur.ai/free/en/countries/france/) | 352 papers |
| [Singapore](https://md.research.elamur.ai/free/en/countries/singapore/) | 337 papers |

Venues are unusually diverse. Retrieval papers appear at SIGIR and CIKM, machine-learning classics at ICML, COLT and ECML and in JMLR, web papers at WWW and in the Journal of the ACM, systems papers at SOSP and MAPL, and software in the Journal of Open Source Software. The language-oriented work appears at EMNLP, ACL, NAACL and COLING. Many of the newest and most cited papers circulated first as preprints.

Where the papers on this shelf were published Annual Meeting of the Association for Computational Linguistics (ACL) carries the most, 1321 of 9085.

**The numbers**

| Venue | Papers |
| --- | --- |
| [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/) | 1,321 papers |
| [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/) | 937 papers |
| [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/) | 830 papers |
| [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/) | 439 papers |
| [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/) | 337 papers |
| [International Conference on Computational Linguistics (COLING)](https://md.research.elamur.ai/free/en/venues/coling/) | 278 papers |
| [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/) | 208 papers |
| [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/) | 182 papers |
| [Conference of the European Chapter of the Association for Computational Linguistics (EACL)](https://md.research.elamur.ai/free/en/venues/eacl/) | 157 papers |
| [Conference on Computational Natural Language Learning (CoNLL)](https://md.research.elamur.ai/free/en/venues/conll/) | 134 papers |

## Open questions

- **Which retrieval architecture wins?** Dense, late-interaction, learned sparse and classical lexical methods each lead on some benchmarks; hybrid systems fused with simple methods such as reciprocal rank fusion often do best. How well learned retrievers transfer to new domains and languages, as tested by BEIR and MIRACL, remains unsettled.
- **Evaluating code honestly.** HumanEval-style tests can be too weak ([EvalPlus](https://md.research.elamur.ai/free/en/papers/liu-2023-your-code-generated-chatgpt-really/) found many accepted solutions wrong), and public problems leak into training data, which LiveCodeBench addresses by using only problems published after a model's cutoff. Repository-level tasks such as SWE-bench are more realistic but costlier to run.
- **The cost of inference.** Serving rather than training now dominates the bill for many models. How far quantisation can go before quality drops, how to handle contexts of millions of tokens, and how to measure energy use fairly are all active questions.
- **Hardware-aware model design.** Work such as Native Sparse Attention and KernelBench asks whether architectures should be designed together with the kernels that run them, and whether models can write those kernels.
- **Topic models in the embedding era.** Whether neural clustering approaches such as BERTopic replace probabilistic models or complement them, and how to evaluate topics reliably, are still debated.
- **Language identification for thousands of languages.** As [Language ID in the Wild](https://md.research.elamur.ai/free/en/papers/caswell-2020-language-id-wild-unexpected/), [GlotLID](https://md.research.elamur.ai/free/en/papers/kargaran-2023-glotlid-language-identification-low-resource/) and [An Open Dataset and Model for Language Identification](https://md.research.elamur.ai/free/en/papers/burchell-2023-open-dataset-model-language-identification/) show, identifiers that work well on clean benchmarks fail on noisy web text for [low-resource languages](https://md.research.elamur.ai/free/en/topics/low-resource-languages/), which affects every multilingual corpus built from the web.
- **Safety filters as infrastructure.** Classifiers such as [Llama Guard](https://md.research.elamur.ai/free/en/papers/inan-2023-llama-guard-llm-based-input/), [WildGuard](https://md.research.elamur.ai/free/en/papers/han-2024-wildguard-open-one-stop-moderation/) and [A Holistic Approach to Undesired Content Detection](https://md.research.elamur.ai/free/en/papers/markov-2022-holistic-approach-undesired-content-detection/) now sit around deployed models; how to make them accurate without over-blocking ties the shelf to [Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/).

## Neighbouring topics

Neighbouring shelves: papers shared Arts and Humanities shares the most: 8261 of the 9,085 papers here (91%) stand on both.

**The numbers**

| Shelf | Papers shared |
| --- | --- |
| [Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) | 8261 shared · 91% |
| [Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) | 604 shared · 7% |
| [Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) | 577 shared · 6% |
| [Biochemistry, Genetics and Molecular Biology](https://md.research.elamur.ai/free/en/shelves/biochemistry-genetics-and-molecular-biology/) | 110 shared · 1% |
| [Decision Sciences](https://md.research.elamur.ai/free/en/shelves/decision-sciences/) | 68 shared · 1% |
| [Medicine](https://md.research.elamur.ai/free/en/shelves/medicine/) | 62 shared · 1% |
| [Neuroscience](https://md.research.elamur.ai/free/en/shelves/neuroscience/) | 47 shared · 1% |
| [Engineering](https://md.research.elamur.ai/free/en/shelves/engineering/) | 45 shared · 0% |
| [Mathematics](https://md.research.elamur.ai/free/en/shelves/mathematics/) | 33 shared · 0% |
| [Physics and Astronomy](https://md.research.elamur.ai/free/en/shelves/physics-and-astronomy/) | 18 shared · 0% |

The largest overlap is with [Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/), followed by [Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/), [Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/), [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) and [Mathematics](https://md.research.elamur.ai/free/en/shelves/mathematics/). The machine-translation overlap comes from syntax-based translation models such as [A Syntax-based Statistical Translation Model](https://md.research.elamur.ai/free/en/papers/yamada-2001-syntax-based-statistical-translation-model/), [What's in a translation rule?](https://md.research.elamur.ai/free/en/papers/galley-2004-what-s-translation-rule/), [Training Tree Transducers](https://md.research.elamur.ai/free/en/papers/graehl-2004-training-tree-transducers/) and [Synchronous Grammars as Tree Transducers](https://md.research.elamur.ai/free/en/papers/shieber-2004-synchronous-grammars-as-tree-transducers/), and from engineering studies such as [Training Tips for the Transformer Model](https://md.research.elamur.ai/free/en/papers/popel-2018-training-tips-transformer-model/) and [Deep Encoder, Shallow Decoder](https://md.research.elamur.ai/free/en/papers/kasai-2020-deep-encoder-shallow-decoder-reevaluating/).

Among topics, [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) is the most common, followed by [Code & programming languages](https://md.research.elamur.ai/free/en/topics/code-and-programming/), [Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/), [Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) and [Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/). Smaller links reach [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/), [GPU & parallel computing](https://md.research.elamur.ai/free/en/topics/gpu-parallel-computing/), [Syntax & grammar formalisms](https://md.research.elamur.ai/free/en/topics/syntax-grammar/), [Semantic parsing & meaning representation](https://md.research.elamur.ai/free/en/topics/semantic-parsing/) (through text-to-SQL work such as [RAT-SQL](https://md.research.elamur.ai/free/en/papers/wang-2020-rat-sql-relation-aware-schema/)), [Dependency parsing](https://md.research.elamur.ai/free/en/topics/dependency-parsing/), [Morphology](https://md.research.elamur.ai/free/en/topics/morphology/) (through [CRF-based Japanese morphological analysis](https://md.research.elamur.ai/free/en/papers/kudo-2004-applying-conditional-random-fields-japanese/)), [Part-of-speech tagging](https://md.research.elamur.ai/free/en/topics/part-of-speech-tagging/), [Semantic role labeling](https://md.research.elamur.ai/free/en/topics/semantic-role-labeling/), [Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/), [Question answering](https://md.research.elamur.ai/free/en/topics/question-answering/), [Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) and [Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/).

The topics on this shelf Language models & pretraining: 1313 of the 9,085 papers here.

**The numbers**

| Topic | Papers |
| --- | --- |
| [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) | 1,313 papers |
| [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) | 1,077 papers |
| [English](https://md.research.elamur.ai/free/en/topics/english/) | 1,042 papers |
| [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) | 945 papers |
| [Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) | 671 papers |
| [Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) | 590 papers |
| [Coreference resolution](https://md.research.elamur.ai/free/en/topics/coreference-resolution/) | 535 papers |
| [Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) | 493 papers |
| [Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) | 452 papers |
| [Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) | 388 papers |
| [Discourse analysis](https://md.research.elamur.ai/free/en/topics/discourse-analysis/) | 352 papers |
| [Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) | 261 papers |

## Key works in this library

- [Okapi at TREC-3](https://md.research.elamur.ai/free/en/papers/robertson-1994-okapi-trec-3/) — the system that gave BM25 its standard form.
- [Three New Probabilistic Models for Dependency Parsing: An Exploration](https://md.research.elamur.ai/free/en/papers/eisner-1996-three-new-probabilistic-models-dependency/) — efficient probabilistic dependency parsing.
- [The Anatomy of a Large-Scale Hypertextual Web Search Engine](https://md.research.elamur.ai/free/en/papers/brin-1998-anatomy-large-scale-hypertextual-web/) — the Google prototype.
- [A Language Modeling Approach to Information Retrieval](https://md.research.elamur.ai/free/en/papers/ponte-1998-language-modeling-approach-information/) — ranking by query likelihood.
- [Text Categorization with Support Vector Machines: Learning with Many Relevant Features](https://md.research.elamur.ai/free/en/papers/joachims-1998-text-categorization-support-vector-machines/) — SVMs for text.
- [Combining Labeled and Unlabeled Data with Co-Training](https://md.research.elamur.ai/free/en/papers/blum-1998-combining-labeled-unlabeled-data-co-training/) — two views, few labels.
- [The PageRank Citation Ranking: Bringing Order to the Web](https://md.research.elamur.ai/free/en/papers/page-1999-pagerank-citation-ranking-bringing/) — importance from links.
- [Authoritative Sources in a Hyperlinked Environment](https://md.research.elamur.ai/free/en/papers/kleinberg-1999-authoritative-sources-hyperlinked-environment/) — hubs and authorities.
- [Probabilistic Latent Semantic Analysis](https://md.research.elamur.ai/free/en/papers/hofmann-1999-probabilistic-latent-semantic-analysis/) — LSA as a mixture model.
- [Latent Dirichlet Allocation](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/) — the generative topic model.
- [Learning to Rank using Gradient Descent](https://md.research.elamur.ai/free/en/papers/burges-2005-learning-rank-using-gradient/) — RankNet.
- [Reading Tea Leaves: How Humans Interpret Topic Models](https://md.research.elamur.ai/free/en/papers/chang-2009-reading-tea-leaves-humans-interpret-topic/) — likelihood versus interpretability.
- [Billion-scale similarity search with GPUs](https://md.research.elamur.ai/free/en/papers/johnson-2017-billion-scale-similarity-search-gpus/) — Faiss.
- [UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction](https://md.research.elamur.ai/free/en/papers/mcinnes-2018-umap-uniform-manifold-approximation-projection/) — fast manifold embedding.
- [SentencePiece: A simple and language independent subword tokenizer and detokenizer for Neural Text Processing](https://md.research.elamur.ai/free/en/papers/kudo-2018-sentencepiece-simple-language-independent-subword/) — tokenisation from raw text.
- [TVM: An Automated End-to-End Optimizing Compiler for Deep Learning](https://md.research.elamur.ai/free/en/papers/chen-2018-tvm-automated-end-end-optimizing/) — compiling tensor programs.
- [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/) — sentence vectors for search.
- [Dense Passage Retrieval for Open-Domain Question Answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/) — learned first-stage retrieval.
- [ColBERT: Efficient and Effective Passage Search via Contextualized Late Interaction over BERT](https://md.research.elamur.ai/free/en/papers/khattab-2020-colbert-efficient-effective-passage/) — token-level late interaction.
- [CodeBERT: A Pre-Trained Model for Programming and Natural Languages](https://md.research.elamur.ai/free/en/papers/feng-2020-codebert-pre-trained-model-programming/) — pretraining on code and text.
- [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/) — Codex and HumanEval.
- [Competition-Level Code Generation with AlphaCode](https://md.research.elamur.ai/free/en/papers/li-2022-competition-level-code-generation-alphacode/) — sampling and filtering at scale for contest problems.
- [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/) — vLLM.
- [SWE-bench: Can Language Models Resolve Real-World GitHub Issues?](https://md.research.elamur.ai/free/en/papers/jimenez-2024-swe-bench-can-language-models/) — repository-level evaluation.
- [AlphaEvolve: A coding agent for scientific and algorithmic discovery](https://md.research.elamur.ai/free/en/papers/novikov-2025-alphaevolve-coding-agent-scientific-algorithmic/) — evolving programs with language models.

## Papers (9,085)

**1** [2](https://md.research.elamur.ai/free/en/shelves/computer-science/page/2/) … [10](https://md.research.elamur.ai/free/en/shelves/computer-science/page/10/) … [92](https://md.research.elamur.ai/free/en/shelves/computer-science/page/92/) [next →](https://md.research.elamur.ai/free/en/shelves/computer-science/page/2/)

1. [**Very Deep Convolutional Networks for Large-Scale Image Recognition**Karen Simonyan, Andrew Zisserman*2014 · 72,829 citations*](https://md.research.elamur.ai/free/en/papers/simonyan-2014-very-deep-convolutional-networks-large/)
2. [**Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks**Shaoqing Ren, Kaiming He, Ross Girshick, et al.*2015 · 64,758 citations*](https://md.research.elamur.ai/free/en/papers/ren-2015-faster-r-cnn-towards-real/)
3. [**BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding**Jacob Devlin, Ming-Wei Chang, Kenton Lee, et al.*2019 · 55,571 citations*](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/)
4. [**SciPy 1.0 — Fundamental Algorithms for Scientific Computing in Python**Pauli Virtanen, Ralf Gommers, Travis E. Oliphant, et al.*2019 · 38,317 citations*](https://md.research.elamur.ai/free/en/papers/virtanen-2019-scipy-1-0-fundamental-algorithms/)
5. [**Swin Transformer: Hierarchical Vision Transformer using Shifted Windows**Ze Liu, Yutong Lin, Yue Cao, et al.*2021 · 33,834 citations*](https://md.research.elamur.ai/free/en/papers/liu-2021-swin-transformer-hierarchical-vision-transformer/)
6. [**SMOTE: Synthetic Minority Over-sampling Technique**Nitesh V. Chawla, Kevin W. Bowyer, Lawrence Hall, et al.*2002 · 32,663 citations*](https://md.research.elamur.ai/free/en/papers/chawla-2002-smote-synthetic-minority-over-sampling/)
7. [**Bleu: a Method for Automatic Evaluation of Machine Translation**Kishore Papineni, Salim Roukos, Todd Ward, et al.*2002 · 32,134 citations*](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
8. [**Rich feature hierarchies for accurate object detection and semantic segmentation**Ross Girshick, Jeff Donahue, Trevor Darrell, et al.*2013 · 31,724 citations*](https://md.research.elamur.ai/free/en/papers/girshick-2013-rich-feature-hierarchies-accurate-object/)
9. [**Efficient Estimation of Word Representations in Vector Space**Tomas Mikolov, Kai Chen, Greg Corrado, et al.*2013 · 28,676 citations*](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/)
10. [**Latent Dirichlet Allocation**David M. Blei, Andrew Y. Ng, Michael I. Jordan*2003 · 28,336 citations*](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/)
11. [**C4.5: Programs for Machine Learning**J. Ross Quinlan*1993 · 23,783 citations*](https://md.research.elamur.ai/free/en/papers/quinlan-1993-c45-programs-machine-learning/)
12. [**Array Programming with NumPy**Charles R. Harris, K. Jarrod Millman, Stéfan J. van der Walt, et al.*2020 · 23,591 citations*](https://md.research.elamur.ai/free/en/papers/harris-2020-array-programming-numpy/)
13. [**An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale**Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, et al.*2020 · 23,273 citations*](https://md.research.elamur.ai/free/en/papers/dosovitskiy-2020-image-worth-16x16-words-transformers/)
14. [**Distributed Representations of Words and Phrases and their Compositionality**Tomas Mikolov, Ilya Sutskever, Kai Chen, et al.*2013 · 21,491 citations*](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/)
15. [**Neural Machine Translation by Jointly Learning to Align and Translate**Dzmitry Bahdanau, Kyunghyun Cho, Yoshua Bengio*2014 · 19,018 citations*](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
16. [**PyTorch: An Imperative Style, High-Performance Deep Learning Library**Adam Paszke, Sam Gross, Francisco Massa, et al.*2019 · 18,400 citations*](https://md.research.elamur.ai/free/en/papers/paszke-2019-pytorch-imperative-style-high-performance/)
17. [**Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks**Nils Reimers, Iryna Gurevych*2019 · 17,026 citations*](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/)
18. [**End-to-End Object Detection with Transformers**Nicolas Carion, Francisco Massa, Gabriel Synnaeve, et al.*2020 · 16,771 citations*](https://md.research.elamur.ai/free/en/papers/carion-2020-end-end-object-detection-transformers/)
19. [**Attention Is All You Need**Ashish Vaswani, Noam Shazeer, Niki Parmar, et al.*2017 · 16,593 citations*](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/)
20. [**Bagging Predictors**Leo Breiman*1996 · 16,498 citations*](https://md.research.elamur.ai/free/en/papers/breiman-1996-bagging-predictors/)
21. [**Convolutional Neural Networks for Sentence Classification**Yoon Kim*2014 · 16,269 citations*](https://md.research.elamur.ai/free/en/papers/kim-2014-convolutional-neural-networks-sentence-classification/)
22. [**The Anatomy of a Large-Scale Hypertextual Web Search Engine**Sergey Brin, Lawrence Page*1998 · 16,230 citations*](https://md.research.elamur.ai/free/en/papers/brin-1998-anatomy-large-scale-hypertextual-web/)
23. [**Transformers: State-of-the-Art Natural Language Processing**Thomas Wolf, Lysandre Debut, Victor Sanh, et al.*2020 · 15,606 citations*](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
24. [**Distilling the Knowledge in a Neural Network**Geoffrey Hinton, Oriol Vinyals, Jeff Dean*2015 · 15,044 citations*](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/)
25. [**Auto-Encoding Variational Bayes**Diederik P Kingma, Max Welling*2013 · 15,018 citations*](https://md.research.elamur.ai/free/en/papers/kingma-2013-auto-encoding-variational-bayes/)

**1** [2](https://md.research.elamur.ai/free/en/shelves/computer-science/page/2/) … [10](https://md.research.elamur.ai/free/en/shelves/computer-science/page/10/) … [92](https://md.research.elamur.ai/free/en/shelves/computer-science/page/92/) [next →](https://md.research.elamur.ai/free/en/shelves/computer-science/page/2/)

Updated 24 October 2026

- [← previous shelf Business, Management and Accounting](https://md.research.elamur.ai/free/en/shelves/business-management-and-accounting/)
- [next shelf → Decision Sciences](https://md.research.elamur.ai/free/en/shelves/decision-sciences/)
- [↑ back to shelves](https://md.research.elamur.ai/free/en/shelves/)