---
title: "Text generation — papers"
description: "Text generation: 114 papers of the Elamur Research library, the most cited first."
canonical: "https://research.elamur.ai/free/en/topics/text-generation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/topics/text-generation/)

[home](https://md.research.elamur.ai/free/en/) / [topics](https://md.research.elamur.ai/free/en/topics/) / Text generation

# ✍️ Text generation

114 papers · page 1 of 5 · the most cited first · [all topics](https://md.research.elamur.ai/free/en/topics/)

Programs that write: from data to text, stories, paraphrases and the choices of a generator.

A survey · about 20 minutes to read · every paper named here has its own page in this library

**In this article**

1. [What it is](#a-what-it-is)
2. [History and milestones](#a-history-and-milestones)
3. [Main problems and tasks](#a-main-problems-and-tasks)
4. [Methods and schools](#a-methods-and-schools)
5. [Datasets, resources and benchmarks](#a-datasets-resources-and-benchmarks)
6. [Landmark papers in this library](#a-landmark-papers-in-this-library)
7. [People](#a-people)
8. [Institutions and places](#a-institutions-and-places)
9. [Open questions](#a-open-questions)
10. [Neighbouring topics](#a-neighbouring-topics)
11. [Key works in this library](#a-key-works-in-this-library)

Text generation, usually called natural language generation (NLG), is the part of language technology that produces text rather than analysing it. A generator starts from something that is not yet prose: a table of numbers, a set of facts from a knowledge base, a semantic graph, a dialogue act, a sentence to be reworded, or simply a prompt to be continued. Its job is to decide what to say, how to organise it and which words and constructions will express it, and then to emit fluent, grammatical text that a reader can trust.

The field is older than most of modern natural language processing. Rule-based generators that planned documents and built sentences from grammars were working in the 1980s, and a large share of the classic ideas (content selection, sentence planning, referring expressions, surface realisation) still shape how researchers describe the task. Since the mid-2010s, neural networks have taken over most of the practical work, and since about 2019 pretrained sequence-to-sequence models and large language models have turned generation into the default interface for many language tasks. That shift has also moved the hardest questions: fluency is now cheap, while faithfulness to the input, control over content and style, social bias and reliable evaluation are not.

The papers here cover data-to-text and table-to-text, graph- and AMR-to-text, surface realisation, referring expression generation, paraphrase and style transfer, controllable and constrained decoding, pretraining for generation, hallucination, and the long-running problem of measuring what a generator has produced.

## What it is

### The classic decomposition

The standard textbook account, associated with Ehud Reiter and Robert Dale, splits a generator into a pipeline. Content determination picks which facts to express. Document planning orders and groups them. Microplanning (or sentence planning) chooses words, decides how facts are aggregated into sentences, and builds referring expressions such as "the black dog" or "it". Surface realisation turns an abstract sentence specification into a well-formed string with inflections, function words and punctuation. Each of these stages became a research area in its own right, and several of them are well represented here.

Referring expression generation is the clearest example of a small problem with deep roots. [Computational Interpretations of the Gricean Maxims in the Generation of Referring Expressions](https://md.research.elamur.ai/free/en/papers/dale-1995-computational-interpretations-gricean-maxims-generation/) asked how a system should describe an object to let a listener identify it, and argued for a fast incremental procedure over the provably shortest description. The later survey [Computational Generation of Referring Expressions: A Survey](https://md.research.elamur.ai/free/en/papers/krahmer-2012-computational-generation-referring-expressions-survey/) traces this line from early planning systems to graph-based and logic-based formulations.

Surface realisation is the other classic pole. Realisers such as REALPRO, described in [A Fast and Portable Realizer for Text Generation Systems](https://md.research.elamur.ai/free/en/papers/lavoie-1997-fast-portable-realizer-text-generation/), take a dependency-style tree of content words and add everything the grammar requires. Grammar-based realisers built on Combinatory Categorial Grammar ([Towards broad coverage surface realization with CCG](https://md.research.elamur.ai/free/en/papers/white-2007-broad-coverage-surface-realization-ccg/)) and statistical realisers ([Broad Coverage Multilingual Deep Sentence Generation with a Stochastic Multi-Level Realizer](https://md.research.elamur.ai/free/en/papers/bohnet-2010-broad-coverage-multilingual-deep-sentence/)) pushed this to broad coverage.

### Kinds of input, kinds of output

Generation tasks can be sorted by their input:

| Input | Typical task | Examples here |
| --- | --- | --- |
| Records, tables, triples | data-to-text, table-to-text | WikiBio, WebNLG, ToTTo, DART, LogicNLG |
| Semantic graphs | AMR-to-text, KG-to-text, DRS-to-text | graph-to-sequence and Graph Transformer models |
| Dialogue acts | response realisation in task-oriented dialogue | SC-LSTM, SC-GPT |
| A sentence | paraphrase, style transfer, simplification | statistical and neural paraphrasers, STRAP |
| A prompt or prefix | open-ended continuation, stories | decoding and control methods for language models |

Two families stand out. In *closed* or *conditional* generation the input largely fixes the meaning, and the main errors are omissions, repetitions and invented content. In *open-ended* generation the model continues a prompt, so the main concerns are coherence, diversity, repetition and the attributes of what gets written (topic, sentiment, toxicity, bias).

### Neighbours

Generation overlaps with [machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) and [summarization](https://md.research.elamur.ai/free/en/topics/summarization/), takes its inputs from [semantic parsing](https://md.research.elamur.ai/free/en/topics/semantic-parsing/), and today relies on [language models](https://md.research.elamur.ai/free/en/topics/language-models/); the last section of this article returns to these links.

## History and milestones

Landmark papers on this topic: the 12 most cited, in time The 12 most cited papers on this topic span 2015–2023; the dot grows with the citations.

**The numbers**

| Paper | Citations |
| --- | --- |
| [2015 — Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems](https://md.research.elamur.ai/free/en/papers/wen-2015-semantically-conditioned-lstm-based-natural/) | 1,192 |
| [2016 — Neural Text Generation from Structured Data with Application to the Biography Domain](https://md.research.elamur.ai/free/en/papers/lebret-2016-neural-text-generation-structured-data/) | 696 |
| [2017 — Creating Training Corpora for NLG Micro-Planners](https://md.research.elamur.ai/free/en/papers/gardent-2017-creating-training-corpora-nlg-micro/) | 526 |
| [2019 — MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://md.research.elamur.ai/free/en/papers/song-2019-mass-masked-sequence-sequence-pre/) | 1,109 |
| [2019 — MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance](https://md.research.elamur.ai/free/en/papers/zhao-2019-moverscore-text-generation-evaluating-contextualized/) | 822 |
| [2019 — The Woman Worked as a Babysitter: On Biases in Language Generation](https://md.research.elamur.ai/free/en/papers/sheng-2019-woman-worked-as-babysitter-biases/) | 751 |
| [2019 — Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://md.research.elamur.ai/free/en/papers/dathathri-2019-plug-play-language-models-simple/) | 562 |
| [2020 — BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/) | 12,752 |
| [2021 — BARTScore: Evaluating Generated Text as Text Generation](https://md.research.elamur.ai/free/en/papers/yuan-2021-bartscore-evaluating-generated-text-text/) | 806 |
| [2022 — Survey of Hallucination in Natural Language Generation](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/) | 5,947 |
| [2022 — Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model](https://md.research.elamur.ai/free/en/papers/smith-2022-using-deepspeed-megatron-train-megatron/) | 595 |
| [2023 — FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://md.research.elamur.ai/free/en/papers/min-2023-factscore-fine-grained-atomic-evaluation/) | 970 |

### Symbolic generation (1980s–1990s)

Early generation research treated language production as planning. Systems reasoned about goals and the hearer, and grammars were often meant to work in both directions, for parsing and for production; [Bidirectional Grammars and the Design of Natural Language Generation Systems](https://md.research.elamur.ai/free/en/papers/appelt-1987-bidirectional-grammars-design-natural-language/) is the oldest paper on this topic in the library. Work in this period also took pragmatics seriously: [Preventing False Temporal Implicatures: Interactive Defaults for Text Generation](https://md.research.elamur.ai/free/en/papers/oberlander-1992-preventing-false-temporal-implicatures-interactive/) looks at how sentence order can suggest unintended time relations, and Dale and Reiter's work on the Gricean maxims shaped referring expression generation for decades. Multilingual generation from an interlingua ([Multi-lingual Sentence Generation from the PIVOT interlingua](https://md.research.elamur.ai/free/en/papers/okumura-1991-multi-lingual-sentence-generation-pivot/)) linked the field to machine translation, and resources such as [DiMLex: A Lexicon of Discourse Markers for Text Generation and Understanding](https://md.research.elamur.ai/free/en/papers/stede-1998-dimlex-lexicon-discourse-markers-text/) supported the choice of connectives.

### Statistical generation (2000s–early 2010s)

Statistical methods arrived first through ranking and through machine translation. [Monolingual Machine Translation for Paraphrase Generation](https://md.research.elamur.ai/free/en/papers/quirk-2004-monolingual-machine-translation-paraphrase-generation/) reused phrase-based translation tools to rewrite English into English, and [Application-driven Statistical Paraphrase Generation](https://md.research.elamur.ai/free/en/papers/zhao-2009-application-driven-statistical-paraphrase-generation/) made the target application part of the model. On the realisation side, broad-coverage grammars were paired with statistical ranking models, and the community began to compare systems on common inputs: [The First Surface Realisation Shared Task: Overview and Evaluation Results](https://md.research.elamur.ai/free/en/papers/belz-2011-first-surface-realisation-shared-task/) set a shared input derived from the Penn Treebank.

### The neural turn (2015–2018)

Recurrent neural networks changed the field quickly. In 2015 Tsung-Hsien Wen and colleagues at Cambridge trained generators for spoken dialogue directly from pairs of dialogue acts and utterances, without alignments or handwritten grammars. In 2016 table-conditioned neural language models scaled data-to-text to hundreds of thousands of Wikipedia biographies, and the first deep paraphrase models appeared. Attention, copy mechanisms and coverage became the standard toolkit, and graph encoders were introduced for semantic graphs. New datasets, notably WebNLG and the E2E challenge data, gave the community harder and cleaner benchmarks.

### Pretraining and large models (2019 onwards)

From 2019, pretraining dominated. MASS and BART showed that a whole encoder-decoder could be pretrained on unlabelled text and then fine-tuned for summarisation, translation, dialogue and data-to-text. GPT-2 and its successors made open-ended continuation fluent enough that control, toxicity and bias became central topics, and a wave of decoding-time steering methods followed. Scale reached hundreds of billions of parameters with [Megatron-Turing NLG](https://md.research.elamur.ai/free/en/papers/smith-2022-using-deepspeed-megatron-train-megatron/). As output quality rose, evaluation and factuality moved to the centre: embedding-based and model-based metrics, studies of hallucination, and in 2023 the use of large language models themselves as judges.

Papers on this topic by year 114 dated papers from 1987 to 2026; the busiest year is 2020 with 15, and 73% of them appeared in 2018 or later.

**The numbers**

| Year | Papers |
| --- | --- |
| 2026 | 7 |
| 2025 | 3 |
| 2024 | 6 |
| 2023 | 8 |
| 2022 | 11 |
| 2021 | 12 |
| 2020 | 15 |
| 2019 | 14 |
| 2018 | 7 |
| 2017 | 6 |
| 2016 | 5 |
| 2015 | 2 |
| 2012 | 1 |
| 2011 | 3 |
| 2010 | 1 |
| 2009 | 1 |
| 2008 | 1 |
| 2007 | 2 |
| 2004 | 1 |
| 1998 | 1 |
| 1997 | 2 |
| 1996 | 1 |
| 1995 | 1 |
| 1992 | 1 |
| 1991 | 1 |
| 1987 | 1 |

The papers here follow this story: few before 2015, then rapid growth, peaking around 2020.

## Main problems and tasks

### Data-to-text and table-to-text

Turning structured records into prose is the most classic NLG task. Its central difficulty is to say everything that matters, exactly once, and nothing that is not supported. Neural systems made this task scalable but also exposed new failure modes: early sequence models dropped facts, repeated them or invented them. Work here moved from restaurant and weather domains to open-domain Wikipedia tables, then to sentences that require reasoning over a table rather than merely restating it, as in [Logical Natural Language Generation from Open-Domain Tables](https://md.research.elamur.ai/free/en/papers/chen-2020-logical-natural-language-generation-open/).

### Graph-to-text and meaning-to-text

Generating from Abstract Meaning Representation graphs, knowledge graphs or discourse representation structures tests whether a model can respect structure. AMR omits tense, number and function words, so the generator must restore them. Papers here range from grammar-based approaches ([AMR-to-text Generation with Synchronous Node Replacement Grammar](https://md.research.elamur.ai/free/en/papers/song-2017-amr-text-generation-synchronous-node/)) and translation-style systems ([Linguistic realisation as machine translation: Comparing different MT models for AMR-to-text generation](https://md.research.elamur.ai/free/en/papers/ferreira-2017-linguistic-realisation-as-machine-translation/)) to graph neural encoders and fully pretrained models, and extend to DRS inputs ([Text Generation from Discourse Representation Structures](https://md.research.elamur.ai/free/en/papers/liu-2021-text-generation-discourse-representation-structures/)).

### Surface realisation

Given a syntactic or shallow semantic input, produce the right sentence. The multilingual shared tasks recast this around Universal Dependencies trees with word order and morphology removed, which made realisation comparable across languages ([The First Multilingual Surface Realisation Shared Task (SR'18): Overview and Evaluation Results](https://md.research.elamur.ai/free/en/papers/mille-2018-first-multilingual-surface-realisation-shared/)).

### Dialogue response realisation

Task-oriented dialogue systems need to render a system act such as "inform(name=X, food=Italian)" as a natural sentence. The challenges are covering every slot, sounding varied rather than templated, and adapting to new domains with little data.

### Paraphrase and style transfer

Paraphrase generation rewords a sentence while keeping its meaning; style transfer changes register or authorial voice. Controlled variants ask for a particular syntax, word order or exemplar. Paraphrasing is also used as a tool: for data augmentation, for adversarial examples, and even for recasting structured prediction as generation, as in [Aspect Sentiment Quad Prediction as Paraphrase Generation](https://md.research.elamur.ai/free/en/papers/zhang-2021-aspect-sentiment-quad-prediction-as/).

### Open-ended generation and control

When a model continues a prompt, the problems are degeneration (dull, repetitive text), incoherence, and steering: keeping to a topic or sentiment, avoiding toxic language, or including required words. Story generation ([Automatic Story Generation: Challenges and Attempts](https://md.research.elamur.ai/free/en/papers/alabdulkarim-2021-automatic-story-generation-challenges-attempts/)) and educational text with controlled difficulty are applications of this line.

### Faithfulness, factuality and bias

Generated text can contradict its source, add unverifiable claims, or portray social groups unequally. Measuring and reducing these failures is now one of the busiest parts of the field.

### Evaluation

Generation has no single correct answer, so evaluation is unusually hard. Overlap metrics such as BLEU and ROUGE reward lexical similarity to one reference; much work here tries to do better with embeddings, with entailment against the input, with trained or prompted models, or with carefully designed human protocols.

## Methods and schools

### Rules, grammars and templates

Rule-based generators remain important wherever exactness and control matter. Template systems are simple and reliable but rigid. Grammar-based realisers encode linguistic knowledge explicitly, whether in Meaning-Text Theory, CCG or systemic grammar ([Some apparently disjoint aims and requirements for grammar development environments” the case of natural language generation](https://md.research.elamur.ai/free/en/papers/bateman-1997-some-apparently-disjoint-aims-requirements/)). Symbolic generation is still alive: [pyrealb at the GEM'24 Data-to-text Task: Symbolic English Text Generation from RDF Triples](https://md.research.elamur.ai/free/en/papers/lapalme-2024-pyrealb-gem-24-data-text/) and [Rule-based Surface Realization of Romanian Weak Pronouns](https://md.research.elamur.ai/free/en/papers/gerstenberger-2025-rule-based-surface-realization-romanian/) show that hand-built realisers continue to be used for precise output and for languages with fewer resources.

### Statistical generation

Statistical NLG typically overgenerated candidates with a grammar or phrase table and ranked them with language models or discriminative rankers. The paraphrase work of Quirk, Brockett and Dolan, the multi-table model of Zhao and colleagues, and the stochastic multi-level realiser of Bohnet, Wanner and Mille are representative. The overgenerate-and-rerank pattern survived into the neural era.

### Neural encoder-decoders

Recurrent networks with attention learned to map inputs to text end to end. Key additions were:

- **Conditioning and gating**, so that the content to be expressed is tracked as it is used up (SC-LSTM, the neural checklist model).
- **Copy mechanisms**, which let a model reproduce names and numbers from the input instead of relying on a fixed vocabulary.
- **Graph encoders**, from graph-state LSTMs to Graph Transformers, which read semantic graphs directly instead of flattening them.
- **Latent-variable models**, such as variational autoencoders for diverse paraphrases and hidden semi-Markov models that learn interpretable templates.

A recurring debate concerned the classic pipeline. End-to-end models were fluent but hard to steer and prone to dropping or inventing facts. Several papers here argue for restoring explicit structure: separating a symbolic plan from neural realisation, learning discrete templates, or keeping an explicit record of covered content.

### Pretrained sequence-to-sequence models and large language models

Pretraining replaced task-specific architectures. Denoising objectives (MASS, BART) and text-to-text models (T5 and its language-specific variants such as [AraT5: Text-to-Text Transformers for Arabic Language Generation](https://md.research.elamur.ai/free/en/papers/nagoudi-2022-arat5-text-text-transformers-arabic/)) can be fine-tuned for almost any generation task. Simply linearising a graph and feeding it to such a model turned out to beat many specialised graph encoders, although this raises the question of whether the model reads the structure or relies on memorised knowledge.

### Decoding and control

Because a language model defines a distribution, much of generation is a search problem. Beam search favours probable but bland and repetitive text; top-k and nucleus sampling add variety at the cost of drift. Work here addresses this at several levels:

- **Training objectives**, such as unlikelihood training, which penalise repetition directly.
- **Decoding-time steering**, which modifies the next-token distribution with an attribute model: gradient-based (PPLM), discriminator-based (FUDGE), or product-of-experts style (DExperts).
- **Search with lookahead**, which estimates whether constraints can still be met (NeuroLogic A*esque decoding), and **contrastive decoding**, which compares a large and a small model.
- **Reward-conditioned fine-tuning**, such as Quark, which conditions on quantised reward levels.
- **Non-autoregressive models**, notably diffusion language models, which refine a whole sequence at once and make global, structural constraints easier.

Pragmatic approaches form a smaller school that treats generation as communication with a listener, often through the Rational Speech Acts framework ([Pragmatically Informative Text Generation](https://md.research.elamur.ai/free/en/papers/shen-2019-pragmatically-informative-text-generation/), [RSA-Control: A Pragmatics-Grounded Lightweight Controllable Text Generation Framework](https://md.research.elamur.ai/free/en/papers/wang-2024-rsa-control-pragmatics-grounded-lightweight/)).

## Datasets, resources and benchmarks

Many of the field's datasets were introduced in papers in this library:

- **WikiBio**, pairing Wikipedia infoboxes with the first sentence of each biography, introduced with a table-conditioned neural language model in 2016.
- **WebNLG**, built from DBpedia triples with crowdsourced texts, designed to exercise microplanning.
- **E2E**, a restaurant-domain dataset whose noisy references were later cleaned, with large effects on faithfulness.
- **ToTTo**, open-domain table-to-text with highlighted cells and carefully edited target sentences.
- **DART**, open-domain triple-to-text built from Wikipedia tables and merged with WebNLG and cleaned E2E.
- **LogicNLG**, which requires statements logically entailed by a table.
- **AGENDA**, scientific knowledge graphs extracted from abstracts.
- **AMR corpora** from the LDC releases, the standard test bed for AMR-to-text.
- **FewShotWOZ**, a low-resource benchmark for dialogue NLG.
- **HADES**, a benchmark for spotting invented spans token by token without a reference.
- **ARGEN**, a multi-task Arabic generation benchmark.
- **CDS**, a corpus of diverse styles for style transfer.

Tools matter too: Texygen offered a shared platform for comparing GAN-based and maximum-likelihood generators, compare-mt contrasts two systems' outputs in detail, and the surface realisation shared tasks (2011, 2018, 2020) supplied common inputs.

## Landmark papers in this library

Citations of the papers on this topic, by when they were published Papers published in 2020–2024 gather 67% of the 40,591 citations of the papers on this topic.

**The numbers**

| Published | Papers and their citations |
| --- | --- |
| 2025–2029 | 10 papers, 14 citations |
| 2020–2024 | 52 papers, 27,260 citations |
| 2015–2019 | 34 papers, 11,197 citations |
| 2010–2014 | 5 papers, 719 citations |
| 2005–2009 | 4 papers, 290 citations |
| 2000–2004 | 1 papers, 452 citations |
| 1995–1999 | 5 papers, 570 citations |
| 1990–1994 | 2 papers, 30 citations |
| 1985–1989 | 1 papers, 59 citations |

### BART (2020)

[BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/) pretrains a Transformer encoder-decoder by corrupting text and teaching the model to restore it. Comparing several kinds of corruption, the authors found that shuffling sentences combined with filling masked spans worked best. One model then performed well on both understanding benchmarks and generation tasks such as abstractive summarisation and dialogue. BART became a standard backbone for generation and even for evaluation metrics.

### MASS (2019)

[MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://md.research.elamur.ai/free/en/papers/song-2019-mass-masked-sequence-sequence-pre/) was an earlier step in the same direction: mask a contiguous span in the encoder input and make the decoder rebuild just that span. It improved low-resource and unsupervised translation, summarisation and response generation, showing that pretraining both halves of a sequence-to-sequence model pays off.

### Survey of Hallucination in Natural Language Generation (2022)

[Survey of Hallucination in Natural Language Generation](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/) brought work on unfaithful output together across summarisation, dialogue, question answering, data-to-text, translation and vision-language tasks. Its distinction between intrinsic hallucination (contradicting the source) and extrinsic hallucination (unverifiable from the source) is now widely used, and a later section extends the analysis to large language models.

### SC-LSTM (2015)

[Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems](https://md.research.elamur.ai/free/en/papers/wen-2015-semantically-conditioned-lstm-based-natural/) added a gated control vector to an LSTM language model so that the dialogue act and its slots are gradually consumed as they are expressed. It learned sentence planning and realisation together from unaligned data and was preferred by human judges over earlier baselines. Its companion, [Stochastic Language Generation in Dialogue using Recurrent Neural Networks with Convolutional Sentence Reranking](https://md.research.elamur.ai/free/en/papers/wen-2015-stochastic-language-generation-dialogue-recurrent/), used a simpler decaying gate and a reranking step. These papers made data-driven dialogue NLG mainstream.

### Neural generation from tables (2016)

[Neural Text Generation from Structured Data with Application to the Biography Domain](https://md.research.elamur.ai/free/en/papers/lebret-2016-neural-text-generation-structured-data/) introduced the large WikiBio corpus and a neural language model conditioned on infobox fields, with copying of table words. It showed that neural data-to-text works on open, varied data where count-based models suffer from sparsity, and its dataset became a standard benchmark, along with a known caveat: the references often contain facts not in the table.

### WebNLG (2017)

[Creating Training Corpora for NLG Micro-Planners](https://md.research.elamur.ai/free/en/papers/gardent-2017-creating-training-corpora-nlg-micro/) described a semi-automatic method for building data-to-text corpora from a knowledge base and produced the WebNLG data from DBpedia. Its inputs are more varied and its texts lexically richer than earlier crowdsourced sets, making it a harder benchmark and the basis of a series of shared challenges.

### FActScore (2023)

[FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://md.research.elamur.ai/free/en/papers/min-2023-factscore-fine-grained-atomic-evaluation/) breaks long outputs into small atomic claims and checks each against a knowledge source. Applied to biographies, it showed that even strong chat models made many factual errors, especially about rarely mentioned people, and its automatic estimator made large-scale comparisons cheap.

### MoverScore and BARTScore (2019, 2021)

[MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance](https://md.research.elamur.ai/free/en/papers/zhao-2019-moverscore-text-generation-evaluating-contextualized/) compares a system output and a reference as distributions of contextual embeddings, rewarding paraphrases that word overlap would miss. [BARTScore: Evaluating Generated Text as Text Generation](https://md.research.elamur.ai/free/en/papers/yuan-2021-bartscore-evaluating-generated-text-text/) goes a step further and scores text by how likely a pretrained BART model finds it given the source or reference, with different directions targeting precision, recall and faithfulness. Together they mark the move from n-gram matching to model-based evaluation.

### Biases in language generation (2019)

[The Woman Worked as a Babysitter: On Biases in Language Generation](https://md.research.elamur.ai/free/en/papers/sheng-2019-woman-worked-as-babysitter-biases/) introduced *regard*, a measure of how a sentence makes readers perceive a demographic group, and showed that sentiment alone understates bias. Prompting GPT-2 with templates mentioning different groups revealed unequal treatment. The same group followed with methods to induce and reduce bias and with a broad survey.

### Plug and Play Language Models (2019)

[Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://md.research.elamur.ai/free/en/papers/dathathri-2019-plug-play-language-models-simple/) steers a frozen GPT-2 by nudging its hidden activations with gradients from tiny attribute models, such as a bag of topic words or a small sentiment classifier. It showed that topic, sentiment and toxicity can be controlled without retraining the language model, and it became the baseline for a family of decoding-time control methods.

### Unlikelihood training (2019)

[Neural Text Generation with Unlikelihood Training](https://md.research.elamur.ai/free/en/papers/welleck-2019-neural-text-generation-unlikelihood-training/) argued that repetitive, dull output stems partly from the maximum-likelihood objective itself. Adding a penalty on tokens that repeat the context, and fine-tuning on the model's own continuations, sharply reduced repetition while keeping perplexity close to the baseline.

### ToTTo (2020)

[ToTTo: A Controlled Table-To-Text Generation Dataset](https://md.research.elamur.ai/free/en/papers/parikh-2020-totto-controlled-table-text-generation/) asks a model to describe highlighted cells of a Wikipedia table, with target sentences edited by annotators to remove anything the table does not support. By pairing natural sentences with faithful targets, it became a central benchmark for precise generation and for faithfulness metrics.

### Diffusion-LM (2022)

[Diffusion-LM Improves Controllable Text Generation](https://md.research.elamur.ai/free/en/papers/li-2022-diffusion-lm-improves-controllable-text/) adapted continuous diffusion to text, generating word vectors from noise and refining them step by step. Because the whole sequence is revised together, gradient-based control of complex targets such as syntax trees became much more successful than with left-to-right models, though decoding is slower.

### Separating planning from realisation (2019)

[Step-by-Step: Separating Planning from Realization in Neural Data-to-Text Generation](https://md.research.elamur.ai/free/en/papers/moryossef-2019-step-step-separating-planning-realization/) restored the classic pipeline inside a neural system: a symbolic planner fixes how WebNLG facts are grouped and ordered, and a neural realiser chooses the words. The result was as fluent as an end-to-end baseline and noticeably more faithful, and it allowed diverse, controllable outputs.

## People

Authors with the most papers on this topic Michael White leads with 5 papers; the 12 authors shown wrote 44 of the papers on this topic between them, counting each paper once per author.

**The numbers**

| Author | Papers here · their citations |
| --- | --- |
| [Michael White](https://md.research.elamur.ai/free/en/authors/michael-white/) | 5 papers · 284 citations |
| [Luke Zettlemoyer](https://md.research.elamur.ai/free/en/authors/luke-zettlemoyer/) | 4 papers · 14,269 citations |
| [Kai-Wei Chang](https://md.research.elamur.ai/free/en/authors/kai-wei-chang/) | 4 papers · 1,184 citations |
| [Yejin Choi](https://md.research.elamur.ai/free/en/authors/yejin-choi/) | 4 papers · 1,000 citations |
| [Bernd Bohnet](https://md.research.elamur.ai/free/en/authors/bernd-bohnet/) | 4 papers · 223 citations |
| [Leo Wanner](https://md.research.elamur.ai/free/en/authors/leo-wanner/) | 4 papers · 223 citations |
| [Simon Mille](https://md.research.elamur.ai/free/en/authors/simon-mille/) | 4 papers · 223 citations |
| [Mike Lewis](https://md.research.elamur.ai/free/en/authors/mike-lewis/) | 3 papers · 13,968 citations |
| [Emily Sheng](https://md.research.elamur.ai/free/en/authors/emily-sheng/) | 3 papers · 1,169 citations |
| [Nanyun Peng](https://md.research.elamur.ai/free/en/authors/nanyun-peng/) | 3 papers · 1,169 citations |
| [Graham Neubig](https://md.research.elamur.ai/free/en/authors/graham-neubig/) | 3 papers · 1,075 citations |
| [Dipanjan Das](https://md.research.elamur.ai/free/en/authors/dipanjan-das/) | 3 papers · 943 citations |

- **[Michael White](https://md.research.elamur.ai/free/en/authors/michael-white/)** (The Ohio State University) is the most prolific author on this topic in the library. His papers develop broad-coverage surface realisation with CCG, including detailed treatments of punctuation and complementizer choice, and he co-organised the first surface realisation shared task.
- **[Luke Zettlemoyer](https://md.research.elamur.ai/free/en/authors/luke-zettlemoyer/)** (University of Washington, Meta) co-authored BART, the neural checklist model for globally coherent text, contrastive decoding and FActScore, spanning pretraining, decoding and factuality.
- **[Mike Lewis](https://md.research.elamur.ai/free/en/authors/mike-lewis/)** (University of Washington) led BART and contributed to contrastive decoding and FActScore.
- **[Yejin Choi](https://md.research.elamur.ai/free/en/authors/yejin-choi/)** (Stanford University) appears on the checklist model, DExperts, Quark and NeuroLogic A*esque decoding, a body of work on steering and constraining large language models.
- **[Kai-Wei Chang](https://md.research.elamur.ai/free/en/authors/kai-wei-chang/)** (University of California, Los Angeles) and **[Nanyun Peng](https://md.research.elamur.ai/free/en/authors/nanyun-peng/)** co-authored with **[Emily Sheng](https://md.research.elamur.ai/free/en/authors/emily-sheng/)** (Microsoft) the series on societal bias in generation: the regard measure, controllable bias triggers and a survey of the area. Chang also worked on syntactically controlled paraphrasing from AMR.
- **[Bernd Bohnet](https://md.research.elamur.ai/free/en/authors/bernd-bohnet/)**, **[Leo Wanner](https://md.research.elamur.ai/free/en/authors/leo-wanner/)** (Universitat Pompeu Fabra, ICREA) and **[Simon Mille](https://md.research.elamur.ai/free/en/authors/simon-mille/)** built a stochastic multi-level realiser and organised the multilingual surface realisation shared tasks based on underspecified Universal Dependencies.
- **[Anja Belz](https://md.research.elamur.ai/free/en/authors/anja-belz/)** (University of Aberdeen, Dublin City University) led the first surface realisation shared task and co-designed the later multilingual ones; her work stands for shared evaluation in NLG.
- **[Graham Neubig](https://md.research.elamur.ai/free/en/authors/graham-neubig/)** (Carnegie Mellon University and others) co-authored BARTScore, the compare-mt analysis tool and a survey on feedback for generation.
- **[Dipanjan Das](https://md.research.elamur.ai/free/en/authors/dipanjan-das/)** worked on ToTTo, the PARENT metric for divergent references, and the AIS framework for attribution.
- **[Sean Welleck](https://md.research.elamur.ai/free/en/authors/sean-welleck/)** (Carnegie Mellon University) introduced unlikelihood training and co-authored Quark and NeuroLogic A*esque decoding with **[Ximing Lu](https://md.research.elamur.ai/free/en/authors/ximing-lu/)**.
- **[Tsung-Hsien Wen](https://md.research.elamur.ai/free/en/authors/tsung-hsien-wen/)**, **[Milica Gašić](https://md.research.elamur.ai/free/en/authors/milica-gasic/)** (Heinrich Heine University Düsseldorf) and **[Steve Young](https://md.research.elamur.ai/free/en/authors/steve-young/)**, with **[Nikola Mrkšić](https://md.research.elamur.ai/free/en/authors/nikola-mrksic/)**, **[Pei-Hao Su](https://md.research.elamur.ai/free/en/authors/pei-hao-su/)** and **[David Vandyke](https://md.research.elamur.ai/free/en/authors/david-vandyke/)**, formed the Cambridge group behind the RNN and SC-LSTM dialogue generators.
- **[Andrea Madotto](https://md.research.elamur.ai/free/en/authors/andrea-madotto/)** (University of Michigan) co-authored both PPLM and the hallucination survey.
- **[Kalpesh Krishna](https://md.research.elamur.ai/free/en/authors/kalpesh-krishna/)** and **[Mohit Iyyer](https://md.research.elamur.ai/free/en/authors/mohit-iyyer/)** (University of Massachusetts Amherst) recast style transfer as paraphrasing and co-authored FActScore.
- **[Tao Qin](https://md.research.elamur.ai/free/en/authors/tao-qin/)**, **[Xu Tan](https://md.research.elamur.ai/free/en/authors/xu-tan/)** and **[Tie-Yan Liu](https://md.research.elamur.ai/free/en/authors/tie-yan-liu/)** (Microsoft) wrote MASS and identified the representation degeneration problem in generation models.
- **[Hannaneh Hajishirzi](https://md.research.elamur.ai/free/en/authors/hannaneh-hajishirzi/)** (University of Washington, Allen Institute for AI) co-authored GraphWriter and FActScore.
- **[Ali Hamdi](https://md.research.elamur.ai/free/en/authors/ali-hamdi/)** (October University of Modern Sciences and Arts) represents recent work on Arabic medical text generation.

Symbolic-era pioneers also appear: [Robert Dale](https://md.research.elamur.ai/free/en/authors/robert-dale/) and [Ehud Reiter](https://md.research.elamur.ai/free/en/authors/ehud-reiter/) on referring expressions, and [Emiel Krahmer](https://md.research.elamur.ai/free/en/authors/emiel-krahmer/) and [Kees van Deemter](https://md.research.elamur.ai/free/en/authors/kees-van-deemter/), whose survey is the standard account of that subfield. [Claire Gardent](https://md.research.elamur.ai/free/en/authors/claire-gardent/) led the WebNLG work, and [Ondřej Dušek](https://md.research.elamur.ai/free/en/authors/ondrej-dusek/) and [Verena Rieser](https://md.research.elamur.ai/free/en/authors/verena-rieser/) showed how much noisy training data hurts neural NLG.

## Institutions and places

Where the authors on this topic work now Carnegie Mellon University is the current home of authors of 17 papers on this topic; counted are the authors with a page here, by their latest institution.

**The numbers**

| Institution | Papers |
| --- | --- |
| [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) | 17 papers |
| [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) | 12 papers |
| [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) | 11 papers |
| [Google (United States)](https://md.research.elamur.ai/free/en/companies/google/) | 9 papers |
| [Microsoft (United States)](https://md.research.elamur.ai/free/en/companies/microsoft/) | 9 papers |
| [Amazon (United States)](https://md.research.elamur.ai/free/en/companies/amazon/) | 8 papers |
| [Centre National de la Recherche Scientifique](https://md.research.elamur.ai/free/en/labs/centre-national-de-la-recherche-scientifique/) | 8 papers |
| [Charles University](https://md.research.elamur.ai/free/en/universities/charles-university/) | 7 papers |
| [Dublin City University](https://md.research.elamur.ai/free/en/universities/dublin-city-university/) | 7 papers |
| [Google DeepMind (United Kingdom)](https://md.research.elamur.ai/free/en/companies/google-deepmind/) | 7 papers |

The current homes of authors here are dominated by large American research universities and industrial labs. [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/), [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) and the [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) lead, reflecting work on decoding, control, evaluation and factuality. Companies follow closely: [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/), [Google](https://md.research.elamur.ai/free/en/companies/google/), [Amazon](https://md.research.elamur.ai/free/en/companies/amazon/), [Meta](https://md.research.elamur.ai/free/en/companies/meta/) and [Google DeepMind](https://md.research.elamur.ai/free/en/companies/google-deepmind/), together with the [Allen Institute for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/allen-institute-for-artificial-intelligence/). Other American centres include [MIT](https://md.research.elamur.ai/free/en/universities/massachusetts-institute-of-technology/), [UC Berkeley](https://md.research.elamur.ai/free/en/universities/university-of-california-berkeley/), [UC San Diego](https://md.research.elamur.ai/free/en/universities/university-of-california-san-diego/) and [The Ohio State University](https://md.research.elamur.ai/free/en/universities/the-ohio-state-university/), the home of grammar-based realisation work.

Europe holds the traditions of symbolic NLG and shared evaluation. The [University of Aberdeen](https://md.research.elamur.ai/free/en/universities/university-of-aberdeen/) has long been a centre of data-to-text research; [Dublin City University](https://md.research.elamur.ai/free/en/universities/dublin-city-university/) and the [ADAPT Centre](https://md.research.elamur.ai/free/en/labs/adapt-centre-for-digital-content-technology/) host surface realisation work; [Charles University](https://md.research.elamur.ai/free/en/universities/charles-university/) and the [CNRS](https://md.research.elamur.ai/free/en/labs/centre-national-de-la-recherche-scientifique/) in France also appear prominently. In Asia, the [Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/government/chinese-academy-of-sciences/) and the [University of Hong Kong](https://md.research.elamur.ai/free/en/universities/university-of-hong-kong/) stand out.

Countries of the authors on this topic United States comes first: authors working there now wrote 65 papers on this topic.

**The numbers**

| Country | Papers |
| --- | --- |
| United States | 65 papers |
| China | 34 papers |
| United Kingdom | 27 papers |
| Germany | 16 papers |
| Hong Kong | 11 papers |
| France | 9 papers |
| Czechia | 8 papers |
| India | 8 papers |
| Ireland | 8 papers |
| Israel | 8 papers |

By country, the United States comes first by a wide margin, followed by China, the United Kingdom and Germany.

Where the papers on this topic were published Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics carries the most, 4 of 114.

**The numbers**

| Venue | Papers |
| --- | --- |
| Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics | 4 papers |
| Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) | 3 papers |
| Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) | 3 papers |
| Transactions of the Association for Computational Linguistics | 3 papers |
| Computational Linguistics | 2 papers |
| Findings of the Association for Computational Linguistics: EMNLP 2020 | 2 papers |
| Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume | 2 papers |
| Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing | 2 papers |
| Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) | 2 papers |
| Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies | 2 papers |

Papers appear mostly in the main venues of the Association for Computational Linguistics: the annual ACL meeting, EMNLP, NAACL and EACL, the journals *Transactions of the ACL* and *Computational Linguistics*, and the specialised natural language generation conferences and workshops.

## Open questions

- **Faithfulness.** Models still add, omit or distort content. Cleaning data, planning explicitly, uncertainty-aware decoding and attribution frameworks all help, but no method guarantees that every statement is supported by the input.
- **Reading the structure.** Pretrained models given linearised graphs do well, yet experiments with shuffled or corrupted inputs suggest they often lean on language modelling and memorised facts. Whether they really use the input structure is unresolved.
- **Reasoning while generating.** Sentences that compare, count or aggregate over tables remain hard; left-to-right decoding must commit to words whose correctness depends on later choices.
- **Evaluation.** Overlap metrics correlate poorly with human judgement, references can diverge from their inputs, and model-based judges are sensitive to prompts and may share the biases of the systems they evaluate. Human evaluation protocols themselves vary widely.
- **Control without cost.** Decoding-time methods trade fluency against attribute strength, and diffusion models offer stronger control at higher cost. Combining several constraints reliably is still open.
- **Bias and harm.** Studies are mostly English-centred and focused on binary gender; measures such as regard are proxies, and links to concrete harms are rarely established.
- **Languages beyond English.** Multilingual realisation tasks and models such as AraT5 show progress, but resources for most languages remain thin, and morphologically rich languages pose particular problems for realisation.

## Neighbouring topics

Neighbouring topics: papers shared Semantic parsing & meaning representation shares the most: 9 of the 114 papers here (8%) stand on both.

**The numbers**

| Topic | Papers shared |
| --- | --- |
| [Semantic parsing & meaning representation](https://md.research.elamur.ai/free/en/topics/semantic-parsing/) | 9 shared · 8% |
| [Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) | 7 shared · 6% |
| [Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) | 6 shared · 5% |
| [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) | 6 shared · 5% |
| [Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) | 6 shared · 5% |
| [Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) | 5 shared · 4% |
| [Discourse analysis](https://md.research.elamur.ai/free/en/topics/discourse-analysis/) | 4 shared · 4% |
| [Implicature](https://md.research.elamur.ai/free/en/topics/implicature/) | 4 shared · 4% |
| [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) | 4 shared · 4% |
| [Knowledge graphs & ontologies](https://md.research.elamur.ai/free/en/topics/knowledge-graphs/) | 2 shared · 2% |

Text generation shares the most papers with [semantic parsing and meaning representation](https://md.research.elamur.ai/free/en/topics/semantic-parsing/), since AMR and DRS are both parsed from text and generated back into it. It shares methods and metrics with [machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/), from monolingual translation for paraphrase to evaluation tools built for translation. Its modern models belong to [language models and pretraining](https://md.research.elamur.ai/free/en/topics/language-models/), while [multilingual and cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) covers its multilingual realisation tasks and language-specific generators.

[Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) contribute the response realisation problem; [evaluation and benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) covers its metrics and meta-evaluation. Pragmatic generation connects it to [implicature](https://md.research.elamur.ai/free/en/topics/implicature/), [speech act theory](https://md.research.elamur.ai/free/en/topics/speech-act-theory/) and [discourse analysis](https://md.research.elamur.ai/free/en/topics/discourse-analysis/), the last through discourse markers, coherence metrics and discourse relations in generated text. Further links run to [knowledge graphs and ontologies](https://md.research.elamur.ai/free/en/topics/knowledge-graphs/), [semantic role labeling](https://md.research.elamur.ai/free/en/topics/semantic-role-labeling/), [syntax and grammar formalisms](https://md.research.elamur.ai/free/en/topics/syntax-grammar/), [Universal Dependencies](https://md.research.elamur.ai/free/en/topics/universal-dependencies/), [sentiment and emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/), [bias, fairness and safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/), [coreference resolution](https://md.research.elamur.ai/free/en/topics/coreference-resolution/), [information retrieval](https://md.research.elamur.ai/free/en/topics/information-retrieval/), [low-resource languages](https://md.research.elamur.ai/free/en/topics/low-resource-languages/) and [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/), where Arabic generation work sits. [Summarization](https://md.research.elamur.ai/free/en/topics/summarization/) is the closest task not covered directly here.

The shelves this topic stands on Natural language processing: 52 of the 114 papers here.

**The numbers**

| Shelf | Papers |
| --- | --- |
| [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) | 52 papers |
| [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) | 48 papers |
| [Machine translation](https://md.research.elamur.ai/free/en/shelves/machine-translation/) | 42 papers |
| [Other](https://md.research.elamur.ai/free/en/shelves/other/) | 14 papers |
| [AI & machine learning](https://md.research.elamur.ai/free/en/shelves/ai/) | 3 papers |
| [Education](https://md.research.elamur.ai/free/en/shelves/education/) | 1 paper |

On the shelves of the library, most papers on this topic sit under [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/), [Linguistics and languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) and [Machine translation](https://md.research.elamur.ai/free/en/shelves/machine-translation/), with a few under [AI and machine learning](https://md.research.elamur.ai/free/en/shelves/ai/) and [Education](https://md.research.elamur.ai/free/en/shelves/education/).

## Key works in this library

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/) — denoising pretraining of a full encoder-decoder.
- [MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://md.research.elamur.ai/free/en/papers/song-2019-mass-masked-sequence-sequence-pre/) — span masking to pretrain generators.
- [Survey of Hallucination in Natural Language Generation](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/) — the reference survey on unfaithful output.
- [Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems](https://md.research.elamur.ai/free/en/papers/wen-2015-semantically-conditioned-lstm-based-natural/) — gated neural generation for dialogue.
- [Neural Text Generation from Structured Data with Application to the Biography Domain](https://md.research.elamur.ai/free/en/papers/lebret-2016-neural-text-generation-structured-data/) — WikiBio and table-conditioned language models.
- [Creating Training Corpora for NLG Micro-Planners](https://md.research.elamur.ai/free/en/papers/gardent-2017-creating-training-corpora-nlg-micro/) — the WebNLG dataset.
- [ToTTo: A Controlled Table-To-Text Generation Dataset](https://md.research.elamur.ai/free/en/papers/parikh-2020-totto-controlled-table-text-generation/) — faithful open-domain table-to-text.
- [DART: Open-Domain Structured Data Record to Text Generation](https://md.research.elamur.ai/free/en/papers/nan-2020-dart-open-domain-structured-data/) — open-domain triples to text.
- [Logical Natural Language Generation from Open-Domain Tables](https://md.research.elamur.ai/free/en/papers/chen-2020-logical-natural-language-generation-open/) — generation that must reason over tables.
- [A Graph-to-Sequence Model for AMR-to-Text Generation](https://md.research.elamur.ai/free/en/papers/song-2018-graph-sequence-model-amr-text/) — encoding AMR graphs directly.
- [AMR-To-Text Generation with Graph Transformer](https://md.research.elamur.ai/free/en/papers/wang-2020-amr-text-generation-graph-transformer/) — attention over graph structure.
- [Text Generation from Knowledge Graphs with Graph Transformers](https://md.research.elamur.ai/free/en/papers/kedziorski-2019-text-generation-knowledge-graphs-graph/) — abstracts from extracted knowledge graphs.
- [Investigating Pretrained Language Models for Graph-to-Text Generation](https://md.research.elamur.ai/free/en/papers/ribeiro-2021-investigating-pretrained-language-models-graph/) — linearised graphs with BART and T5.
- [Globally Coherent Text Generation with Neural Checklist Models](https://md.research.elamur.ai/free/en/papers/kiddon-2016-globally-coherent-text-generation-neural/) — tracking which inputs have been mentioned.
- [Learning Neural Templates for Text Generation](https://md.research.elamur.ai/free/en/papers/wiseman-2018-learning-neural-templates-text-generation/) — interpretable latent templates.
- [Step-by-Step: Separating Planning from Realization in Neural Data-to-Text Generation](https://md.research.elamur.ai/free/en/papers/moryossef-2019-step-step-separating-planning-realization/) — symbolic plans, neural wording.
- [Semantic Noise Matters for Neural Natural Language Generation](https://md.research.elamur.ai/free/en/papers/dusek-2019-semantic-noise-matters-neural-natural/) — cleaning data to cut slot errors.
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://md.research.elamur.ai/free/en/papers/dathathri-2019-plug-play-language-models-simple/) — steering a frozen model with small attribute models.
- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://md.research.elamur.ai/free/en/papers/liu-2021-dexperts-decoding-time-controlled-text/) — expert and anti-expert models at decoding time.
- [FUDGE: Controlled Text Generation With Future Discriminators](https://md.research.elamur.ai/free/en/papers/yang-2021-fudge-controlled-text-generation-future/) — prefix classifiers that predict the final attribute.
- [Neural Text Generation with Unlikelihood Training](https://md.research.elamur.ai/free/en/papers/welleck-2019-neural-text-generation-unlikelihood-training/) — a training fix for repetition.
- [Contrastive Decoding: Open-ended Text Generation as Optimization](https://md.research.elamur.ai/free/en/papers/li-2022-contrastive-decoding-open-ended-text/) — large-versus-small model scoring.
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://md.research.elamur.ai/free/en/papers/krishna-2020-reformulating-unsupervised-style-transfer-as/) — style transfer through paraphrasing.
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://md.research.elamur.ai/free/en/papers/sheng-2019-woman-worked-as-babysitter-biases/) — the regard measure of bias.
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://md.research.elamur.ai/free/en/papers/min-2023-factscore-fine-grained-atomic-evaluation/) — factual precision over atomic claims.
- [Handling Divergent Reference Texts when Evaluating Table-to-Text Generation](https://md.research.elamur.ai/free/en/papers/dhingra-2019-handling-divergent-reference-texts-when/) — the PARENT metric.
- [Is ChatGPT a Good NLG Evaluator? A Preliminary Study](https://md.research.elamur.ai/free/en/papers/wang-2023-chatgpt-good-nlg-evaluator-preliminary/) — an early study of language models as judges.
- [Computational Interpretations of the Gricean Maxims in the Generation of Referring Expressions](https://md.research.elamur.ai/free/en/papers/dale-1995-computational-interpretations-gricean-maxims-generation/) — the incremental algorithm.

## All 114 papers on this topic

**1** [2](https://md.research.elamur.ai/free/en/topics/text-generation/page/2/) … [5](https://md.research.elamur.ai/free/en/topics/text-generation/page/5/) [next →](https://md.research.elamur.ai/free/en/topics/text-generation/page/2/)

- [BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/) — Mike Lewis, Yinhan Liu, Naman Goyal, et al. · 2020 · 12 752 citations
- [Survey of Hallucination in Natural Language Generation](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/) — Ziwei Ji, Nayeon Lee, Rita Frieske, et al. · 2022 · 5 947 citations
- [Semantically Conditioned LSTM-based Natural Language Generation for Spoken Dialogue Systems](https://md.research.elamur.ai/free/en/papers/wen-2015-semantically-conditioned-lstm-based-natural/) — Tsung-Hsien Wen, Milica Gašić, Nikola Mrkšić, et al. · 2015 · 1 192 citations
- [MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://md.research.elamur.ai/free/en/papers/song-2019-mass-masked-sequence-sequence-pre/) — Kaitao Song, Xu Tan, Tao Qin, et al. · 2019 · 1 109 citations
- [FActScore: Fine-grained Atomic Evaluation of Factual Precision in Long Form Text Generation](https://md.research.elamur.ai/free/en/papers/min-2023-factscore-fine-grained-atomic-evaluation/) — Sewon Min, Kalpesh Krishna, Xinxi Lyu, et al. · 2023 · 970 citations
- [MoverScore: Text Generation Evaluating with Contextualized Embeddings and Earth Mover Distance](https://md.research.elamur.ai/free/en/papers/zhao-2019-moverscore-text-generation-evaluating-contextualized/) — Wei Zhao, Maxime Peyrard, Fei Liu, et al. · 2019 · 822 citations
- [BARTScore: Evaluating Generated Text as Text Generation](https://md.research.elamur.ai/free/en/papers/yuan-2021-bartscore-evaluating-generated-text-text/) — Weizhe Yuan, Graham Neubig, Pengfei Liu · 2021 · 806 citations
- [The Woman Worked as a Babysitter: On Biases in Language Generation](https://md.research.elamur.ai/free/en/papers/sheng-2019-woman-worked-as-babysitter-biases/) — Emily Sheng, Kai-Wei Chang, Premkumar Natarajan, et al. · 2019 · 751 citations
- [Neural Text Generation from Structured Data with Application to the Biography Domain](https://md.research.elamur.ai/free/en/papers/lebret-2016-neural-text-generation-structured-data/) — Remi Lebret, David Grangier, Michael Auli · 2016 · 696 citations
- [Using DeepSpeed and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model](https://md.research.elamur.ai/free/en/papers/smith-2022-using-deepspeed-megatron-train-megatron/) — Shaden Smith, Mostofa Patwary, Brandon Norick, et al. · 2022 · 595 citations
- [Plug and Play Language Models: A Simple Approach to Controlled Text Generation](https://md.research.elamur.ai/free/en/papers/dathathri-2019-plug-play-language-models-simple/) — Sumanth Dathathri, Andrea Madotto, Janice Lan, et al. · 2019 · 562 citations
- [Creating Training Corpora for NLG Micro-Planners](https://md.research.elamur.ai/free/en/papers/gardent-2017-creating-training-corpora-nlg-micro/) — Claire Gardent, Anastasia Shimorina, Shashi Narayan, et al. · 2017 · 526 citations
- [Neural Text Generation with Unlikelihood Training](https://md.research.elamur.ai/free/en/papers/welleck-2019-neural-text-generation-unlikelihood-training/) — Sean Welleck, Ilia Kulikov, Stephen Roller, et al. · 2019 · 498 citations
- [Texygen: A Benchmarking Platform for Text Generation Models](https://md.research.elamur.ai/free/en/papers/zhu-2018-texygen-benchmarking-platform-text-generation/) — Yaoming Zhu, Sidi Lu, Lei Zheng, et al. · 2018 · 486 citations
- [ToTTo: A Controlled Table-To-Text Generation Dataset](https://md.research.elamur.ai/free/en/papers/parikh-2020-totto-controlled-table-text-generation/) — Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, et al. · 2020 · 479 citations
- [Monolingual Machine Translation for Paraphrase Generation](https://md.research.elamur.ai/free/en/papers/quirk-2004-monolingual-machine-translation-paraphrase-generation/) — Chris Quirk, Chris Brockett, William Dolan · 2004 · 452 citations
- [Computational Generation of Referring Expressions: A Survey](https://md.research.elamur.ai/free/en/papers/krahmer-2012-computational-generation-referring-expressions-survey/) — Emiel Krahmer, Kees van Deemter · 2012 · 430 citations
- [Controlling Linguistic Style Aspects in Neural Language Generation](https://md.research.elamur.ai/free/en/papers/ficler-2017-controlling-linguistic-style-aspects-neural/) — Jessica Ficler, Yoav Goldberg · 2017 · 424 citations
- [Diffusion-LM Improves Controllable Text Generation](https://md.research.elamur.ai/free/en/papers/li-2022-diffusion-lm-improves-controllable-text/) — Xiang Lisa Li, John Thickstun, Ishaan Gulrajani, et al. · 2022 · 421 citations
- [DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts](https://md.research.elamur.ai/free/en/papers/liu-2021-dexperts-decoding-time-controlled-text/) — Alisa Liu, Maarten Sap, Ximing Lu, et al. · 2021 · 399 citations
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://md.research.elamur.ai/free/en/papers/krishna-2020-reformulating-unsupervised-style-transfer-as/) — Kalpesh Krishna, John Wieting, Mohit Iyyer · 2020 · 348 citations
- [Neural Paraphrase Generation with Stacked Residual LSTM Networks](https://md.research.elamur.ai/free/en/papers/prakash-2016-neural-paraphrase-generation-stacked-residual/) — Aaditya Prakash, Sadid A. Hasan, Kathy Lee, et al. · 2016 · 335 citations
- [A Graph-to-Sequence Model for AMR-to-Text Generation](https://md.research.elamur.ai/free/en/papers/song-2018-graph-sequence-model-amr-text/) — Linfeng Song, Yue Zhang, Zhiguo Wang, et al. · 2018 · 329 citations
- [Aspect Sentiment Quad Prediction as Paraphrase Generation](https://md.research.elamur.ai/free/en/papers/zhang-2021-aspect-sentiment-quad-prediction-as/) — Wenxuan Zhang, Yang Deng, Xin Li, et al. · 2021 · 325 citations
- [Globally Coherent Text Generation with Neural Checklist Models](https://md.research.elamur.ai/free/en/papers/kiddon-2016-globally-coherent-text-generation-neural/) — Chloé Kiddon, Luke Zettlemoyer, Yejin Choi · 2016 · 301 citations

**1** [2](https://md.research.elamur.ai/free/en/topics/text-generation/page/2/) … [5](https://md.research.elamur.ai/free/en/topics/text-generation/page/5/) [next →](https://md.research.elamur.ai/free/en/topics/text-generation/page/2/)

- [← previous topic Syntax & grammar formalisms](https://md.research.elamur.ai/free/en/topics/syntax-grammar/)
- [next topic → Tokenization & segmentation](https://md.research.elamur.ai/free/en/topics/tokenization/)
- [↑ back to topics](https://md.research.elamur.ai/free/en/topics/)