---
title: "Controllable Natural Language Generation with Contrastive"
description: "The authors build on prefix-tuning, where small learned vectors are prepended to the model's… 22 references, cited by 56 papers here."
authors: ["Jing Qian", "Li Dong", "Yelong Shen", "Furu Wei", "Weizhu Chen"]
year: 2022
venue: "Findings of the Association for Computational Linguistics: ACL 2022"
source: "https://aclanthology.org/2022.findings-acl.229/"
doi: "10.18653/v1/2022.findings-acl.229"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/qian-2022-controllable-natural-language-generation-contrastive/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/qian-2022-controllable-natural-language-generation-contrastive/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/3/) / Controllable Natural Language Generation with Contrastive Prefixes

# Controllable Natural Language Generation with Contrastive Prefixes

**Authors:** [Jing Qian](https://md.research.elamur.ai/free/en/authors/jing-qian/), [Li Dong](https://md.research.elamur.ai/free/en/authors/li-dong/), [Yelong Shen](https://md.research.elamur.ai/free/en/authors/yelong-shen/), [Furu Wei](https://md.research.elamur.ai/free/en/authors/furu-wei/) and [Weizhu Chen](https://md.research.elamur.ai/free/en/authors/weizhu-chen/) · **Year:** 2022 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2022](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2022](https://md.research.elamur.ai/free/en/venues/findings/2022/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.findings-acl.229/) · [PDF](https://aclanthology.org/2022.findings-acl.229.pdf) · [DOI 10.18653/v1/2022.findings-acl.229](https://doi.org/10.18653/v1/2022.findings-acl.229) · [OpenAlex](https://openalex.org/W4221144473)

**Citations:** 72 citations (OpenAlex, 2026-10-14) · [56](https://md.research.elamur.ai/free/en/papers/qian-2022-controllable-natural-language-generation-contrastive/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper addresses steering a frozen GPT2 toward chosen text attributes, such as sentiment, topic or non-toxicity, without costly fine-tuning or slow decoding. Existing approaches have drawbacks. CTRL requires training a huge conditional model, GeDi adds a 345M-parameter discriminator, and PPLM's gradient-based decoding is slow. The authors build on prefix-tuning, where small learned vectors are prepended to the model's activations. Instead of training each attribute's prefix in isolation, they train all prefixes of an aspect jointly so that each one learns both what to produce and what to avoid.

- **Supervised variant:** adds a discriminative loss to the usual LM objective.
- **Unsupervised variant:** treats the prefix index as a latent variable, inspired by VQ-VAE.
- **Semi-supervised variant:** combines the two to handle several aspects at once.

The method was tested on sentiment control, detoxification, topic control, and joint sentiment+topic control. It steered generation effectively while preserving fluency, even with only a few dozen labelled examples. It adds only about 0.2–2% extra parameters and keeps inference speed close to plain GPT2, making controllable generation cheap and modular.

## Key points
- Prefixes form a tensor of size N × M × D, with one prefix per attribute. They are reparametrized through a larger matrix during training, and only the final prefixes need to be stored because GPT2 stays frozen.
- The supervised loss combines L_LM with a discriminative term L_d. This term raises the likelihood of text under its correct prefix and lowers it under the competing prefixes, assuming a uniform prior.
- The unsupervised encoder is GPT-2 plus a linear layer. It picks a prefix by distance to the prefixes, using Gumbel-Softmax relaxation to keep the selection differentiable.
- Multi-aspect experiments used Amazon review sentiment data (Zhang et al., 2015) and DBPedia topics, with 20 prompts × 45 completions:
  - Simply concatenating single-aspect prefixes already worked, and prefix order did not matter.
  - Semi-supervised training with the extra L_enc loss improved attribute alignment further.
- Ablations showed that dropping L_d clearly hurts attribute relevance, especially for topic. Dropping L_enc can raise relevance but worsens perplexity. GeDi's fluency suffered with the short PPLM-style prompts used in evaluation.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 240 works · 123,537 citations
- [University of California, Santa Barbara](https://md.research.elamur.ai/free/en/universities/university-of-california-santa-barbara/) — University, Santa Barbara, United States · 35 works · 9,287 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [✍️ Text generation](https://md.research.elamur.ai/free/en/topics/text-generation/) · 145 works

## Cite

Cite as: Jing Qian, Li Dong, Yelong Shen, Furu Wei and Weizhu Chen. 2022. Controllable Natural Language Generation with Contrastive Prefixes. Findings of the Association for Computational Linguistics: ACL 2022.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{qian2022controllable,
  title = {{Controllable Natural Language Generation with Contrastive Prefixes}},
  author = {Jing Qian, Li Dong, Yelong Shen, Furu Wei and Weizhu Chen},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2022},
  year = {2022},
  doi = {10.18653/v1/2022.findings-acl.229},
  url = {https://doi.org/10.18653/v1/2022.findings-acl.229}
}
```

- [← previous The Text Anonymization Benchmark (TAB): A Dedicated Corpus and Evaluation Framework for Text Anonymization](https://md.research.elamur.ai/free/en/papers/pilan-2022-text-anonymization-benchmark-tab-dedicated/)
- [next → InternVideo: General Video Foundation Models via Generative and Discriminative Learning](https://md.research.elamur.ai/free/en/papers/wang-2022-internvideo-general-video-foundation/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/3/)

## References

22 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Sumanth Dathathri, Andrea Madotto, Janice Lan, Jane Hung, Eric Frank, Piero Molino, Jason Yosinski, and Rosanne Liu. 2020. [Plug and play language models: A simple approach to controlled text generation](https://md.research.elamur.ai/free/en/papers/dathathri-2019-plug-play-language-models-simple/). In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, Ap…
2. Jessica Ficler and Yoav Goldberg. 2017. [Controlling linguistic style aspects in neural language generation](https://md.research.elamur.ai/free/en/papers/ficler-2017-controlling-linguistic-style-aspects-neural/). CoRR, abs/1707.02633.
3. Samuel Gehman, Suchin Gururangan, Maarten Sap, Yejin Choi, and Noah A. Smith. 2020. [Realtoxicityprompts: Evaluating neural toxic degeneration in language models](https://md.research.elamur.ai/free/en/papers/gehman-2020-realtoxicityprompts-evaluating-neural-toxic-degeneration/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: Findings, EMNLP 2020, Online Event, 16-20 Novem…
4. Ari Holtzman, Jan Buys, Maxwell Forbes, Antoine Bosselut, David Golub, and Yejin Choi. 2018. [Learning to write with cooperative discriminators](https://md.research.elamur.ai/free/en/papers/holtzman-2018-learning-write-cooperative-discriminators/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, ACL 2018, Melbourne, Australia, July 15-20, 2018, Volume 1: Long…
5. Zhiting Hu, Zichao Yang, Xiaodan Liang, Ruslan Salakhutdinov, and Eric P. Xing. 2017. [Toward controlled generation of text](https://md.research.elamur.ai/free/en/papers/hu-2017-toward-controlled-generation-text/). In Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6-11 August 2017, volume 70 of Proceedings of Machine Learning Resea…
6. Eric Jang, Shixiang Gu, and Ben Poole. 2017. [Categorical reparameterization with gumbel-softmax](https://md.research.elamur.ai/free/en/papers/jang-2016-categorical-reparameterization-gumbel-softmax/). In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
7. Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, Caiming Xiong, and Richard Socher. 2019. [CTRL: A conditional transformer language model for controllable generation](https://md.research.elamur.ai/free/en/papers/keskar-2019-ctrl-conditional-transformer-language-model/). CoRR, abs/1909.05858.
8. Ben Krause, Akhilesh Deepak Gotmare, Bryan Mc-Cann, Nitish Shirish Keskar, Shafiq R. Joty, Richard Socher, and Nazneen Fatema Rajani. 2020. [Gedi: Generative discriminator guided sequence generation](https://md.research.elamur.ai/free/en/papers/krause-2021-gedi-generative-discriminator-guided-sequence/). CoRR, abs/2009.06367.
9. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Pa… [find](https://scholar.google.com/scholar?q=Xiang+Lisa+Li+and+Percy+Liang%2E+2021%2E+Prefix%2Dtuning%3A+Optimizing+continuous+prompts+for+generation%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+11t)
10. Zhiyu Lin and Mark Riedl. 2021. Plug-and-blend: A framework for controllable story generation with blended control codes. CoRR, abs/2104.04039. [find](https://scholar.google.com/scholar?q=Zhiyu+Lin+and+Mark+Riedl%2E+2021%2E+Plug%2Dand%2Dblend%3A+A+framework+for+controllable+story+generation+with+blended+control+codes%2E+CoRR%2C+abs%2F2104%2E04039%2E)
11. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E)
12. Roberta: A robustly optimized BERT pretraining approach. CoRR, abs/1907.11692. [find](https://scholar.google.com/scholar?q=Roberta%3A+A+robustly+optimized+BERT+pretraining+approach%2E+CoRR%2C+abs%2F1907%2E11692%2E)
13. Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. [Learning word vectors for sentiment analysis](https://md.research.elamur.ai/free/en/papers/maas-2011-learning-word-vectors-sentiment-analysis/). In The 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, Proceedings of the Conference, 19-24 June, 201…
14. Xiang Zhang, Junbo Jake Zhao, and Yann LeCun. 2015. [Character-level convolutional networks for text classification](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/). In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pages 649– 657.
15. Chris J. Maddison, Andriy Mnih, and Yee Whye Teh. 2017. [The concrete distribution: A continuous relaxation of discrete random variables](https://md.research.elamur.ai/free/en/papers/maddison-2016-concrete-distribution-continuous-relaxation-discrete/). In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net.
16. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+et+al%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2E)
17. Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, Scott Gray, Chelsea Voss, Alec Radford, Mark Chen, and Ilya Sutskever. 2021. [Zero-shot text-to-image generation](https://md.research.elamur.ai/free/en/papers/ramesh-2021-zero-shot-text-image-generation/). In Proceedings of the 38th International Conference on Machine Learning, ICML 2021, 18-24 July 2021, Virtual Event, volume 139 of Proceedings o…
18. Casper Kaae Sønderby, Ben Poole, and Andriy Mnih. 2017. Continuous relaxation training of discrete latent variable image models. In Beysian DeepLearning workshop, NIPS, volume 201. [find](https://scholar.google.com/scholar?q=Casper+Kaae+S%C3%B8nderby%2C+Ben+Poole%2C+and+Andriy+Mnih%2E+2017%2E+Continuous+relaxation+training+of+discrete+latent+variable+image+models%2E+In+Beysian+DeepLearning+workshop%2C+NIPS%2C+volume+201%2E)
19. Aäron van den Oord, Oriol Vinyals, and Koray Kavukcuoglu. 2017. Neural discrete representation learning. In Advances in Neural Information Processing Systems 30: Annual Conference on Neural Information Processing Systems 2017, December 4-9, 2017, Long Beach, CA, USA, pages 6306–6315. [find](https://scholar.google.com/scholar?q=A%C3%A4ron+van+den+Oord%2C+Oriol+Vinyals%2C+and+Koray+Kavukcuoglu%2E+2017%2E+Neural+discrete+representation+learning%2E+In+Advances+in+Neural+Information+Processing+Systems+30%3A+Annual+Conference+on+Neural+Informatio)
20. Johannes Welbl, Amelia Glaese, Jonathan Uesato, Sumanth Dathathri, John Mellor, Lisa Anne Hendricks, Kirsty Anderson, Pushmeet Kohli, Ben Coppin, and Po-Sen Huang. 2021. [Challenges in detoxifying language models](https://md.research.elamur.ai/free/en/papers/welbl-2021-challenges-detoxifying-language-models/). In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2447–24…
21. Kevin Yang and Dan Klein. 2021. [FUDGE: controlled text generation with future discriminators](https://md.research.elamur.ai/free/en/papers/yang-2021-fudge-controlled-text-generation-future/). In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2021, Online, June 6-11, 2021, pages 3511–3535. A…
22. Dian Yu, Kenji Sagae, and Zhou Yu. 2021. Attribute alignment: Controlling text generation from pre-trained language models. CoRR, abs/2103.11070. [find](https://scholar.google.com/scholar?q=Dian+Yu%2C+Kenji+Sagae%2C+and+Zhou+Yu%2E+2021%2E+Attribute+alignment%3A+Controlling+text+generation+from+pre%2Dtrained+language+models%2E+CoRR%2C+abs%2F2103%2E11070%2E)

*Updated 24 October 2026*