---
title: "Style Transfer as Unsupervised Machine — Zhang 2018"
description: "Text style transfer means rewriting a sentence so that it carries a different attribute, such as positive instead of negative sentiment, while what it…"
authors: ["Zhirui Zhang", "Shuo Ren", "Shujie Liu", "Jianyong Wang", "Peng Chen", "Mu Li", "Ming Zhou", "Enhong Chen"]
year: 2018
venue: "arXiv:1808.07894"
source: "https://arxiv.org/abs/1808.07894"
doi: "10.48550/arxiv.1808.07894"
arxiv: "1808.07894"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/zhang-2018-style-transfer-unsupervised-machine-translation/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/zhang-2018-style-transfer-unsupervised-machine-translation/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2018](https://md.research.elamur.ai/free/papers/2018/page/3/) / Style Transfer as Unsupervised Machine Translation

# Style Transfer as Unsupervised Machine Translation

**Authors:** [Zhirui Zhang](https://md.research.elamur.ai/free/authors/zhirui-zhang/), [Shuo Ren](https://md.research.elamur.ai/free/authors/shuo-ren/), [Shujie Liu](https://md.research.elamur.ai/free/authors/shujie-liu/), Jianyong Wang, Peng Chen, [Mu Li](https://md.research.elamur.ai/free/authors/mu-li/), [Ming Zhou](https://md.research.elamur.ai/free/authors/ming-zhou/), [Enhong Chen](https://md.research.elamur.ai/free/authors/enhong-chen/) · **Year:** 2018 · **Venue:** arXiv:1808.07894 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1808.07894) · [PDF](https://arxiv.org/pdf/1808.07894) · [DOI 10.48550/arxiv.1808.07894](https://doi.org/10.48550/arxiv.1808.07894) · [arXiv 1808.07894](https://arxiv.org/abs/1808.07894) · [OpenAlex](https://openalex.org/W2888173624)

**Citations:** 99 citations (openalex-snapshot · 2026-10-14) · [65](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Text style transfer means rewriting a sentence so that it carries a different attribute, such as positive instead of negative sentiment, while what it says stays the same. It is hard to train because sentence pairs with identical content but different styles rarely exist. Zhang et al. treat the two styles as two "languages" and adapt the unsupervised machine translation recipe to this setting.

The method has two stages:
- **Initialization.** A word-to-word transfer table is induced without supervision. It combines embedding similarity with each word's statistical leaning toward one style. Together with style-specific language models, the table powers two SMT systems, one per direction, that generate pseudo-parallel pairs.
- **Refinement.** Attention-based GRU NMT models are pre-trained on these pairs and then improved jointly through iterative back-translation. A style classifier filters and penalizes generated pairs that have the wrong style.

The approach was tested on Yelp and Amazon sentiment transfer and on romantic/humorous Captions. It outperforms earlier state-of-the-art systems in both human and automatic evaluation, on style accuracy and on content preservation. The work shows that unsupervised MT can be reused for same-language rewriting once two key components are modified.

## Key points
- **Word transfer probability.** The score factorizes into three terms: P(s|x_w), the embedding-based similarity P(y_w|x_w), and P(t|y_w).
  - The style terms are relative frequency ratios of a word across the two style corpora.
  - The similarity term is normalized cosine similarity of word2vec-style (Mikolov et al. 2013) embeddings.
- **Model architecture.** The models follow the Bahdanau et al. attention encoder-decoder with GRU units, trained in both directions (P_{s→t}, P_{t→s}).
- **Ablation results (Table 5, style accuracy / BLEU).**

  | System | Yelp | Amazon | Captions |
  |---|---|---|---|
  | SMT only | 94.6% / 8.82 | 81.2% / 7.46 | 82.8% / 5.04 |
  | NMT, iteration 0 | 70.5% / 15.81 | 68.4% / 16.36 | 66.5% / 8.72 |
  | Full system | 96.6% / 22.79 | 84.1% / 33.90 | 99.5% / 12.69 |

  - SMT reaches high style accuracy but preserves content poorly.
  - NMT at iteration 0 improves content, and iterative back-translation then raises both metrics.
- **Effect of the style classifier.** Without it, accuracy falls to 80.4% (Yelp), 75.6% (Amazon) and 79.3% (Captions). BLEU rises slightly at the same time, which points to a trade-off that the classifier controls.
- **Differences from unsupervised NMT.** Two components are changed:
  - a monolingual word mapping replaces cross-lingual embedding alignment;
  - classifier-based filtering and reward stabilize back-translation.

## Interesting ideas
- **Same-language word mapping.** Two words that fill the same grammatical slot (high embedding similarity) but lean toward opposite corpora (high style preference) make good substitution candidates, e.g., "hate" and "love". This cheap trick could seed lexicons for other paired-corpus contrasts, such as formal vs. informal register or dialect vs. standard variety.
- **Bootstrapping neural models with SMT.** Phrase-based SMT is used only to produce rough pseudo-parallel data, and the neural models then take over. The paper's numbers show NMT smoothing SMT's content errors, so this staging could help any low-resource rewriting task that lacks paired data.
- **Classifier as a noise guard.** An external attribute classifier scores back-translated pairs and penalizes those with the wrong style. This keeps iterative self-training from drifting, and the pattern generalizes to any self-training loop with a checkable target property.
- **Ablating across both metrics.** Reporting style accuracy and BLEU side by side for each component exposes the trade-off between them. For example, removing the classifier slightly raises BLEU while sharply cutting accuracy, which a single metric would hide.
- **Paired directions that train each other.** Building both directions together lets each model supply training data for the other through back-translation. This suits any task with two symmetric unpaired corpora.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Machine translation](https://md.research.elamur.ai/free/shelves/machine-translation/)

Topics: [Machine translation](https://md.research.elamur.ai/free/topics/machine-translation/)

## Cited by

Cited by

65 papers of our library cite it (of the 140673 whose bibliographies we read); 5 of them have a page here, 60 not yet.

- [Style Transformer: Unpaired Text Style Transfer without Disentangled Latent Representation](https://md.research.elamur.ai/free/papers/dai-2019-style-transformer-unpaired-text-style/)
- [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://md.research.elamur.ai/free/papers/krishna-2020-reformulating-unsupervised-style-transfer-as/)
- [A Dual Reinforcement Learning Framework for Unsupervised Text Style Transfer](https://md.research.elamur.ai/free/papers/luo-2019-dual-reinforcement-learning-framework-unsupervised/)
- [A Modular Architecture for Unsupervised Sarcasm Generation](https://md.research.elamur.ai/free/papers/mishra-2019-modular-architecture-unsupervised-sarcasm-generation/)
- [“Transforming” Delete, Retrieve, Generate Approach for Controlled Text Style Transfer](https://md.research.elamur.ai/free/papers/sudhakar-2019-transforming-delete-retrieve-generate-approach/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{zhang2018style,
  title = {{Style Transfer as Unsupervised Machine Translation}},
  author = {Zhang, Zhirui and Ren, Shuo and Liu, Shujie and Wang, Jianyong and Chen, Peng and Li, Mu and Zhou, Ming and Chen, Enhong},
  year = {2018},
  doi = {10.48550/arxiv.1808.07894},
  eprint = {1808.07894},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1808.07894}
}
```

- [← previous Stop Explaining Black Box Machine Learning Models for High Stakes Decisions and Use Interpretable Models Instead](https://md.research.elamur.ai/free/papers/rudin-2018-stop-explaining-black-box/)
- [next → Style Transfer Through Back-Translation](https://md.research.elamur.ai/free/papers/prabhumoye-2018-style-transfer-through-back-translation/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/papers/2018/page/3/)

## References

29 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Artetxe, M.; Labaka, G.; Agirre, E.; and Cho, K. 2018. Unsupervised neural machine translation. In ICLR. [↗](https://doi.org/10.48550/arxiv.1710.11041) [our card](https://md.research.elamur.ai/free/papers/artetxe-2017-unsupervised-neural-machine-translation/)
2. Artetxe, M.; Labaka, G.; and Agirre, E. 2017. Learning bilingual word embeddings with (almost) no bilingual data. In ACL. [find](https://scholar.google.com/scholar?q=Artetxe%2C+M%2E%3B+Labaka%2C+G%2E%3B+and+Agirre%2C+E%2E+2017%2E+Learning+bilingual+word+embeddings+with+%28almost%29+no+bilingual+data%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/papers/artetxe-2017-learning-bilingual-word-embeddings-almost/)
3. Bahdanau, D.; Cho, K.; and Bengio, Y. 2014. Neural machine translation by jointly learning to align and translate. CoRR abs/1409.0473. [↗](https://arxiv.org/abs/1409.0473) [our card](https://md.research.elamur.ai/free/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
4. Chiang, D. 2007. Hierarchical phrase-based translation. Computational Linguistics. Cho, K.; van Merrienboer, B.; aglar Gülehre; Bahdanau, D.; Bougares, F.; Schwenk, H.; and Bengio, Y. 2014. Learning phrase representations using rnn encoder-decoder for statistical machine translation. In EMNLP. [↗](https://doi.org/10.1162/coli.2007.33.2.201) [our card](https://md.research.elamur.ai/free/papers/chiang-2007-hierarchical-phrase-based-translation/)
5. Conneau, A.; Lample, G.; Ranzato, M.; Denoyer, L.; and Jégou, H. 2017. Word translation without parallel data. CoRR abs/1710.04087. [↗](https://arxiv.org/abs/1710.04087)
6. Fu, Z.; Tan, X.; Peng, N.; Zhao, D.; and Yan, R. 2018. Style transfer in text: Exploration and evaluation. In AAAI. [find](https://scholar.google.com/scholar?q=Fu%2C+Z%2E%3B+Tan%2C+X%2E%3B+Peng%2C+N%2E%3B+Zhao%2C+D%2E%3B+and+Yan%2C+R%2E+2018%2E+Style+transfer+in+text%3A+Exploration+and+evaluation%2E+In+AAAI%2E) [our card](https://md.research.elamur.ai/free/papers/fu-2017-style-transfer-text-exploration-evaluation/)
7. Gan, C.; Gan, Z.; He, X.; Gao, J.; and Deng, L. 2017. Stylenet: Generating attractive visual captions with styles. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 955–964. [find](https://scholar.google.com/scholar?q=Gan%2C+C%2E%3B+Gan%2C+Z%2E%3B+He%2C+X%2E%3B+Gao%2C+J%2E%3B+and+Deng%2C+L%2E+2017%2E+Stylenet%3A+Generating+attractive+visual+captions+with+styles%2E+2017+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVPR%29+955%E2%80%93964%2E)
8. Glorot, X., and Bengio, Y. 2010. Understanding the difficulty of training deep feedforward neural networks. In AIS-TATS. [find](https://scholar.google.com/scholar?q=Glorot%2C+X%2E%2C+and+Bengio%2C+Y%2E+2010%2E+Understanding+the+difficulty+of+training+deep+feedforward+neural+networks%2E+In+AIS%2DTATS%2E)
9. He, R., and McAuley, J. 2016. Ups and downs: Modeling the visual evolution of fashion trends with one-class collaborative filtering. In WWW. [↗](https://doi.org/10.48550/arxiv.1602.01585)
10. Hochreiter, S., and Schmidhuber, J. 1997. Long short-term memory. Neural computation 9 8:1735–80. [↗](https://doi.org/10.1162/neco.1997.9.8.1735)
11. Hu, Z.; Yang, Z.; Liang, X.; Salakhutdinov, R.; and Xing, E. P. 2017. Toward controlled generation of text. In ICML. [↗](https://doi.org/10.48550/arxiv.1703.00955) [our card](https://md.research.elamur.ai/free/papers/hu-2017-toward-controlled-generation-text/)
12. Kim, Y., and Rush, A. M. 2016. Sequence-level knowledge distillation. In EMNLP. [find](https://scholar.google.com/scholar?q=Kim%2C+Y%2E%2C+and+Rush%2C+A%2E+M%2E+2016%2E+Sequence%2Dlevel+knowledge+distillation%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/papers/kim-2016-sequence-level-knowledge-distillation/)
13. Lample, G.; Ott, M.; Conneau, A.; Denoyer, L.; and Ranzato, M. 2018. Phrase-based & neural unsupervised machine translation. arXiv preprint arXiv:1804.07755. [↗](https://arxiv.org/abs/1804.07755)
14. Lample, G.; Denoyer, L.; and Ranzato, M. 2018. Unsupervised machine translation using monolingual corpora only. In ICLR. [↗](https://doi.org/10.48550/arxiv.1711.00043) [our card](https://md.research.elamur.ai/free/papers/lample-2017-unsupervised-machine-translation-using-monolingual/)
15. Li, J.; Galley, M.; Brockett, C.; Spithourakis, G. P.; Gao, J.; and Dolan, W. B. 2016. A persona-based neural conversation model. In ACL. [↗](https://doi.org/10.48550/arxiv.1603.06155) [our card](https://md.research.elamur.ai/free/papers/li-2016-persona-based-neural-conversation-model/)
16. Li, J.; Jia, R.; Hé, H.; and Liang, P. 2018. Delete, retrieve, generate: A simple approach to sentiment and style transfer. In NAACL. [↗](https://doi.org/10.48550/arxiv.1804.06437) [our card](https://md.research.elamur.ai/free/papers/li-2018-delete-retrieve-generate-simple-approach/)
17. Mikolov, T.; Sutskever, I.; Chen, K.; Corrado, G. S.; and Dean, J. 2013. Distributed representations of words and phrases and their compositionality. In NIPS. [↗](https://doi.org/10.48550/arxiv.1310.4546)
18. Och, F. J. 2003. Minimum error rate training in statistical machine translation. In ACL. [find](https://scholar.google.com/scholar?q=Och%2C+F%2E+J%2E+2003%2E+Minimum+error+rate+training+in+statistical+machine+translation%2E+In+ACL%2E)
19. Papineni, K.; Roukos, S.; Ward, T.; and Zhu, W.-J. 2002. Bleu: a method for automatic evaluation of machine translation. In ACL. [find](https://scholar.google.com/scholar?q=Papineni%2C+K%2E%3B+Roukos%2C+S%2E%3B+Ward%2C+T%2E%3B+and+Zhu%2C+W%2E%2DJ%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
20. Prabhumoye, S.; Tsvetkov, Y.; Salakhutdinov, R.; and Black, A. W. 2018. Style transfer through back-translation. In ACL. [find](https://scholar.google.com/scholar?q=Prabhumoye%2C+S%2E%3B+Tsvetkov%2C+Y%2E%3B+Salakhutdinov%2C+R%2E%3B+and+Black%2C+A%2E+W%2E+2018%2E+Style+transfer+through+back%2Dtranslation%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/papers/prabhumoye-2018-style-transfer-through-back-translation/)
21. Sennrich, R.; Haddow, B.; and Birch, A. 2016a. Controlling politeness in neural machine translation via side constraints. In HLT-NAACL. [find](https://scholar.google.com/scholar?q=Sennrich%2C+R%2E%3B+Haddow%2C+B%2E%3B+and+Birch%2C+A%2E+2016a%2E+Controlling+politeness+in+neural+machine+translation+via+side+constraints%2E+In+HLT%2DNAACL%2E) [our card](https://md.research.elamur.ai/free/papers/sennrich-2016-controlling-politeness-neural-machine-translation/)
22. Sennrich, R.; Haddow, B.; and Birch, A. 2016b. Improving neural machine translation models with monolingual data. In ACL. [find](https://scholar.google.com/scholar?q=Sennrich%2C+R%2E%3B+Haddow%2C+B%2E%3B+and+Birch%2C+A%2E+2016b%2E+Improving+neural+machine+translation+models+with+monolingual+data%2E+In+ACL%2E)
23. Shen, S.; Cheng, Y.; He, Z.; He, W.; Wu, H.; Sun, M.; and Liu, Y. 2016. Minimum risk training for neural machine translation. In ACL. [find](https://scholar.google.com/scholar?q=Shen%2C+S%2E%3B+Cheng%2C+Y%2E%3B+He%2C+Z%2E%3B+He%2C+W%2E%3B+Wu%2C+H%2E%3B+Sun%2C+M%2E%3B+and+Liu%2C+Y%2E+2016%2E+Minimum+risk+training+for+neural+machine+translation%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/papers/shen-2015-minimum-risk-training-neural-machine/)
24. Shen, T.; Lei, T.; Barzilay, R.; and Jaakkola, T. S. 2017. Style transfer from non-parallel text by cross-alignment. In NIPS. [↗](https://doi.org/10.48550/arxiv.1705.09655) [our card](https://md.research.elamur.ai/free/papers/shen-2017-style-transfer-non-parallel-text/)
25. Sutskever, I.; Vinyals, O.; and Le, Q. V. 2014. Sequence to sequence learning with neural networks. In NIPS. Wen, T.-H.; Gasic, M.; Mrksic, N.; hao Su, P.; Vandyke, D.; and Young, S. J. 2015. Semantically conditioned lstm-based natural language generation for spoken dialogue systems. In EMNLP. [↗](https://doi.org/10.48550/arxiv.1409.3215)
26. Wintner, S.; Mirkin, S.; Specia, L.; Rabinovich, E.; and Patel, R. N. 2017. Personalized machine translation: Preserving original author traits. In EACL. [find](https://scholar.google.com/scholar?q=Wintner%2C+S%2E%3B+Mirkin%2C+S%2E%3B+Specia%2C+L%2E%3B+Rabinovich%2C+E%2E%3B+and+Patel%2C+R%2E+N%2E+2017%2E+Personalized+machine+translation%3A+Preserving+original+author+traits%2E+In+EACL%2E)
27. Xu, J.; Sun, X.; Zeng, Q.; Ren, X.; Zhang, X.; Wang, H.; and Li, W. 2018. Unpaired sentiment-to-sentiment translation: A cycled reinforcement learning approach. In ACL. [↗](https://doi.org/10.60692/kzdbq-6en85) [our card](https://md.research.elamur.ai/free/papers/xu-2018-unpaired-sentiment-sentiment-translation-cycled/)
28. Zeiler, M. D. 2012. Adadelta: An adaptive learning rate method. CoRR abs/1212.5701. [↗](https://arxiv.org/abs/1212.5701) [our card](https://md.research.elamur.ai/free/papers/zeiler-2012-adadelta-adaptive-learning-rate-method/)
29. Zhang, Z.; Liu, S.; Li, M.; Zhou, M.; and Chen, E. 2018. Joint training for neural machine translation models with monolingual data. In AAAI. [find](https://scholar.google.com/scholar?q=Zhang%2C+Z%2E%3B+Liu%2C+S%2E%3B+Li%2C+M%2E%3B+Zhou%2C+M%2E%3B+and+Chen%2C+E%2E+2018%2E+Joint+training+for+neural+machine+translation+models+with+monolingual+data%2E+In+AAAI%2E)