---
title: "Extreme Adaptation for Personalized Neural Machine Translation"
description: "Standard NMT systems use one model for everyone, but people differ in vocabulary… 26.87 BLEU. 26 references, cited by 58 papers here."
authors: ["Paul Michel", "Graham Neubig"]
year: 2018
venue: "Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)"
source: "https://aclanthology.org/P18-2050/"
doi: "10.18653/v1/p18-2050"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/michel-2018-extreme-adaptation-personalized-neural-machine/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/michel-2018-extreme-adaptation-personalized-neural-machine/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/) / Extreme Adaptation for Personalized Neural Machine Translation

# Extreme Adaptation for Personalized Neural Machine Translation

**Authors:** [Paul Michel](https://md.research.elamur.ai/free/en/authors/paul-michel/) and [Graham Neubig](https://md.research.elamur.ai/free/en/authors/graham-neubig/) · **Year:** 2018 · **Venue:** [Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2018](https://md.research.elamur.ai/free/en/venues/acl/2018/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P18-2050/) · [PDF](https://aclanthology.org/P18-2050.pdf) · [DOI 10.18653/v1/p18-2050](https://doi.org/10.18653/v1/p18-2050) · [OpenAlex](https://openalex.org/W2798878995)

**Citations:** 100 citations (OpenAlex, 2026-10-14) · [58](https://md.research.elamur.ai/free/en/papers/michel-2018-extreme-adaptation-personalized-neural-machine/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Standard NMT systems use one model for everyone, but people differ in vocabulary, topics, gender, and other traits that affect how a sentence should be translated. Michel and Neubig treat personalization as an extreme domain adaptation problem: there are about 1,800 "domains" (speakers), each with very little data. Retraining a full model for every speaker would cost too much memory and would overfit. Instead, they adapt only the bias of the output softmax for each speaker. The *full bias* variant learns a vocabulary-sized vector per speaker. The *fact bias* variant represents each speaker as a weighted mix of a few shared prototype bias vectors. To test this, they built SATED, a speaker-annotated corpus of TED talks for en-fr, en-de and en-es. Both variants give small but significant BLEU gains over a speaker-agnostic LSTM baseline. A speaker classifier can identify the author more easily from the adapted outputs than from the baseline. This suggests that the methods keep more of each person's style. Because only a handful of parameters are needed per user, the approach could scale to very large user bases.

## Key points
- **SATED dataset:** built from ted.com transcripts, with roughly 271K sentences per language across 2,324 talks.
  - Talks with fewer than 10 translated sentences were dropped.
  - The dev and test sets each take 2 sentence pairs from every talk.
  - It is released under CC BY-NC-ND 4.0.
- **Parameter cost:**
  - A full per-speaker model would have about 47M parameters, roughly 188MB per speaker.
  - The full bias vector is only about 0.09% of the model.
  - fact bias (r = 10) uses 99.38–99.45% fewer adaptation parameters than full bias, leaving just 10 parameters per speaker.
- **Test BLEU on SATED (baseline → best):**
  - en-fr: 38.05 → 39.01 (fact bias)
  - en-de: 26.46 → 27.20 (full bias)
  - en-es: 39.89 → 40.30 (full bias)
  - A speaker-token baseline (spk token) was also compared.
- **Control experiment on Europarl:** this checks that the gains are not just topic effects. On a speaker-annotated Europarl en-de corpus (747 speakers, about 300k sentences), fact bias reached 26.87 BLEU versus 26.04 for the baseline.
- **Implementation:** a 1-layer LSTM encoder-decoder with attention, built in DyNet, with a 40k-word vocabulary. Code and data are publicly released.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) — University, Pittsburgh, United States · 365 works · 158,048 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🔁 Machine translation](https://md.research.elamur.ai/free/en/topics/machine-translation/) · 958 works

## Cite

Cite as: Paul Michel and Graham Neubig. 2018. Extreme Adaptation for Personalized Neural Machine Translation. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{michel2018extreme,
  title = {{Extreme Adaptation for Personalized Neural Machine Translation}},
  author = {Paul Michel and Graham Neubig},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-2050},
  url = {https://doi.org/10.18653/v1/p18-2050}
}
```

- [← previous Diachronic Usage Relatedness (DURel): A Framework for the Annotation of Lexical Semantic Change](https://md.research.elamur.ai/free/en/papers/schlechtweg-2018-diachronic-usage-relatedness-durel-framework/)
- [next → Rearranging the Familiar: Testing Compositional Generalization in Recurrent Networks](https://md.research.elamur.ai/free/en/papers/loula-2018-rearranging-familiar-testing-compositional-generalization/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/)

## References

26 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. International Conference on Computational Linguistics (Coling 2010). [find](https://scholar.google.com/scholar?q=International+Conference+on+Computational+Linguistics+%28Coling+2010%29%2E)
2. Coling 2010 Organizing Committee, Beijing, China, pages 662–670. [http://www.aclweb.org/anthology/C10-1075](http://www.aclweb.org/anthology/C10-1075). [↗](http://www.aclweb.org/anthology/C10-1075)
3. Boxing Chen, Colin Cherry, George Foster, and Samuel Larkin. 2017. Cost weighting for neural machine translation domain adaptation. In Proceedings of the First Workshop on Neural Machine Translation. Association for Computational Linguistics, Vancouver, pages 40–46. [http://www.aclweb.org/anthology/W…](http://www.aclweb.org/anthology/W…) [↗](http://www.aclweb.org/anthology/W…)
4. Xiaoqing Li, Jiajun Zhang, and Chengqing Zong. 2016. One sentence one model for neural machine translation. arXiv preprint arXiv:1609.06490 . [find](https://scholar.google.com/scholar?q=Xiaoqing+Li%2C+Jiajun+Zhang%2C+and+Chengqing+Zong%2E+2016%2E+One+sentence+one+model+for+neural+machine+translation%2E+arXiv+preprint+arXiv%3A1609%2E06490+%2E)
5. Minh-Thang Luong and Christopher D Manning. 2015. Stanford neural machine translation systems for spoken language domains. In Proceedings of the International Workshop on Spoken Language Translation. [find](https://scholar.google.com/scholar?q=Minh%2DThang+Luong+and+Christopher+D+Manning%2E+2015%2E+Stanford+neural+machine+translation+systems+for+spoken+language+domains%2E+In+Proceedings+of+the+International+Workshop+on+Spoken+Language+Translation%2E)
6. Chenhui Chu, Raj Dabre, and Sadao Kurohashi. 2017. [An empirical comparison of domain adaptation methods for neural machine translation](https://md.research.elamur.ai/free/en/papers/chu-2017-empirical-comparison-domain-adaptation-methods/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). Association for Computational Linguistics, Vanc…
7. Antonio Valerio Miceli Barone, Barry Haddow, Ulrich Germann, and Rico Sennrich. 2017. [Regularization techniques for fine-tuning in neural machine translation](https://md.research.elamur.ai/free/en/papers/barone-2017-regularization-techniques-fine-tuning-neural/). In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenha…
8. Michael Denkowski and Graham Neubig. 2017. [Stronger baselines for trustable results in neural machine translation](https://md.research.elamur.ai/free/en/papers/denkowski-2017-stronger-baselines-trustable-results-neural/). arXiv preprint arXiv:1706.09733 .
9. Hideki Mima, Osamu Furuse, and Hitoshi Iida. 1997. Improving performance of transfer-driven machine translation with extra-linguistic informatioon from context, situation and environment. In IJCAI (2). pages 983–989. [find](https://scholar.google.com/scholar?q=Hideki+Mima%2C+Osamu+Furuse%2C+and+Hitoshi+Iida%2E+1997%2E+Improving+performance+of+transfer%2Ddriven+machine+translation+with+extra%2Dlinguistic+informatioon+from+context%2C+situation+and+environment%2E+In+IJCAI+%282%29)
10. Yarin Gal and Zoubin Ghahramani. 2016. [A theoretically grounded application of dropout in recurrent neural networks](https://md.research.elamur.ai/free/en/papers/gal-2015-theoretically-grounded-application-dropout-recurrent/). In Advances in neural information processing systems. pages 1019–1027.
11. Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III. 2015. [Deep unordered composition rivals syntactic methods for text classification](https://md.research.elamur.ai/free/en/papers/iyyer-2015-deep-unordered-composition-rivals-syntactic/). In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural L…
12. Shachar Mirkin, Scott Nowson, Caroline Brun, and Julien Perez. 2015. Motivating personality-aware machine translation. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Lisbon, Portugal, pages 1102–1108. http://aclwe… [find](https://scholar.google.com/scholar?q=Shachar+Mirkin%2C+Scott+Nowson%2C+Caroline+Brun%2C+and+Julien+Perez%2E+2015%2E+Motivating+personality%2Daware+machine+translation%2E+In+Proceedings+of+the+2015+Conference+on+Empirical+Methods+in+Natural+Language+Pr)
13. Robert C. Moore and William Lewis. 2010. [Intelligent selection of language model training data](https://md.research.elamur.ai/free/en/papers/moore-2010-intelligent-selection-language-model-training/). In Proceedings of the ACL 2010 Conference Short Papers. Association for Computational Linguistics, Uppsala, Sweden, pages 220– 224. [http://www.aclweb.org/anthology/P10-2041](http://www.aclweb.org/anthology/P10-2041). [↗](http://www.aclweb.org/anthology/P10-2041)
14. Diederik P. Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. In ICLR. [find](https://scholar.google.com/scholar?q=Diederik+P%2E+Kingma+and+Jimmy+Ba%2E+2014%2E+Adam%3A+A+method+for+stochastic+optimization%2E+In+ICLR%2E)
15. Philipp Koehn. 2004. [Statistical significance tests for machine translation evaluation](https://md.research.elamur.ai/free/en/papers/koehn-2004-statistical-significance-tests-machine-translation/). In Dekang Lin and Dekai Wu, editors, Proceedings of EMNLP 2004. Association for Computational Linguistics, Barcelona, Spain, pages 388–395.
16. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondřej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Pro… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+Chris+Dyer%2C+Ondr%CC%8Cej+Bojar%2C+Alexandra+Const)
17. Mu Li, Yinggong Zhao, Dongdong Zhang, and Ming Zhou. 2010. [Adaptive development data selection for log-linear model in statistical machine translation](https://md.research.elamur.ai/free/en/papers/neubig-2017-dynet-dynamic-neural-network-toolkit/). In Proceedings of the 23rd Graham Neubig, Chris Dyer, Yoav Goldberg, Austin Matthews, Waleed Ammar, Antonios Anastasopoulos, Miguel Ballesteros, Dav…
18. Ofir Press and Lior Wolf. 2017. [Using the output embedding to improve language models](https://md.research.elamur.ai/free/en/papers/press-2017-output-embedding-improve-language-models/). In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers. Association for Computational Linguistics, Valencia, Spain, pages 157–163. ht…
19. Ella Rabinovich, Raj Nath Patel, Shachar Mirkin, Lucia Specia, and Shuly Wintner. 2017. [Personalized machine translation: Preserving original author traits](https://md.research.elamur.ai/free/en/papers/rabinovich-2017-personalized-machine-translation-preserving-original/). In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 1, Long Papers. Associat…
20. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016a. [Controlling politeness in neural machine translation via side constraints](https://md.research.elamur.ai/free/en/papers/sennrich-2016-controlling-politeness-neural-machine-translation/). In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Compu…
21. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016b. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers). Association for Computational Linguistics, Berlin, Germany, pages…
22. Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. Journal of machine learning research 15(1):1929–1958. [find](https://scholar.google.com/scholar?q=Nitish+Srivastava%2C+Geoffrey+E+Hinton%2C+Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Ruslan+Salakhutdinov%2E+2014%2E+Dropout%3A+a+simple+way+to+prevent+neural+networks+from+overfitting%2E+Journal+of+machine+learning+re)
23. Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. pages 2818–2826. [find](https://scholar.google.com/scholar?q=Christian+Szegedy%2C+Vincent+Vanhoucke%2C+Sergey+Ioffe%2C+Jon+Shlens%2C+and+Zbigniew+Wojna%2E+2016%2E+Rethinking+the+inception+architecture+for+computer+vision%2E+In+Proceedings+of+the+IEEE+Conference+on+Computer+V)
24. Marco Turchi, Matteo Negri, M Amin Farajian, and Marcello Federico. 2017. Continuous learning from human post-edits for neural machine translation. The Prague Bulletin of Mathematical Linguistics 108(1):233–244. [find](https://scholar.google.com/scholar?q=Marco+Turchi%2C+Matteo+Negri%2C+M+Amin+Farajian%2C+and+Marcello+Federico%2E+2017%2E+Continuous+learning+from+human+post%2Dedits+for+neural+machine+translation%2E+The+Prague+Bulletin+of+Mathematical+Linguistics+108%28)
25. Rui Wang, Masao Utiyama, Lemao Liu, Kehai Chen, and Eiichiro Sumita. 2017. [Instance weighting for neural machine translation domain adaptation](https://md.research.elamur.ai/free/en/papers/wang-2017-instance-weighting-neural-machine-translation/). In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Copenhagen, Denmark, p…
26. Barret Zoph, Deniz Yuret, Jonathan May, and Kevin Knight. 2016. Transfer learning for low-resource neural machine translation. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, Austin, Texas, pages 1568–1575. https:/… [find](https://scholar.google.com/scholar?q=Barret+Zoph%2C+Deniz+Yuret%2C+Jonathan+May%2C+and+Kevin+Knight%2E+2016%2E+Transfer+learning+for+low%2Dresource+neural+machine+translation%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Lan)

*Updated 24 October 2026*