---
title: "Polylingual Topic Models — paper by Mimno et al. (2009)"
description: "Mimno et al. propose the polylingual topic model (PLTM), an extension of LDA. 14 references, cited by 90 papers here."
authors: ["David Mimno", "Hanna M. Wallach", "Jason Naradowsky", "David A. Smith", "Andrew McCallum"]
year: 2009
venue: "EMNLP 2009"
source: "https://aclanthology.org/D09-1092/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mimno-2009-polylingual-topic-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mimno-2009-polylingual-topic-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2009](https://md.research.elamur.ai/free/en/papers/2009/) / Polylingual Topic Models

# Polylingual Topic Models

**Authors:** [David Mimno](https://md.research.elamur.ai/free/en/authors/david-mimno/), [Hanna M. Wallach](https://md.research.elamur.ai/free/en/authors/hanna-wallach/), [Jason Naradowsky](https://md.research.elamur.ai/free/en/authors/jason-naradowsky/), [David A. Smith](https://md.research.elamur.ai/free/en/authors/david-smith/) and [Andrew McCallum](https://md.research.elamur.ai/free/en/authors/andrew-mccallum/) · **Year:** 2009 · **Venue:** [EMNLP 2009](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2009](https://md.research.elamur.ai/free/en/venues/emnlp/2009/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/D09-1092/) · [PDF](https://aclanthology.org/D09-1092.pdf) · [OpenAlex](https://openalex.org/W2033593667)

**Citations:** 328 citations (OpenAlex, 2026-10-14) · [90](https://md.research.elamur.ai/free/en/papers/mimno-2009-polylingual-topic-models/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary

Topic models had so far been used mostly on single-language collections, or at most on language pairs. Existing bilingual variants were also built for machine translation and depended on sentence-aligned parallel text. Mimno et al. propose the polylingual topic model (PLTM), an extension of LDA. In PLTM, a "tuple" of loosely equivalent documents in different languages shares one topic distribution, while each topic has a separate word distribution for every language. Inference uses Gibbs sampling, and the hyperparameters are re-estimated during training.

The model is tested on two corpora:
- **EuroParl**, with 11 languages aligned at the speech level, which checks behaviour on true translations.
- **Wikipedia**, with 12 languages, where linked articles are comparable but not translations.

PLTM recovers clearly aligned topics in every language, covering both technical acronyms and everyday vocabulary. Linked Wikipedia articles turn out to be topically consistent on average across languages, yet each language emphasises topics differently overall. This matters because comparable multilingual data such as Wikipedia is far more plentiful than parallel corpora. The model also helps with lexicon induction for low-resource pairs and with cross-lingual trend analysis.

## Key points

- **EuroParl setup:** 11 Western European languages, roughly a decade of parliamentary proceedings, and more than 121 million words. The 25 most frequent word types were removed. Lexical variety per 10,000 tokens differs widely, for example Finnish at 336.2 versus English at 43.1.
- **Wikipedia setup:** 12 languages, namely English (EN), German (DE), French (FR), Italian (IT), Polish (PL), Russian (RU), Turkish (TR), Greek (EL), Finnish (FI), Hebrew (HE), Persian (FA) and Welsh (CY).
- **Training settings:** 1000 Gibbs iterations, β set to 0.01 for each language, and α initialised to 0.01·T with a uniform base measure. αm is re-optimised every 10 iterations, and the example topics come from T=400.
- **Cross-language divergence:** average Jensen-Shannon divergence between language pairs on Wikipedia falls mostly in the 0.08–0.12 range at T=400. This matches EuroParl, even for language pairs with a history of conflict.
- **Uneven emphasis:** topic emphasis differs by language. Cross-country skiing is prominent in Finnish but almost absent in Welsh, while topics about actors are uniform across languages.

## Interesting ideas
- **Share the mixture, separate the vocabularies:** a single topic mixture per tuple combined with per-language word distributions avoids explicit word alignment. The design scales easily past two languages and could suit Ukrainian–Russian–English comparable corpora or multilingual trade documents.
- **Translations as a controlled testbed:** the authors first run the model on genuine translations, where the shared-topic assumption is known to hold, and only then move to noisier comparable data. This is a sensible validation step for any cross-lingual method.
- **Per-language divergence inside a tuple:** splitting each tuple's topic assignments by language and computing the JS divergence between the parts measures how much versions of the "same" document disagree. It could flag Wikipedia articles whose coverage differs across languages.
- **Language-level topic shares:** counting the proportion of tokens per topic in each language reveals cultural or regional focus, which is useful for geography-oriented or media comparison studies.
- **Topics as a lexicon seed:** aligned topics provide candidate translation pairs without parallel sentences. These candidates can feed heavier alignment methods for low-resource language pairs.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Massachusetts Amherst](https://md.research.elamur.ai/free/en/universities/university-of-massachusetts-amherst/) — University, Amherst Center, United States · 106 works · 66,692 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 General Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/general-social-sciences/) · 227 works

💡 Concepts (1): [💡 Topic modeling](https://md.research.elamur.ai/free/en/concepts/topic-modeling/) · 47 works

## Cite

Cite as: David Mimno, Hanna M. Wallach, Jason Naradowsky, David A. Smith and Andrew McCallum. 2009. Polylingual Topic Models. EMNLP 2009.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mimno2009polylingual,
  title = {{Polylingual Topic Models}},
  author = {David Mimno, Hanna M. Wallach, Jason Naradowsky, David A. Smith and Andrew McCallum},
  booktitle = {EMNLP 2009},
  year = {2009},
  doi = {10.3115/1699571.1699627},
  url = {https://doi.org/10.3115/1699571.1699627}
}
```

- [← previous Automatic sense prediction for implicit discourse relations in text](https://md.research.elamur.ai/free/en/papers/pitler-2009-automatic-sense-prediction-implicit-discourse/)
- [next → Recognizing Implicit Discourse Relations in the Penn Discourse Treebank](https://md.research.elamur.ai/free/en/papers/lin-2009-recognizing-implicit-discourse-relations-penn/)
- [↑ back to papers of 2009](https://md.research.elamur.ai/free/en/papers/2009/)

## References

14 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. David Blei and Michael Jordan. 2003. Modeling annotated data. In SIGIR. [find](https://scholar.google.com/scholar?q=David+Blei+and+Michael+Jordan%2E+2003%2E+Modeling+annotated+data%2E+In+SIGIR%2E)
2. David Blei, Andrew Ng, and Michael Jordan. 2003. [Latent Dirichlet allocation](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/). JMLR.
3. Peter F Brown, Stephen A Della Pietra, Vincent J Della Pietra, and Robert L Mercer. 1993. The mathematics of statistical machine translation: Parameter estimation. CL, 19(2):263–311. [find](https://scholar.google.com/scholar?q=Peter+F+Brown%2C+Stephen+A+Della+Pietra%2C+Vincent+J+Della+Pietra%2C+and+Robert+L+Mercer%2E+1993%2E+The+mathematics+of+statistical+machine+translation%3A+Parameter+estimation%2E+CL%2C+19%282%29%3A263%E2%80%93311%2E)
4. Aria Haghighi, Percy Liang, Taylor Berg-Kirkpatrick, and Dan Klein. 2008. [Learning bilingual lexicons from monolingual corpora](https://md.research.elamur.ai/free/en/papers/haghighi-2008-learning-bilingual-lexicons-monolingual-corpora/). In ACL, pages 771–779.
5. David Hall, Daniel Jurafsky, and Christopher D. Manning. 2008. Studying the history of ideas using topic models. In EMNLP. [find](https://scholar.google.com/scholar?q=David+Hall%2C+Daniel+Jurafsky%2C+and+Christopher+D%2E+Manning%2E+2008%2E+Studying+the+history+of+ideas+using+topic+models%2E+In+EMNLP%2E)
6. Mariko Kawaba, Hiroyuki Nakasaki, Takehito Utsuro, and Tomohiro Fukuhara. 2008. Cross-lingual blog analysis based on multilingual blog distillation from multilingual Wikipedia entries. In ICWSM. [find](https://scholar.google.com/scholar?q=Mariko+Kawaba%2C+Hiroyuki+Nakasaki%2C+Takehito+Utsuro%2C+and+Tomohiro+Fukuhara%2E+2008%2E+Cross%2Dlingual+blog+analysis+based+on+multilingual+blog+distillation+from+multilingual+Wikipedia+entries%2E+In+ICWSM%2E)
7. Philipp Koehn and Kevin Knight. 2002. [Learning a translation lexicon from monolingual corpora](https://md.research.elamur.ai/free/en/papers/koehn-2002-learning-translation-lexicon-monolingual-corpora/). In Proceedings of ACL Workshop on Unsupervised Lexical Acquisition.
8. Gideon Mann, David Mimno, and Andrew McCallum. 2006. Bibliometric impact measures leveraging topic analysis. In JCDL. [find](https://scholar.google.com/scholar?q=Gideon+Mann%2C+David+Mimno%2C+and+Andrew+McCallum%2E+2006%2E+Bibliometric+impact+measures+leveraging+topic+analysis%2E+In+JCDL%2E)
9. Andrew McCallum, Andrés Corrada-Emmanuel, and Xuerui Wang. 2005. Topic and role discovery in social networks. In IJCAI. [find](https://scholar.google.com/scholar?q=Andrew+McCallum%2C+Andre%CC%81s+Corrada%2DEmmanuel%2C+and+Xuerui+Wang%2E+2005%2E+Topic+and+role+discovery+in+social+networks%2E+In+IJCAI%2E)
10. Xiaochuan Ni, Jian-Tao Sun, Jian Hu, and Zheng Chen. 2009. Mining multilingual topics from Wikipedia. In WWW. [find](https://scholar.google.com/scholar?q=Xiaochuan+Ni%2C+Jian%2DTao+Sun%2C+Jian+Hu%2C+and+Zheng+Chen%2E+2009%2E+Mining+multilingual+topics+from+Wikipedia%2E+In+WWW%2E)
11. Yik-Cheung Tam, Ian Lane, and Tanja Schultz. 2007. Bilingual LSA-based adaptation for statistical machine translation. Machine Translation, 28:187– 207. [find](https://scholar.google.com/scholar?q=Yik%2DCheung+Tam%2C+Ian+Lane%2C+and+Tanja+Schultz%2E+2007%2E+Bilingual+LSA%2Dbased+adaptation+for+statistical+machine+translation%2E+Machine+Translation%2C+28%3A187%E2%80%93+207%2E)
12. Hanna Wallach, Iain Murray, Ruslan Salakhutdinov, and David Mimno. 2009. Evaluation methods for topic models. In ICML. [find](https://scholar.google.com/scholar?q=Hanna+Wallach%2C+Iain+Murray%2C+Ruslan+Salakhutdinov%2C+and+David+Mimno%2E+2009%2E+Evaluation+methods+for+topic+models%2E+In+ICML%2E)
13. Xing Wei and Bruce Croft. 2006. LDA-based document models for ad-hoc retrieval. In SIGIR. [find](https://scholar.google.com/scholar?q=Xing+Wei+and+Bruce+Croft%2E+2006%2E+LDA%2Dbased+document+models+for+ad%2Dhoc+retrieval%2E+In+SIGIR%2E)
14. Bing Zhao and Eric P. Xing. 2007. HM-BiTAM: Bilingual topic exploration, word alignment, and translation. In NIPS. [find](https://scholar.google.com/scholar?q=Bing+Zhao+and+Eric+P%2E+Xing%2E+2007%2E+HM%2DBiTAM%3A+Bilingual+topic+exploration%2C+word+alignment%2C+and+translation%2E+In+NIPS%2E)

*Updated 30 October 2026*