---
title: "A Latent Variable Model for Geographic Lexical Variation (2010)"
description: "The work offers an exploratory, data-driven complement to traditional dialectology. 34 references, cited by 109 papers here."
authors: ["Jacob Eisenstein", "Brendan O'Connor", "Noah A. Smith", "Eric P. Xing"]
year: 2010
venue: "Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/D10-1124/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/eisenstein-2010-latent-variable-model-geographic-lexical/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/eisenstein-2010-latent-variable-model-geographic-lexical/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2010](https://md.research.elamur.ai/free/en/papers/2010/) / A Latent Variable Model for Geographic Lexical Variation

# A Latent Variable Model for Geographic Lexical Variation

**Authors:** [Jacob Eisenstein](https://md.research.elamur.ai/free/en/authors/jacob-eisenstein/), [Brendan O'Connor](https://md.research.elamur.ai/free/en/authors/brendan-o-connor/), [Noah A. Smith](https://md.research.elamur.ai/free/en/authors/noah-a-smith/) and [Eric P. Xing](https://md.research.elamur.ai/free/en/authors/eric-xing/) · **Year:** 2010 · **Venue:** [Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2010](https://md.research.elamur.ai/free/en/venues/emnlp/2010/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/D10-1124/) · [PDF](https://aclanthology.org/D10-1124.pdf)

**Citations:** [109](https://md.research.elamur.ai/free/en/papers/eisenstein-2010-latent-variable-model-geographic-lexical/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Studies of dialect and sociolinguistic variation usually start from a small set of variables that researchers choose by hand. This paper asks whether regional word usage can be found directly from raw text instead. The authors collected geotagged Twitter posts from the contiguous United States and merged each author's messages into one document. They then built a generative model in which every author belongs to a hidden region. That region does two things: it produces the author's GPS coordinates, and it selects a regional version of each latent topic. Regional topics are modeled as noisy deviations from shared base topics using stacked logistic normal priors, an approach the authors call cascading topic models. Inference uses mean-field variational methods. The model finds coherent topics, plausible regional slang and spatially compact linguistic regions. When predicting an author's location from text alone, it beats both text regression and supervised topic models. The work offers an exploratory, data-driven complement to traditional dialectology.

## Key points
- **Data source:** one week of the Twitter "Gardenhose" stream (early March 2010, about 15% of public posts).
- **Filtering:** the authors kept only authors who
  - posted from mobile clients with GPS coordinates,
  - wrote at least 20 messages,
  - had under 1,000 followers and under 1,000 followees,
  - and they also removed messages with URLs.
- **Final corpus:**
  - about 9,500 users, 380,000 messages and 4.7M tokens;
  - vocabulary of 5,216 word types, of which 1,332 are missing from the English, French and Spanish aspell dictionaries;
  - released publicly at ark.cs.cmu.edu/GeoTwitter.
- **Model structure:** base topics are drawn from a Gaussian, and region-topic variants are Gaussians centered on them with a topic-specific variance. Words come from the softmax of the region-topic vector, and locations come from region-specific bivariate Gaussians.
- **Findings:** the model associates terms such as "hella" with Northern California. In one run, the induced regions included nine compact metro areas, Florida, a Lake Erie area, and a broad north/south split.

## Interesting ideas
- **Cascading priors:** each level of the hierarchy serves as the mean for the next level, so sparse region-topic pairs automatically fall back to the base topic. This shrinkage trick could carry over to other dimensions, such as time, genre or speaker group.
- **Latent regions instead of fixed boundaries:** regions are learned from location and text together, rather than taken from fixed political borders. This lets the data define linguistically coherent areas, which is useful wherever administrative units may not match real usage patterns.
- **Separating topic from geography:** because a word's frequency can reflect what people talk about rather than where they live, modeling topic and region jointly avoids confusing the two. The same concern applies to any study linking word use to metadata.
- **Location prediction as validation:** predicting authors' locations from text provides a quantitative check on an otherwise exploratory model. This is a good pattern whenever qualitative clusters need objective support.
- **Filtering out atypical accounts:** dropping highly connected accounts and URL-bearing posts removes celebrities, marketers and bots. These are cheap heuristics that help any social-media corpus focus on ordinary personal language.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works

## Cite

Cite as: Jacob Eisenstein, Brendan O'Connor, Noah A. Smith and Eric P. Xing. 2010. A Latent Variable Model for Geographic Lexical Variation. Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{eisenstein2010latent,
  title = {{A Latent Variable Model for Geographic Lexical Variation}},
  author = {Jacob Eisenstein, Brendan O'Connor, Noah A. Smith and Eric P. Xing},
  booktitle = {Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing},
  year = {2010},
  url = {https://aclanthology.org/D10-1124/}
}
```

- [← previous Coreference Resolution with Reconcile](https://md.research.elamur.ai/free/en/papers/stoyanov-2010-coreference-resolution-reconcile/)
- [next → Painless Unsupervised Learning with Features](https://md.research.elamur.ai/free/en/papers/kirkpatrick-2010-painless-unsupervised-learning-features/)
- [↑ back to papers of 2010](https://md.research.elamur.ai/free/en/papers/2010/)

## References

34 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. L. Backstrom, J. Kleinberg, R. Kumar, and J. Novak. 2008. Spatial variation in search engine queries. In Proceedings of WWW. [find](https://scholar.google.com/scholar?q=L%2E+Backstrom%2C+J%2E+Kleinberg%2C+R%2E+Kumar%2C+and+J%2E+Novak%2E+2008%2E+Spatial+variation+in+search+engine+queries%2E+In+Proceedings+of+WWW%2E)
2. C. M. Bishop. 2006. Pattern Recognition and Machine Learning. Springer. [find](https://scholar.google.com/scholar?q=C%2E+M%2E+Bishop%2E+2006%2E+Pattern+Recognition+and+Machine+Learning%2E+Springer%2E)
3. D. M. Blei and M. I. Jordan. 2006. Variational inference for Dirichlet process mixtures. Bayesian Analysis, 1:121–144. [find](https://scholar.google.com/scholar?q=D%2E+M%2E+Blei+and+M%2E+I%2E+Jordan%2E+2006%2E+Variational+inference+for+Dirichlet+process+mixtures%2E+Bayesian+Analysis%2C+1%3A121%E2%80%93144%2E)
4. D. M. Blei and J. Lafferty. 2006a. Correlated topic models. In NIPS. [find](https://scholar.google.com/scholar?q=D%2E+M%2E+Blei+and+J%2E+Lafferty%2E+2006a%2E+Correlated+topic+models%2E+In+NIPS%2E)
5. D. M. Blei and J. Lafferty. 2006b. [Dynamic topic models](https://md.research.elamur.ai/free/en/papers/blei-2006-dynamic-topic-models/). In Proceedings of ICML.
6. D. M. Blei and J. D. McAuliffe. 2007. Supervised topic models. In NIPS. [find](https://scholar.google.com/scholar?q=D%2E+M%2E+Blei+and+J%2E+D%2E+McAuliffe%2E+2007%2E+Supervised+topic+models%2E+In+NIPS%2E)
7. D. M. Blei, A. Y. Ng, and M. I. Jordan. 2003. [Latent Dirichlet allocation](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/). JMLR, 3:993–1022.
8. M. Bucholtz, N. Bermudez, V. Fung, L. Edwards, and R. Vargas. 2007. Hella Nor Cal or totally So Cal? the perceptual dialectology of California. Journal of English Linguistics, 35(4):325–352. [find](https://scholar.google.com/scholar?q=M%2E+Bucholtz%2C+N%2E+Bermudez%2C+V%2E+Fung%2C+L%2E+Edwards%2C+and+R%2E+Vargas%2E+2007%2E+Hella+Nor+Cal+or+totally+So+Cal%3F+the+perceptual+dialectology+of+California%2E+Journal+of+English+Linguistics%2C+35%284%29%3A325%E2%80%93352%2E)
9. F. G. Cassidy and J. H. Hall. 1985. Dictionary of American Regional English, volume 1. Harvard University Press. [find](https://scholar.google.com/scholar?q=F%2E+G%2E+Cassidy+and+J%2E+H%2E+Hall%2E+1985%2E+Dictionary+of+American+Regional+English%2C+volume+1%2E+Harvard+University+Press%2E)
10. J. Chambers. 2009. Sociolinguistic Theory: Linguistic Variation and its Social Significance. Blackwell. [find](https://scholar.google.com/scholar?q=J%2E+Chambers%2E+2009%2E+Sociolinguistic+Theory%3A+Linguistic+Variation+and+its+Social+Significance%2E+Blackwell%2E)
11. D. J Crandall, L. Backstrom, D. Huttenlocher, and J. Kleinberg. 2009. Mapping the world’s photos. In Proceedings of WWW, page 761770. [find](https://scholar.google.com/scholar?q=D%2E+J+Crandall%2C+L%2E+Backstrom%2C+D%2E+Huttenlocher%2C+and+J%2E+Kleinberg%2E+2009%2E+Mapping+the+world%E2%80%99s+photos%2E+In+Proceedings+of+WWW%2C+page+761770%2E)
12. J. Friedman, T. Hastie, and R. Tibshirani. 2010. Regularization paths for generalized linear models via coordinate descent. Journal of Statistical Software, 33(1). [find](https://scholar.google.com/scholar?q=J%2E+Friedman%2C+T%2E+Hastie%2C+and+R%2E+Tibshirani%2E+2010%2E+Regularization+paths+for+generalized+linear+models+via+coordinate+descent%2E+Journal+of+Statistical+Software%2C+33%281%29%2E)
13. D. E. Johnson. 2009. Getting off the GoldVarb standard: Introducing Rbrul for mixed-effects variable rule analysis. Language and Linguistics Compass, 3(1):359– 383. [find](https://scholar.google.com/scholar?q=D%2E+E%2E+Johnson%2E+2009%2E+Getting+off+the+GoldVarb+standard%3A+Introducing+Rbrul+for+mixed%2Deffects+variable+rule+analysis%2E+Language+and+Linguistics+Compass%2C+3%281%29%3A359%E2%80%93+383%2E)
14. B. Johnstone. 2010. Language and place. In R. Mesthrie and W. Wolfram, editors, Cambridge Handbook of Sociolinguistics. Cambridge University Press. [find](https://scholar.google.com/scholar?q=B%2E+Johnstone%2E+2010%2E+Language+and+place%2E+In+R%2E+Mesthrie+and+W%2E+Wolfram%2C+editors%2C+Cambridge+Handbook+of+Sociolinguistics%2E+Cambridge+University+Press%2E)
15. M. Joshi, D. Das, K. Gimpel, and N. A. Smith. 2010. Movie reviews and revenues: An experiment in text regression. In Proceedings of NAACL-HLT. [find](https://scholar.google.com/scholar?q=M%2E+Joshi%2C+D%2E+Das%2C+K%2E+Gimpel%2C+and+N%2E+A%2E+Smith%2E+2010%2E+Movie+reviews+and+revenues%3A+An+experiment+in+text+regression%2E+In+Proceedings+of+NAACL%2DHLT%2E)
16. H. Kurath. 1949. A Word Geography of the Eastern United States. University of Michigan Press. [find](https://scholar.google.com/scholar?q=H%2E+Kurath%2E+1949%2E+A+Word+Geography+of+the+Eastern+United+States%2E+University+of+Michigan+Press%2E)
17. H. Kwak, C. Lee, H. Park, and S. Moon. 2010. What is Twitter, a social network or a news media? In Proceedings of WWW. [find](https://scholar.google.com/scholar?q=H%2E+Kwak%2C+C%2E+Lee%2C+H%2E+Park%2C+and+S%2E+Moon%2E+2010%2E+What+is+Twitter%2C+a+social+network+or+a+news+media%3F+In+Proceedings+of+WWW%2E)
18. W. Labov, S. Ash, and C. Boberg. 2006. The Atlas of North American English: Phonetics, Phonology, and Sound Change. Walter de Gruyter. [find](https://scholar.google.com/scholar?q=W%2E+Labov%2C+S%2E+Ash%2C+and+C%2E+Boberg%2E+2006%2E+The+Atlas+of+North+American+English%3A+Phonetics%2C+Phonology%2C+and+Sound+Change%2E+Walter+de+Gruyter%2E)
19. W. Labov. 1966. The Social Stratification of English in New York City. Center for Applied Linguistics. [find](https://scholar.google.com/scholar?q=W%2E+Labov%2E+1966%2E+The+Social+Stratification+of+English+in+New+York+City%2E+Center+for+Applied+Linguistics%2E)
20. Q. Mei, C. Liu, H. Su, and C. X Zhai. 2006. A probabilistic approach to spatiotemporal theme pattern mining on weblogs. In Proceedings of WWW, page 542. [find](https://scholar.google.com/scholar?q=Q%2E+Mei%2C+C%2E+Liu%2C+H%2E+Su%2C+and+C%2E+X+Zhai%2E+2006%2E+A+probabilistic+approach+to+spatiotemporal+theme+pattern+mining+on+weblogs%2E+In+Proceedings+of+WWW%2C+page+542%2E)
21. Q. Mei, X. Ling, M. Wondra, H. Su, and C. X. Zhai. 2007. Topic sentiment mixture: modeling facets and opinions in weblogs. In Proceedings of WWW. [find](https://scholar.google.com/scholar?q=Q%2E+Mei%2C+X%2E+Ling%2C+M%2E+Wondra%2C+H%2E+Su%2C+and+C%2E+X%2E+Zhai%2E+2007%2E+Topic+sentiment+mixture%3A+modeling+facets+and+opinions+in+weblogs%2E+In+Proceedings+of+WWW%2E)
22. T. P. Minka. 2003. Estimating a Dirichlet distribution. Technical report, Massachusetts Institute of Technology. [find](https://scholar.google.com/scholar?q=T%2E+P%2E+Minka%2E+2003%2E+Estimating+a+Dirichlet+distribution%2E+Technical+report%2C+Massachusetts+Institute+of+Technology%2E)
23. J. Nerbonne. 2009. Data-driven dialectology. Language and Linguistics Compass, 3(1). [find](https://scholar.google.com/scholar?q=J%2E+Nerbonne%2E+2009%2E+Data%2Ddriven+dialectology%2E+Language+and+Linguistics+Compass%2C+3%281%29%2E)
24. B. O’Connor, M. Krieger, and D. Ahn. 2010. TweetMotif: Exploratory search and topic summarization for twitter. In Proceedings of ICWSM. [find](https://scholar.google.com/scholar?q=B%2E+O%E2%80%99Connor%2C+M%2E+Krieger%2C+and+D%2E+Ahn%2E+2010%2E+TweetMotif%3A+Exploratory+search+and+topic+summarization+for+twitter%2E+In+Proceedings+of+ICWSM%2E)
25. J. C. Paolillo. 2002. Analyzing Linguistic Variation: Statistical Models and Methods. CSLI Publications. [find](https://scholar.google.com/scholar?q=J%2E+C%2E+Paolillo%2E+2002%2E+Analyzing+Linguistic+Variation%3A+Statistical+Models+and+Methods%2E+CSLI+Publications%2E)
26. M. Paul and R. Girju. 2010. A two-dimensional topic-aspect model for discovering multi-faceted topics. In Proceedings of AAAI. [find](https://scholar.google.com/scholar?q=M%2E+Paul+and+R%2E+Girju%2E+2010%2E+A+two%2Ddimensional+topic%2Daspect+model+for+discovering+multi%2Dfaceted+topics%2E+In+Proceedings+of+AAAI%2E)
27. W. D. Penny. 2001. Variational Bayes for d-dimensional Gaussian mixture models. Technical report, University College London. [find](https://scholar.google.com/scholar?q=W%2E+D%2E+Penny%2E+2001%2E+Variational+Bayes+for+d%2Ddimensional+Gaussian+mixture+models%2E+Technical+report%2C+University+College+London%2E)
28. D. Sankoff, S. A. Tagliamonte, and E. Smith. 2005. Goldvarb X: A variable rule application for Macintosh and Windows. Technical report, Department of Linguistics, University of Toronto. [find](https://scholar.google.com/scholar?q=D%2E+Sankoff%2C+S%2E+A%2E+Tagliamonte%2C+and+E%2E+Smith%2E+2005%2E+Goldvarb+X%3A+A+variable+rule+application+for+Macintosh+and+Windows%2E+Technical+report%2C+Department+of+Linguistics%2C+University+of+Toronto%2E)
29. R. W. Sinnott. 1984. Virtues of the Haversine. Sky and Telescope, 68(2). [find](https://scholar.google.com/scholar?q=R%2E+W%2E+Sinnott%2E+1984%2E+Virtues+of+the+Haversine%2E+Sky+and+Telescope%2C+68%282%29%2E)
30. B. Szmrecsanyi. 2010. Geography is overrated. In S. Hansen, C. Schwarz, P. Stoeckle, and T. Streck, editors, Dialectological and Folk Dialectological Concepts of Space. Walter de Gruyter. [find](https://scholar.google.com/scholar?q=B%2E+Szmrecsanyi%2E+2010%2E+Geography+is+overrated%2E+In+S%2E+Hansen%2C+C%2E+Schwarz%2C+P%2E+Stoeckle%2C+and+T%2E+Streck%2C+editors%2C+Dialectological+and+Folk+Dialectological+Concepts+of+Space%2E+Walter+de+Gruyter%2E)
31. S. A. Tagliamonte and D. Denis. 2008. Linguistic ruin? LOL! Instant messanging and teen language. American Speech, 83. [find](https://scholar.google.com/scholar?q=S%2E+A%2E+Tagliamonte+and+D%2E+Denis%2E+2008%2E+Linguistic+ruin%3F+LOL%21+Instant+messanging+and+teen+language%2E+American+Speech%2C+83%2E)
32. S. A. Tagliamonte. 2006. Analysing Sociolinguistic Variation. Cambridge University Press. [find](https://scholar.google.com/scholar?q=S%2E+A%2E+Tagliamonte%2E+2006%2E+Analysing+Sociolinguistic+Variation%2E+Cambridge+University+Press%2E)
33. M. J. Wainwright and M. I. Jordan. 2008. Graphical Models, Exponential Families, and Variational Inference. Now Publishers. [find](https://scholar.google.com/scholar?q=M%2E+J%2E+Wainwright+and+M%2E+I%2E+Jordan%2E+2008%2E+Graphical+Models%2C+Exponential+Families%2C+and+Variational+Inference%2E+Now+Publishers%2E)
34. E. P. Xing. 2005. On topic evolution. Technical Report 05-115, Center for Automated Learning and Discovery, Carnegie Mellon University. [find](https://scholar.google.com/scholar?q=E%2E+P%2E+Xing%2E+2005%2E+On+topic+evolution%2E+Technical+Report+05%2D115%2C+Center+for+Automated+Learning+and+Discovery%2C+Carnegie+Mellon+University%2E)

*Updated 24 October 2026*