---
title: "Word Representations — Turian 2010"
description: "Supervised NLP systems usually encode words as one-hot vectors, so rare words get poorly estimated parameters and unseen words get none."
authors: ["Joseph Turian", "Lev-Arie Ratinov", "Yoshua Bengio"]
year: 2010
venue: "Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/P10-1040/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/turian-2010-word-representations-simple-general-method/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/turian-2010-word-representations-simple-general-method/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2010](https://md.research.elamur.ai/free/en/papers/2010/page/2/) / Word Representations: A Simple and General Method for Semi-Supervised Learning

# Word Representations: A Simple and General Method for Semi-Supervised Learning

**Authors:** [Joseph Turian](https://md.research.elamur.ai/free/en/authors/joseph-turian/), Lev-Arie Ratinov, [Yoshua Bengio](https://md.research.elamur.ai/free/en/authors/yoshua-bengio/) · **Year:** 2010 · **Venue:** Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P10-1040/) · [PDF](https://aclanthology.org/P10-1040.pdf) · [OpenAlex](https://openalex.org/W2158139315)

**Citations:** 1857 citations (openalex-snapshot · 2026-10-14) · [564](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Supervised NLP systems usually encode words as one-hot vectors, so rare words get poorly estimated parameters and unseen words get none. Complex semi-supervised frameworks force a specific model and training procedure. The authors propose a lighter alternative: induce word representations from unlabeled text once, then add them as extra features to any existing supervised system. They run what they describe as the first controlled comparison of three representations: clusters produced by the Brown algorithm, embeddings following the approach of Collobert and Weston (2008), and the HLBL embeddings of Mnih & Hinton (2009). These were induced on RCV1 and tested on NER and chunking with near-state-of-the-art baselines. Every representation raised accuracy, and combining several gave further gains, larger for NER than for chunking. Brown clusters were strongest for NER because they handle rare words better than embeddings that received few training updates. The authors also give a default scaling rule for embeddings, so these can be used without tuning. They release the features, the code and the full experimental system. The method offers a cheap, model-agnostic route to semi-supervised gains.

## Key points
- Three representations are compared: Brown hierarchical clustering, C&W embeddings and HLBL embeddings. Brown clustering is a class-based bigram model with O(V·K²) cost and includes a 1000-cluster setting. C&W embeddings use a ranking loss that scores real n-grams above corrupted ones.
- Evaluation covers NER and chunking. For NER, the gains were bigger on out-of-domain test data than in-domain, even though the out-of-domain text was never used for induction.
- Error analysis by word frequency shows that most NER errors fall on rare words, where Brown clusters err less. For chunking, Brown and C&W make almost identical numbers of errors, mostly on frequent words.
- With improved training, the authors retract earlier negative findings about C&W embeddings from Turian et al. (2009).
- Representations, induction code and the NER/chunking system are available at http://metaoptimize.com/projects/wordreprs/. The exact F1 scores are in a section missing from the provided excerpt.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/)

## Cited by

Cited by

564 papers of our library cite it (of the 140673 whose bibliographies we read); 74 of them have a page here, 490 not yet.

- [A Dependency Parser for Tweets](https://md.research.elamur.ai/free/en/papers/aclbulk-2014-kong-dependency-parser-tweets/)
- [A* CCG Parsing with a Supertag-factored Model](https://md.research.elamur.ai/free/en/papers/aclbulk-2014-lewis-ccg-parsing-supertag-factored-model/)
- [LSTM CCG Parsing](https://md.research.elamur.ai/free/en/papers/aclbulk-2016-lewis-lstm-ccg-parsing/)
- [Shared Tasks of the 2015 Workshop on Noisy User-generated Text: Twitter Lexical Normalization and Named Entity Recognition](https://md.research.elamur.ai/free/en/papers/baldwin-2015-shared-tasks-2015-workshop-noisy/)
- [Tailoring Continuous Word Representations for Dependency Parsing](https://md.research.elamur.ai/free/en/papers/bansal-2014-tailoring-continuous-word-representations-dependency/)
- [Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors](https://md.research.elamur.ai/free/en/papers/baroni-2014-don-t-count-predict-systematic/)
- [Experience Grounds Language](https://md.research.elamur.ai/free/en/papers/bisk-2020-experience-grounds-language/)
- [A Comparison of Vector-based Representations for Semantic Composition](https://md.research.elamur.ai/free/en/papers/blacoe-2012-comparison-vector-based-representations-semantic/)
- [On the Opportunities and Risks of Foundation Models](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/)
- [Comparing Word Representations for Implicit Discourse Relation Classification](https://md.research.elamur.ai/free/en/papers/braud-2015-comparing-word-representations-implicit-discourse/)
- [Learning Connective-based Word Representations for Implicit Discourse Relation Identification](https://md.research.elamur.ai/free/en/papers/braud-2016-learning-connective-based-word-representations/)
- [A Unified Model for Word Sense Representation and Disambiguation](https://md.research.elamur.ai/free/en/papers/chen-2014-unified-model-word-sense-representation/)
- [Event Extraction via Dynamic Multi-Pooling Convolutional Neural Networks](https://md.research.elamur.ai/free/en/papers/chen-2015-event-extraction-dynamic-multi-pooling/)
- [Implicit Discourse Relation Detection via a Deep Architecture with Gated Relevance Network](https://md.research.elamur.ai/free/en/papers/chen-2016-implicit-discourse-relation-detection-deep/)
- [Named Entity Recognition with Bidirectional LSTM-CNNs](https://md.research.elamur.ai/free/en/papers/chiu-2015-named-entity-recognition-bidirectional-lstm/)
- [Natural Language Processing (almost) from Scratch](https://md.research.elamur.ai/free/en/papers/collobert-2011-natural-language-processing-almost-scratch/)
- [Frame-Semantic Parsing](https://md.research.elamur.ai/free/en/papers/das-2014-frame-semantic-parsing/)
- [An Empirical Comparison of Parsing Methods for Stanford Dependencies](https://md.research.elamur.ai/free/en/papers/enud-2014-kong-empirical-comparison-parsing-methods-stanford/)
- [Improving Vector Space Word Representations Using Multilingual Correlation](https://md.research.elamur.ai/free/en/papers/faruqui-2014-improving-vector-space-word-representations/)
- [Problems With Evaluation of Word Embeddings Using Word Similarity Tasks](https://md.research.elamur.ai/free/en/papers/faruqui-2016-problems-evaluation-word-embeddings-word/)
- [Learning Semantic Hierarchies via Word Embeddings](https://md.research.elamur.ai/free/en/papers/fu-2014-learning-semantic-hierarchies-word-embeddings/)
- [SimVerb-3500: A Large-Scale Evaluation Set of Verb Similarity](https://md.research.elamur.ai/free/en/papers/gerz-2016-simverb-3500-large-scale-evaluation/)
- [Part-of-Speech Tagging for Twitter: Annotation, Features, and Experiments](https://md.research.elamur.ai/free/en/papers/gimpel-2011-part-speech-tagging-twitter-annotation/)
- [Cross-lingual Dependency Parsing Based on Distributed Representations](https://md.research.elamur.ai/free/en/papers/guo-2015-cross-lingual-dependency-parsing-based/)
- [Multilingual Models for Compositional Distributed Semantics](https://md.research.elamur.ai/free/en/papers/hermann-2014-multilingual-models-compositional-distributed-semantics/)
- [SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation](https://md.research.elamur.ai/free/en/papers/hill-2015-simlex-999-evaluating-semantic-models/)
- [Improving Word Representations via Global Context and Multiple Word Prototypes](https://md.research.elamur.ai/free/en/papers/huang-2012-improving-word-representations-global-context/)
- [SensEmbed: Learning Sense Embeddings for Word and Relational Similarity](https://md.research.elamur.ai/free/en/papers/iacobacci-2015-sensembed-learning-sense-embeddings-word/)
- [Modeling Speech Acts in Asynchronous Conversations: A Neural-CRF Approach](https://md.research.elamur.ai/free/en/papers/joty-2018-modeling-speech-acts-asynchronous-conversations/)
- [A Convolutional Neural Network for Modelling Sentences](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2014-convolutional-neural-network-modelling-sentences/)
- [Dialogue Act Classification in Domain-Independent Conversations Using a Deep Recurrent Neural Network](https://md.research.elamur.ai/free/en/papers/khanpour-2016-dialogue-act-classification-domain-independent/)
- [Inducing Crosslingual Distributed Representations of Words](https://md.research.elamur.ai/free/en/papers/klementiev-2012-inducing-crosslingual-distributed-representations-words/)
- [Distributed Representations of Sentences and Documents](https://md.research.elamur.ai/free/en/papers/le-2014-distributed-representations-sentences-documents/)
- [Linguistic Regularities in Sparse and Explicit Word Representations](https://md.research.elamur.ai/free/en/papers/levy-2014-linguistic-regularities-sparse-explicit-word/)
- [Two/Too Simple Adaptations of Word2Vec for Syntax Problems](https://md.research.elamur.ai/free/en/papers/ling-2015-two-too-simple-adaptations-word2vec/)
- [Prepositional Phrase Attachment over Word Embedding Products](https://md.research.elamur.ai/free/en/papers/lingbulk-2017-madhyastha-prepositional-phrase-attachment-over-word/)
- [Fine-grained Opinion Mining with Recurrent Neural Networks and Word Embeddings](https://md.research.elamur.ai/free/en/papers/liu-2015-fine-grained-opinion-mining-recurrent/)
- [Representation Learning Using Multi-Task Deep Neural Networks for Semantic Classification and Information Retrieval](https://md.research.elamur.ai/free/en/papers/liu-2015-representation-learning-multi-task-deep/)
- [Better Word Representations with Recursive Neural Networks for Morphology](https://md.research.elamur.ai/free/en/papers/luong-2013-better-word-representations-recursive-neural/)
- [Bilingual Word Representations with Monolingual Quality in Mind](https://md.research.elamur.ai/free/en/papers/luong-2015-bilingual-word-representations-monolingual-quality/)
- [Learning Word Vectors for Sentiment Analysis](https://md.research.elamur.ai/free/en/papers/maas-2011-learning-word-vectors-sentiment-analysis/)
- [Transferring Coreference Resolvers with Posterior Regularization](https://md.research.elamur.ai/free/en/papers/martins-2015-transferring-coreference-resolvers-posterior-regularization/)
- [Linguistic Regularities in Continuous Space Word Representations](https://md.research.elamur.ai/free/en/papers/mikolov-2013-linguistic-regularities-continuous-space-word/)
- [Efficient Non-parametric Estimation of Multiple Embeddings per Word in Vector Space](https://md.research.elamur.ai/free/en/papers/neelakantan-2014-efficient-non-parametric-estimation-multiple/)
- [Event Detection and Domain Adaptation with Convolutional Neural Networks](https://md.research.elamur.ai/free/en/papers/nguyen-2015-event-detection-domain-adaptation-convolutional/)
- [Relation Extraction: Perspective from Convolutional Neural Networks](https://md.research.elamur.ai/free/en/papers/nguyen-2015-relation-extraction-perspective-convolutional-neural/)
- [Joint Event Extraction via Recurrent Neural Networks](https://md.research.elamur.ai/free/en/papers/nguyen-2016-joint-event-extraction-recurrent-neural/)
- [Improved Part-of-Speech Tagging for Online Conversational Text with Word Clusters](https://md.research.elamur.ai/free/en/papers/owoputi-2013-improved-part-speech-tagging-online/)
- [Lexicon Infused Phrase Embeddings for Named Entity Resolution](https://md.research.elamur.ai/free/en/papers/passos-2014-lexicon-infused-phrase-embeddings-named/)
- [Named Entity Recognition for Chinese Social Media with Jointly Trained Embeddings](https://md.research.elamur.ai/free/en/papers/peng-2015-named-entity-recognition-chinese-social/)
- [Polyglot: Distributed Word Representations for Multilingual NLP](https://md.research.elamur.ai/free/en/papers/rfou-2013-polyglot-distributed-word-representations-multilingual/)
- [Named Entity Recognition in Tweets: An Experimental Study](https://md.research.elamur.ai/free/en/papers/ritter-2011-named-entity-recognition-tweets-experimental/)
- [Discovering Implicit Discourse Relations Through Brown Cluster Pair Representation and Coreference Patterns](https://md.research.elamur.ai/free/en/papers/rutherford-2014-discovering-implicit-discourse-relations-through/)
- [Evaluation methods for unsupervised word embeddings](https://md.research.elamur.ai/free/en/papers/schnabel-2015-evaluation-methods-unsupervised-word-embeddings/)
- [Discriminative Lexical Semantic Segmentation with Gaps: Running the MWE Gamut](https://md.research.elamur.ai/free/en/papers/schneider-2014-discriminative-lexical-semantic-segmentation-gaps/)
- [Semi-Supervised Recursive Autoencoders for Predicting Sentiment Distributions](https://md.research.elamur.ai/free/en/papers/socher-2011-semi-supervised-recursive-autoencoders-predicting/)
- [Fast and Accurate Entity Recognition with Iterated Dilated Convolutions](https://md.research.elamur.ai/free/en/papers/strubell-2017-fast-accurate-entity-recognition-iterated/)
- [Cross-lingual Word Clusters for Direct Transfer of Linguistic Structure](https://md.research.elamur.ai/free/en/papers/tackstrom-2012-cross-lingual-word-clusters-direct/)
- [Exploring Compositional Architectures and Word Vector Representations for Prepositional Phrase Attachment](https://md.research.elamur.ai/free/en/papers/tacl-2014-belinkov-exploring-compositional-architectures-word-vector/)
- [Learning Sentiment-Specific Word Embedding for Twitter Sentiment Classification](https://md.research.elamur.ai/free/en/papers/tang-2014-learning-sentiment-specific-word-embedding/)
- [Representing Text for Joint Embedding of Text and Knowledge Bases](https://md.research.elamur.ai/free/en/papers/toutanova-2015-representing-text-joint-embedding-text/)
- [Metaphor Detection with Cross-Lingual Model Transfer](https://md.research.elamur.ai/free/en/papers/tsvetkov-2014-metaphor-detection-cross-lingual-model/)
- [Boosting Transition-based AMR Parsing with Refined Actions and Auxiliary Analyzers](https://md.research.elamur.ai/free/en/papers/wang-2015-boosting-transition-based-amr-parsing/)
- [That's So Annoying!!!: A Lexical and Frame-Semantic Embedding Based Data Augmentation Approach to Automatic Categorization of Annoying Behaviors using #petpeeve Tweets](https://md.research.elamur.ai/free/en/papers/wang-2015-that-s-so-annoying-lexical/)
- [CAMR at SemEval-2016 Task 8: An Extended Transition-based AMR Parser](https://md.research.elamur.ai/free/en/papers/wang-2016-camr-semeval-2016-task-8/)
- [A Two-Stage Parsing Method for Text-Level Discourse Analysis](https://md.research.elamur.ai/free/en/papers/wang-2017-two-stage-parsing-method-text/)
- [CorefQA: Coreference Resolution as Query-based Span Prediction](https://md.research.elamur.ai/free/en/papers/wu-2020-corefqa-coreference-resolution-as-query/)
- [Question Answering on Freebase via Relation Extraction and Textual Evidence](https://md.research.elamur.ai/free/en/papers/xu-2016-question-answering-freebase-relation-extraction/)
- [Learning Discriminative Projections for Text Similarity Measures](https://md.research.elamur.ai/free/en/papers/yih-2011-learning-discriminative-projections-text-similarity/)
- [Improving Lexical Embeddings with Semantic Knowledge](https://md.research.elamur.ai/free/en/papers/yu-2014-improving-lexical-embeddings-semantic-knowledge/)
- [Relation Classification via Convolutional Deep Neural Network](https://md.research.elamur.ai/free/en/papers/zeng-2014-relation-classification-convolutional-deep-neural/)
- [Neural Coreference Resolution with Deep Biaffine Attention by Joint Mention Detection and Mention Clustering](https://md.research.elamur.ai/free/en/papers/zhang-2018-neural-coreference-resolution-deep-biaffine/)
- [Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification](https://md.research.elamur.ai/free/en/papers/zhou-2016-attention-based-bidirectional-long-short/)
- [Bilingual Word Embeddings for Phrase-Based Machine Translation](https://md.research.elamur.ai/free/en/papers/zou-2013-bilingual-word-embeddings-phrase-based/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{turian2010word,
  title = {{Word Representations: A Simple and General Method for Semi-Supervised Learning}},
  author = {Turian, Joseph and Ratinov, Lev-Arie and Bengio, Yoshua},
  booktitle = {Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics},
  year = {2010},
  url = {https://aclanthology.org/P10-1040/}
}
```

- [← previous Why Synchronous Tree Substitution Grammars?](https://md.research.elamur.ai/free/en/papers/maletti-2010-why-synchronous-tree-substitution-grammars/)
- [↑ back to papers of 2010](https://md.research.elamur.ai/free/en/papers/2010/page/2/)

## References

52 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ando, R., & Zhang, T. (2005). A high-performance semi-supervised learning method for text chunking. ACL. [find](https://scholar.google.com/scholar?q=Ando%2C+R%2E%2C+%26+Zhang%2C+T%2E+%282005%29%2E+A+high%2Dperformance+semi%2Dsupervised+learning+method+for+text+chunking%2E+ACL%2E)
2. Bengio, Y. (2008). Neural net language models. Scholarpedia, 3, 3881. [find](https://scholar.google.com/scholar?q=Bengio%2C+Y%2E+%282008%29%2E+Neural+net+language+models%2E+Scholarpedia%2C+3%2C+3881%2E)
3. Bengio, Y., Ducharme, R., & Vincent, P. (2001). A neural probabilistic language model. NIPS. [find](https://scholar.google.com/scholar?q=Bengio%2C+Y%2E%2C+Ducharme%2C+R%2E%2C+%26+Vincent%2C+P%2E+%282001%29%2E+A+neural+probabilistic+language+model%2E+NIPS%2E)
4. Bengio, Y., Ducharme, R., Vincent, P., & Jauvin, C. (2003). A neural probabilistic language model. Journal of Machine Learning Research, 3, 1137–1155. [find](https://scholar.google.com/scholar?q=Bengio%2C+Y%2E%2C+Ducharme%2C+R%2E%2C+Vincent%2C+P%2E%2C+%26+Jauvin%2C+C%2E+%282003%29%2E+A+neural+probabilistic+language+model%2E+Journal+of+Machine+Learning+Research%2C+3%2C+1137%E2%80%931155%2E)
5. Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009). Curriculum learning. ICML. [find](https://scholar.google.com/scholar?q=Bengio%2C+Y%2E%2C+Louradour%2C+J%2E%2C+Collobert%2C+R%2E%2C+%26+Weston%2C+J%2E+%282009%29%2E+Curriculum+learning%2E+ICML%2E) [our card](https://md.research.elamur.ai/free/en/papers/bengio-2009-curriculum-learning/)
6. Bengio, Y., & Sénécal, J.-S. (2003). Quick training of probabilistic neural nets by importance sampling. AISTATS. [find](https://scholar.google.com/scholar?q=Bengio%2C+Y%2E%2C+%26+Se%CC%81ne%CC%81cal%2C+J%2E%2DS%2E+%282003%29%2E+Quick+training+of+probabilistic+neural+nets+by+importance+sampling%2E+AISTATS%2E)
7. Blei, D. M., Ng, A. Y., & Jordan, M. I. (2003). Latent dirichlet allocation. Journal of Machine Learning Research, 3, 993–1022. [find](https://scholar.google.com/scholar?q=Blei%2C+D%2E+M%2E%2C+Ng%2C+A%2E+Y%2E%2C+%26+Jordan%2C+M%2E+I%2E+%282003%29%2E+Latent+dirichlet+allocation%2E+Journal+of+Machine+Learning+Research%2C+3%2C+993%E2%80%931022%2E) [our card](https://md.research.elamur.ai/free/en/papers/blei-2003-latent-dirichlet-allocation/)
8. Brown, P. F., deSouza, P. V., Mercer, R. L., Pietra, V. J. D., & Lai, J. C. (1992). Class-based n-gram models of natural language. Computational Linguistics, 18, 467–479. [find](https://scholar.google.com/scholar?q=Brown%2C+P%2E+F%2E%2C+deSouza%2C+P%2E+V%2E%2C+Mercer%2C+R%2E+L%2E%2C+Pietra%2C+V%2E+J%2E+D%2E%2C+%26+Lai%2C+J%2E+C%2E+%281992%29%2E+Class%2Dbased+n%2Dgram+models+of+natural+language%2E+Computational+Linguistics%2C+18%2C+467%E2%80%93479%2E)
9. Candito, M., & Crabbé, B. (2009). Improving generative statistical parsing with semi-supervised word clustering. IWPT (pp. 138–141). [find](https://scholar.google.com/scholar?q=Candito%2C+M%2E%2C+%26+Crabbe%CC%81%2C+B%2E+%282009%29%2E+Improving+generative+statistical+parsing+with+semi%2Dsupervised+word+clustering%2E+IWPT+%28pp%2E+138%E2%80%93141%29%2E)
10. Collobert, R., & Weston, J. (2008). A unified architecture for natural language processing: Deep neural networks with multitask learning. ICML. [find](https://scholar.google.com/scholar?q=Collobert%2C+R%2E%2C+%26+Weston%2C+J%2E+%282008%29%2E+A+unified+architecture+for+natural+language+processing%3A+Deep+neural+networks+with+multitask+learning%2E+ICML%2E)
11. Deschacht, K., & Moens, M.-F. (2009). Semi-supervised semantic role labeling using the Latent Words Language Model. EMNLP (pp. 21–29). [find](https://scholar.google.com/scholar?q=Deschacht%2C+K%2E%2C+%26+Moens%2C+M%2E%2DF%2E+%282009%29%2E+Semi%2Dsupervised+semantic+role+labeling+using+the+Latent+Words+Language+Model%2E+EMNLP+%28pp%2E+21%E2%80%9329%29%2E)
12. Dumais, S. T., Furnas, G. W., Landauer, T. K., Deerwester, S., & Harshman, R. (1988). Using latent semantic analysis to improve access to textual information. SIGCHI Conference on Human Factors in Computing Systems (pp. 281–285). ACM. [find](https://scholar.google.com/scholar?q=Dumais%2C+S%2E+T%2E%2C+Furnas%2C+G%2E+W%2E%2C+Landauer%2C+T%2E+K%2E%2C+Deerwester%2C+S%2E%2C+%26+Harshman%2C+R%2E+%281988%29%2E+Using+latent+semantic+analysis+to+improve+access+to+textual+information%2E+SIGCHI+Conference+on+Human+Factors+in+Com)
13. Elman, J. L. (1993). Learning and development in neural networks: The importance of starting small. Cognition, 48, 781–799. [find](https://scholar.google.com/scholar?q=Elman%2C+J%2E+L%2E+%281993%29%2E+Learning+and+development+in+neural+networks%3A+The+importance+of+starting+small%2E+Cognition%2C+48%2C+781%E2%80%93799%2E)
14. Goldberg, Y., Tsarfaty, R., Adler, M., & Elhadad, M. (2009). Enhancing unlexicalized parsing performance using a wide coverage lexicon, fuzzy tag-set mapping, and EM-HMM-based lexical probabilities. EACL. [find](https://scholar.google.com/scholar?q=Goldberg%2C+Y%2E%2C+Tsarfaty%2C+R%2E%2C+Adler%2C+M%2E%2C+%26+Elhadad%2C+M%2E+%282009%29%2E+Enhancing+unlexicalized+parsing+performance+using+a+wide+coverage+lexicon%2C+fuzzy+tag%2Dset+mapping%2C+and+EM%2DHMM%2Dbased+lexical+probabilities%2E+E)
15. Honkela, T. (1997). Self-organizing maps of words for natural language processing applications. Proceedings of the International ICSC Symposium on Soft Computing. [find](https://scholar.google.com/scholar?q=Honkela%2C+T%2E+%281997%29%2E+Self%2Dorganizing+maps+of+words+for+natural+language+processing+applications%2E+Proceedings+of+the+International+ICSC+Symposium+on+Soft+Computing%2E)
16. Honkela, T., Pulkki, V., & Kohonen, T. (1995). Contextual relations of words in grimm tales, analyzed by self-organizing map. ICANN. [find](https://scholar.google.com/scholar?q=Honkela%2C+T%2E%2C+Pulkki%2C+V%2E%2C+%26+Kohonen%2C+T%2E+%281995%29%2E+Contextual+relations+of+words+in+grimm+tales%2C+analyzed+by+self%2Dorganizing+map%2E+ICANN%2E)
17. Huang, F., & Yates, A. (2009). Distributional representations for handling sparsity in supervised sequence labeling. ACL. [find](https://scholar.google.com/scholar?q=Huang%2C+F%2E%2C+%26+Yates%2C+A%2E+%282009%29%2E+Distributional+representations+for+handling+sparsity+in+supervised+sequence+labeling%2E+ACL%2E)
18. Kaski, S. (1998). Dimensionality reduction by random mapping: Fast similarity computation for clustering. IJCNN (pp. 413–418). [find](https://scholar.google.com/scholar?q=Kaski%2C+S%2E+%281998%29%2E+Dimensionality+reduction+by+random+mapping%3A+Fast+similarity+computation+for+clustering%2E+IJCNN+%28pp%2E+413%E2%80%93418%29%2E)
19. Koo, T., Carreras, X., & Collins, M. (2008). Simple semi-supervised dependency parsing. ACL (pp. 595–603). [find](https://scholar.google.com/scholar?q=Koo%2C+T%2E%2C+Carreras%2C+X%2E%2C+%26+Collins%2C+M%2E+%282008%29%2E+Simple+semi%2Dsupervised+dependency+parsing%2E+ACL+%28pp%2E+595%E2%80%93603%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/koo-2008-simple-semi-supervised-dependency-parsing/)
20. Krishnan, V., & Manning, C. D. (2006). An effective two-stage model for exploiting non-local dependencies in named entity recognition. COLING-ACL. [find](https://scholar.google.com/scholar?q=Krishnan%2C+V%2E%2C+%26+Manning%2C+C%2E+D%2E+%282006%29%2E+An+effective+two%2Dstage+model+for+exploiting+non%2Dlocal+dependencies+in+named+entity+recognition%2E+COLING%2DACL%2E)
21. Landauer, T. K., Foltz, P. W., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 259–284. [find](https://scholar.google.com/scholar?q=Landauer%2C+T%2E+K%2E%2C+Foltz%2C+P%2E+W%2E%2C+%26+Laham%2C+D%2E+%281998%29%2E+An+introduction+to+latent+semantic+analysis%2E+Discourse+Processes%2C+259%E2%80%93284%2E)
22. Li, W., & McCallum, A. (2005). Semi-supervised sequence modeling with syntactic topic models. AAAI. [find](https://scholar.google.com/scholar?q=Li%2C+W%2E%2C+%26+McCallum%2C+A%2E+%282005%29%2E+Semi%2Dsupervised+sequence+modeling+with+syntactic+topic+models%2E+AAAI%2E)
23. Liang, P. (2005). Semi-supervised learning for natural language. Master’s thesis, Massachusetts Institute of Technology. [find](https://scholar.google.com/scholar?q=Liang%2C+P%2E+%282005%29%2E+Semi%2Dsupervised+learning+for+natural+language%2E+Master%E2%80%99s+thesis%2C+Massachusetts+Institute+of+Technology%2E)
24. Lin, D., & Wu, X. (2009). Phrase clustering for discriminative learning. ACL-IJCNLP (pp. 1030–1038). [find](https://scholar.google.com/scholar?q=Lin%2C+D%2E%2C+%26+Wu%2C+X%2E+%282009%29%2E+Phrase+clustering+for+discriminative+learning%2E+ACL%2DIJCNLP+%28pp%2E+1030%E2%80%931038%29%2E)
25. Lund, K., & Burgess, C. (1996). Producing highdimensional semantic spaces from lexical co-occurrence. Behavior Research Methods, Instrumentation, and Computers, 28, 203–208. [find](https://scholar.google.com/scholar?q=Lund%2C+K%2E%2C+%26+Burgess%2C+C%2E+%281996%29%2E+Producing+highdimensional+semantic+spaces+from+lexical+co%2Doccurrence%2E+Behavior+Research+Methods%2C+Instrumentation%2C+and+Computers%2C+28%2C+203%E2%80%93208%2E)
26. Lund, K., Burgess, C., & Atchley, R. A. (1995). Semantic and associative priming in high-dimensional semantic space. Cognitive Science Proceedings, LEA (pp. 660–665). [find](https://scholar.google.com/scholar?q=Lund%2C+K%2E%2C+Burgess%2C+C%2E%2C+%26+Atchley%2C+R%2E+A%2E+%281995%29%2E+Semantic+and+associative+priming+in+high%2Ddimensional+semantic+space%2E+Cognitive+Science+Proceedings%2C+LEA+%28pp%2E+660%E2%80%93665%29%2E)
27. Martin, S., Liermann, J., & Ney, H. (1998). Algorithms for bigram and trigram word clustering. Speech Communication, 24, 19–37. [find](https://scholar.google.com/scholar?q=Martin%2C+S%2E%2C+Liermann%2C+J%2E%2C+%26+Ney%2C+H%2E+%281998%29%2E+Algorithms+for+bigram+and+trigram+word+clustering%2E+Speech+Communication%2C+24%2C+19%E2%80%9337%2E)
28. Miller, S., Guinness, J., & Zamanian, A. (2004). Name tagging with word clusters and discriminative training. HLT-NAACL (pp. 337–342). [find](https://scholar.google.com/scholar?q=Miller%2C+S%2E%2C+Guinness%2C+J%2E%2C+%26+Zamanian%2C+A%2E+%282004%29%2E+Name+tagging+with+word+clusters+and+discriminative+training%2E+HLT%2DNAACL+%28pp%2E+337%E2%80%93342%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/miller-2004-name-tagging-word-clusters-discriminative/)
29. Mnih, A., & Hinton, G. E. (2007). Three new graphical models for statistical language modelling. ICML. [find](https://scholar.google.com/scholar?q=Mnih%2C+A%2E%2C+%26+Hinton%2C+G%2E+E%2E+%282007%29%2E+Three+new+graphical+models+for+statistical+language+modelling%2E+ICML%2E)
30. Mnih, A., & Hinton, G. E. (2009). A scalable hierarchical distributed language model. NIPS (pp. 1081–1088). [find](https://scholar.google.com/scholar?q=Mnih%2C+A%2E%2C+%26+Hinton%2C+G%2E+E%2E+%282009%29%2E+A+scalable+hierarchical+distributed+language+model%2E+NIPS+%28pp%2E+1081%E2%80%931088%29%2E)
31. Morin, F., & Bengio, Y. (2005). Hierarchical probabilistic neural network language model. AISTATS. [find](https://scholar.google.com/scholar?q=Morin%2C+F%2E%2C+%26+Bengio%2C+Y%2E+%282005%29%2E+Hierarchical+probabilistic+neural+network+language+model%2E+AISTATS%2E)
32. Pereira, F., Tishby, N., & Lee, L. (1993). Distributional clustering of english words. ACL (pp. 183–190). [find](https://scholar.google.com/scholar?q=Pereira%2C+F%2E%2C+Tishby%2C+N%2E%2C+%26+Lee%2C+L%2E+%281993%29%2E+Distributional+clustering+of+english+words%2E+ACL+%28pp%2E+183%E2%80%93190%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/pereira-1993-distributional-clustering-english-words/)
33. Ratinov, L., & Roth, D. (2009). Design challenges and misconceptions in named entity recognition. CoNLL. [find](https://scholar.google.com/scholar?q=Ratinov%2C+L%2E%2C+%26+Roth%2C+D%2E+%282009%29%2E+Design+challenges+and+misconceptions+in+named+entity+recognition%2E+CoNLL%2E) [our card](https://md.research.elamur.ai/free/en/papers/ratinov-2009-design-challenges-misconceptions-named-entity/)
34. Ritter, H., & Kohonen, T. (1989). Self-organizing semantic maps. Biological Cybernetics, 241–254. [find](https://scholar.google.com/scholar?q=Ritter%2C+H%2E%2C+%26+Kohonen%2C+T%2E+%281989%29%2E+Self%2Dorganizing+semantic+maps%2E+Biological+Cybernetics%2C+241%E2%80%93254%2E)
35. Sahlgren, M. (2001). Vector-based semantic analysis: Representing word meanings based on random labels. Proceedings of the Semantic Knowledge Acquisition and Categorisation Workshop, ESSLLI. [find](https://scholar.google.com/scholar?q=Sahlgren%2C+M%2E+%282001%29%2E+Vector%2Dbased+semantic+analysis%3A+Representing+word+meanings+based+on+random+labels%2E+Proceedings+of+the+Semantic+Knowledge+Acquisition+and+Categorisation+Workshop%2C+ESSLLI%2E)
36. Sahlgren, M. (2005). An introduction to random indexing. Methods and Applications of Semantic Indexing Workshop at the 7th International Conference on Terminology and Knowledge Engineering (TKE). [find](https://scholar.google.com/scholar?q=Sahlgren%2C+M%2E+%282005%29%2E+An+introduction+to+random+indexing%2E+Methods+and+Applications+of+Semantic+Indexing+Workshop+at+the+7th+International+Conference+on+Terminology+and+Knowledge+Engineering+%28TKE%29%2E)
37. Sahlgren, M. (2006). The word-space model: Using distributional analysis to represent syntagmatic and paradigmatic relations between words in high-dimensional vector spaces. Doctoral dissertation, Stockholm University. [find](https://scholar.google.com/scholar?q=Sahlgren%2C+M%2E+%282006%29%2E+The+word%2Dspace+model%3A+Using+distributional+analysis+to+represent+syntagmatic+and+paradigmatic+relations+between+words+in+high%2Ddimensional+vector+spaces%2E+Doctoral+dissertation%2C+Sto)
38. Sang, E. T., & Buchholz, S. (2000). Introduction to the CoNLL-2000 shared task: Chunking. CoNLL. [find](https://scholar.google.com/scholar?q=Sang%2C+E%2E+T%2E%2C+%26+Buchholz%2C+S%2E+%282000%29%2E+Introduction+to+the+CoNLL%2D2000+shared+task%3A+Chunking%2E+CoNLL%2E) [our card](https://md.research.elamur.ai/free/en/papers/sang-2000-introduction-conll-2000-shared-task/)
39. Schwenk, H., & Gauvain, J.-L. (2002). Connectionist language modeling for large vocabulary continuous speech recognition. International Conference on Acoustics, Speech and Signal Processing (ICASSP) (pp. 765–768). Orlando, Florida. [find](https://scholar.google.com/scholar?q=Schwenk%2C+H%2E%2C+%26+Gauvain%2C+J%2E%2DL%2E+%282002%29%2E+Connectionist+language+modeling+for+large+vocabulary+continuous+speech+recognition%2E+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28ICASSP%29+%28)
40. Sha, F., & Pereira, F. C. N. (2003). Shallow parsing with conditional random fields. HLT-NAACL. [find](https://scholar.google.com/scholar?q=Sha%2C+F%2E%2C+%26+Pereira%2C+F%2E+C%2E+N%2E+%282003%29%2E+Shallow+parsing+with+conditional+random+fields%2E+HLT%2DNAACL%2E)
41. Spitkovsky, V., Alshawi, H., & Jurafsky, D. (2010). From baby steps to leapfrog: How “less is more” in unsupervised dependency parsing. NAACL-HLT. [find](https://scholar.google.com/scholar?q=Spitkovsky%2C+V%2E%2C+Alshawi%2C+H%2E%2C+%26+Jurafsky%2C+D%2E+%282010%29%2E+From+baby+steps+to+leapfrog%3A+How+%E2%80%9Cless+is+more%E2%80%9D+in+unsupervised+dependency+parsing%2E+NAACL%2DHLT%2E)
42. Suzuki, J., & Isozaki, H. (2008). Semi-supervised sequential labeling and segmentation using giga-word scale unlabeled data. ACL-08: HLT (pp. 665–673). [find](https://scholar.google.com/scholar?q=Suzuki%2C+J%2E%2C+%26+Isozaki%2C+H%2E+%282008%29%2E+Semi%2Dsupervised+sequential+labeling+and+segmentation+using+giga%2Dword+scale+unlabeled+data%2E+ACL%2D08%3A+HLT+%28pp%2E+665%E2%80%93673%29%2E)
43. Suzuki, J., Isozaki, H., Carreras, X., & Collins, M. (2009). An empirical study of semi-supervised structured conditional models for dependency parsing. EMNLP. [find](https://scholar.google.com/scholar?q=Suzuki%2C+J%2E%2C+Isozaki%2C+H%2E%2C+Carreras%2C+X%2E%2C+%26+Collins%2C+M%2E+%282009%29%2E+An+empirical+study+of+semi%2Dsupervised+structured+conditional+models+for+dependency+parsing%2E+EMNLP%2E)
44. Turian, J., Ratinov, L., Bengio, Y., & Roth, D. (2009). A preliminary evaluation of word representations for named-entity recognition. NIPS Workshop on Grammar Induction, Representation of Language and Language Learning. [find](https://scholar.google.com/scholar?q=Turian%2C+J%2E%2C+Ratinov%2C+L%2E%2C+Bengio%2C+Y%2E%2C+%26+Roth%2C+D%2E+%282009%29%2E+A+preliminary+evaluation+of+word+representations+for+named%2Dentity+recognition%2E+NIPS+Workshop+on+Grammar+Induction%2C+Representation+of+Language+an)
45. Turney, P. D., & Pantel, P. (2010). From frequency to meaning: Vector space models of semantics. Journal of Artificial Intelligence Research. [find](https://scholar.google.com/scholar?q=Turney%2C+P%2E+D%2E%2C+%26+Pantel%2C+P%2E+%282010%29%2E+From+frequency+to+meaning%3A+Vector+space+models+of+semantics%2E+Journal+of+Artificial+Intelligence+Research%2E)
46. Ushioda, A. (1996). Hierarchical clustering of words. COLING (pp. 1159–1162). [find](https://scholar.google.com/scholar?q=Ushioda%2C+A%2E+%281996%29%2E+Hierarchical+clustering+of+words%2E+COLING+%28pp%2E+1159%E2%80%931162%29%2E)
47. Väyrynen, J., & Honkela, T. (2005). Comparison of independent component analysis and singular value decomposition in word context analysis. AKRR’05, International and Interdisciplinary Conference on Adaptive Knowledge Representation and Reasoning. [find](https://scholar.google.com/scholar?q=Va%CC%88yrynen%2C+J%2E%2C+%26+Honkela%2C+T%2E+%282005%29%2E+Comparison+of+independent+component+analysis+and+singular+value+decomposition+in+word+context+analysis%2E+AKRR%E2%80%9905%2C+International+and+Interdisciplinary+Conference+on+)
48. Väyrynen, J. J., & Honkela, T. (2004). Word category maps based on emergent features created by ICA. Proceedings of the STeP’2004 Cognition + Cybernetics Symposium (pp. 173–185). Finnish Artificial Intelligence Society. [find](https://scholar.google.com/scholar?q=Va%CC%88yrynen%2C+J%2E+J%2E%2C+%26+Honkela%2C+T%2E+%282004%29%2E+Word+category+maps+based+on+emergent+features+created+by+ICA%2E+Proceedings+of+the+STeP%E2%80%992004+Cognition+%2B+Cybernetics+Symposium+%28pp%2E+173%E2%80%93185%29%2E+Finnish+Artificial+I)
49. Väyrynen, J. J., Honkela, T., & Lindqvist, L. (2007). Towards explicit semantic features using independent component analysis. Proceedings of the Workshop Semantic Content Acquisition and Representation (SCAR). Stockholm, Sweden: Swedish Institute of Computer Science. [find](https://scholar.google.com/scholar?q=Va%CC%88yrynen%2C+J%2E+J%2E%2C+Honkela%2C+T%2E%2C+%26+Lindqvist%2C+L%2E+%282007%29%2E+Towards+explicit+semantic+features+using+independent+component+analysis%2E+Proceedings+of+the+Workshop+Semantic+Content+Acquisition+and+Representat)
50. Řehůřek, R., & Sojka, P. (2010). Software framework for topic modelling with large corpora. LREC. [find](https://scholar.google.com/scholar?q=R%CC%8Cehu%CC%8Ar%CC%8Cek%2C+R%2E%2C+%26+Sojka%2C+P%2E+%282010%29%2E+Software+framework+for+topic+modelling+with+large+corpora%2E+LREC%2E)
51. Zhang, T., & Johnson, D. (2003). A robust risk minimization based named entity recognition system. CoNLL. [find](https://scholar.google.com/scholar?q=Zhang%2C+T%2E%2C+%26+Johnson%2C+D%2E+%282003%29%2E+A+robust+risk+minimization+based+named+entity+recognition+system%2E+CoNLL%2E)
52. Zhao, H., Chen, W., Kit, C., & Zhou, G. (2009). Multilingual dependency learning: a huge feature engineering method to semantic dependency parsing. CoNLL (pp. 55–60). [find](https://scholar.google.com/scholar?q=Zhao%2C+H%2E%2C+Chen%2C+W%2E%2C+Kit%2C+C%2E%2C+%26+Zhou%2C+G%2E+%282009%29%2E+Multilingual+dependency+learning%3A+a+huge+feature+engineering+method+to+semantic+dependency+parsing%2E+CoNLL+%28pp%2E+55%E2%80%9360%29%2E)