---
title: "Finding Function in Form — Ling 2015"
description: "Most neural NLP systems give each word type its own independently learned vector."
authors: ["Wang Ling", "Tiago Luis", "Luis Marujo", "Ramon Fernandez Astudillo", "Silvio Amir", "Chris Dyer", "Alan W. Black", "Isabel Trancoso"]
year: 2015
venue: "arXiv:1508.02096"
source: "https://arxiv.org/abs/1508.02096"
doi: "10.48550/arxiv.1508.02096"
arxiv: "1508.02096"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ling-2015-finding-function-form-compositional-character/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ling-2015-finding-function-form-compositional-character/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2015](https://md.research.elamur.ai/free/en/papers/2015/) / Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation

# Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation

**Authors:** [Wang Ling](https://md.research.elamur.ai/free/en/authors/wang-ling/), Tiago Luis, Luis Marujo, Ramon Fernandez Astudillo, [Silvio Amir](https://md.research.elamur.ai/free/en/authors/silvio-amir/), [Chris Dyer](https://md.research.elamur.ai/free/en/authors/chris-dyer/), [Alan W. Black](https://md.research.elamur.ai/free/en/authors/alan-w-black/), [Isabel Trancoso](https://md.research.elamur.ai/free/en/authors/isabel-trancoso/) · **Year:** 2015 · **Venue:** arXiv:1508.02096 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1508.02096) · [PDF](https://arxiv.org/pdf/1508.02096) · [DOI 10.48550/arxiv.1508.02096](https://doi.org/10.48550/arxiv.1508.02096) · [arXiv 1508.02096](https://arxiv.org/abs/1508.02096) · [OpenAlex](https://openalex.org/W2949563612)

**Citations:** 116 citations (openalex-snapshot · 2026-10-14) · [185](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Most neural NLP systems give each word type its own independently learned vector. This ignores the shared structure of related forms, offers nothing for words unseen in training, and makes parameter counts grow with vocabulary size. These problems are especially acute for morphologically rich languages such as Turkish.

The authors propose C2W, which builds a word's embedding from its spelling. Characters are mapped to small vectors, two LSTMs read them in opposite directions, and their final states are linearly merged into a word vector. C2W takes the same input and produces the same output as a word lookup table, so it can replace one in existing architectures. Embeddings of frequent words can be cached to save computation.

The model was evaluated on language modeling and POS tagging in several languages, including the English Penn Treebank. Despite using only a fraction of the parameters, it matched or beat lookup-table baselines, with the largest gains in morphologically rich languages, and it reached a new best English PTB tagging accuracy. This suggests that much manual lexical feature engineering can be learned from characters, giving compact, open-vocabulary representations.

## Key points
- Architecture: a character lookup table feeds a bidirectional LSTM, and the word vector is a learned linear combination of the last forward and backward states. The model stores one vector per character plus fixed composition weights instead of one vector per word.
- English PTB POS tagging accuracy:
  - Without extra features or external data, C2W scores 97.36 versus 96.70 for a word lookup table.
  - With hand-crafted features, C2W reaches 97.57.
  - C2W(tanh)+word (sskip) reaches 97.78, 0.28 points above the previous best system, SCCN (97.50).
- As motivation, the authors note that an English Wikipedia dump of 60M sentences contains roughly 20M distinct word types after lowercasing and tokenization, and a lookup table would have to assign a separate vector to every one of them.
- The learned embeddings place functionally similar but differently spelled words close together, such as October and January. The model can also represent words never seen in training.
- Caching frequent words trades memory for speed. The approach was later applied to dependency parsing (Ballesteros et al., 2015), with similar gains in morphologically rich languages.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

## Cited by

Cited by

185 papers of our library cite it (of the 140673 whose bibliographies we read); 24 of them have a page here, 161 not yet.

- [LSTM CCG Parsing](https://md.research.elamur.ai/free/en/papers/aclbulk-2016-lewis-lstm-ccg-parsing/)
- [Morphological Word Embeddings for Arabic Neural Machine Translation in Low-Resource Settings](https://md.research.elamur.ai/free/en/papers/aclbulk-2018-shapiro-morphological-word-embeddings-arabic-neural/)
- [Data Augmentation via Dependency Tree Morphing for Low-Resource Languages](https://md.research.elamur.ai/free/en/papers/ahin-2018-data-augmentation-dependency-tree-morphing/)
- [Globally Normalized Transition-Based Neural Networks](https://md.research.elamur.ai/free/en/papers/andor-2016-globally-normalized-transition-based-neural/)
- [Improved Transition-based Parsing by Modeling Characters instead of Words with LSTMs](https://md.research.elamur.ai/free/en/papers/ballesteros-2015-improved-transition-based-parsing-modeling/)
- [Named Entity Recognition with Bidirectional LSTM-CNNs](https://md.research.elamur.ai/free/en/papers/chiu-2015-named-entity-recognition-bidirectional-lstm/)
- [A Character-level Decoder without Explicit Segmentation for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/chung-2016-character-level-decoder-without-explicit/)
- [Representations and Architectures in Neural Sentiment Analysis for Morphologically Rich Languages: A Case Study from Modern Hebrew](https://md.research.elamur.ai/free/en/papers/coling-2018-amram-representations-architectures-neural-sentiment-analysis/)
- [A Neural Model for Part-of-Speech Tagging in Historical Texts](https://md.research.elamur.ai/free/en/papers/hardmeier-2016-neural-model-part-speech-tagging/)
- [A Joint Many-Task Model: Growing a Neural Network for Multiple NLP Tasks](https://md.research.elamur.ai/free/en/papers/hashimoto-2016-joint-many-task-model-growing/)
- [Character-Aware Neural Language Models](https://md.research.elamur.ai/free/en/papers/kim-2015-character-aware-neural-language-models/)
- [Achieving Open Vocabulary Neural Machine Translation with Hybrid Word-Character Models](https://md.research.elamur.ai/free/en/papers/luong-2016-achieving-open-vocabulary-neural-machine/)
- [End-to-end Sequence Labeling via Bi-directional LSTM-CNNs-CRF](https://md.research.elamur.ai/free/en/papers/ma-2016-end-end-sequence-labeling-bi/)
- [Multilingual Part-of-Speech Tagging with Bidirectional Long Short-Term Memory Models and Auxiliary Loss](https://md.research.elamur.ai/free/en/papers/plank-2016-multilingual-part-speech-tagging-bidirectional/)
- [Implicit Discourse Relation Recognition with Context-aware Character-enhanced Embeddings](https://md.research.elamur.ai/free/en/papers/qin-2016-implicit-discourse-relation-recognition-context/)
- [Neural Architectures for Nested NER through Linearization](https://md.research.elamur.ai/free/en/papers/strakova-2019-neural-architectures-nested-ner-through/)
- [Fast and Accurate Entity Recognition with Iterated Dilated Convolutions](https://md.research.elamur.ai/free/en/papers/strubell-2017-fast-accurate-entity-recognition-iterated/)
- [Stanford's Graph-based Neural Dependency Parser at the CoNLL 2017 Shared Task](https://md.research.elamur.ai/free/en/papers/ud-2017-dozat-stanford-s-graph-based-neural/)
- [UDPipe 2.0 Prototype at CoNLL 2018 UD Shared Task](https://md.research.elamur.ai/free/en/papers/ud-2018-straka-udpipe-2-0-prototype-conll/)
- [75 Languages, 1 Model: Parsing Universal Dependencies Universally](https://md.research.elamur.ai/free/en/papers/ud-2019-kondratyuk-75-languages-1-model-parsing/)
- [A Pointer Network Architecture for Joint Morphological Segmentation and Tagging](https://md.research.elamur.ai/free/en/papers/ud-2020-seker-pointer-network-architecture-joint-morphological/)
- [From Shakespeare to Twitter: What are Language Styles all about?](https://md.research.elamur.ai/free/en/papers/xu-2017-shakespeare-twitter-what-are-language/)
- [A Survey on Recent Advances in Named Entity Recognition from Deep Learning models](https://md.research.elamur.ai/free/en/papers/yadav-2018-survey-recent-advances-named-entity/)
- [Hierarchical Attention Networks for Document Classification](https://md.research.elamur.ai/free/en/papers/yang-2016-hierarchical-attention-networks-document-classification/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{ling2015finding,
  title = {{Finding Function in Form: Compositional Character Models for Open Vocabulary Word Representation}},
  author = {Ling, Wang and Luis, Tiago and Marujo, Luis and Astudillo, Ramon Fernandez and Amir, Silvio and Dyer, Chris and Black, Alan W. and Trancoso, Isabel},
  year = {2015},
  doi = {10.48550/arxiv.1508.02096},
  eprint = {1508.02096},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1508.02096}
}
```

- [← previous FDTB1, première étape du projet « French Discourse Treebank » : repérage des connecteurs de discours en corpus](https://md.research.elamur.ai/free/en/papers/danlos-2015-fdtb1-premiere-etape-projet-french/)
- [next → Findings of the 2015 Workshop on Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/bojar-2015-findings-2015-workshop-statistical-machine/)
- [↑ back to papers of 2015](https://md.research.elamur.ai/free/en/papers/2015/)

## References

39 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Susana Afonso, Eckhard Bick, Renato Haber, and Diana Santos. 2002. “Floresta sintá(c)tica”: a treebank for Portuguese. In Proc. LREC. [find](https://scholar.google.com/scholar?q=Susana+Afonso%2C+Eckhard+Bick%2C+Renato+Haber%2C+and+Diana+Santos%2E+2002%2E+%E2%80%9CFloresta+sinta%CC%81%28c%29tica%E2%80%9D%3A+a+treebank+for+Portuguese%2E+In+Proc%2E+LREC%2E) [our card](https://md.research.elamur.ai/free/en/papers/afonso-2002-floresta-sinta-c-tica-treebank/)
2. Nart B. Atalay, Kemal Oflazer, and Bilge Say. 2003. The annotation process in the Turkish treebank. In In Proc. the 4th International Workshop on Linguistically Interpreted Corpora (LINC). [find](https://scholar.google.com/scholar?q=Nart+B%2E+Atalay%2C+Kemal+Oflazer%2C+and+Bilge+Say%2E+2003%2E+The+annotation+process+in+the+Turkish+treebank%2E+In+In+Proc%2E+the+4th+International+Workshop+on+Linguistically+Interpreted+Corpora+%28LINC%29%2E) [our card](https://md.research.elamur.ai/free/en/papers/atalay-2003-annotation-process-turkish-treebank/)
3. Miguel Ballesteros, Chris Dyer, and Noah A. Smith. 2015. Improved transition-based parsing by modeling characters instead of words with LSTMs. In Proc. EMNLP. [find](https://scholar.google.com/scholar?q=Miguel+Ballesteros%2C+Chris+Dyer%2C+and+Noah+A%2E+Smith%2E+2015%2E+Improved+transition%2Dbased+parsing+by+modeling+characters+instead+of+words+with+LSTMs%2E+In+Proc%2E+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/ballesteros-2015-improved-transition-based-parsing-modeling/)
4. Jan A. Botha and Phil Blunsom. 2014. Compositional morphology for word representations and language modelling. In Proc. ICML. [find](https://scholar.google.com/scholar?q=Jan+A%2E+Botha+and+Phil+Blunsom%2E+2014%2E+Compositional+morphology+for+word+representations+and+language+modelling%2E+In+Proc%2E+ICML%2E)
5. Sabine Brants, Stefanie Dipper, Silvia Hansen, Wolfgang Lezius, and George Smith. 2002. The TIGER treebank. [find](https://scholar.google.com/scholar?q=Sabine+Brants%2C+Stefanie+Dipper%2C+Silvia+Hansen%2C+Wolfgang+Lezius%2C+and+George+Smith%2E+2002%2E+The+TIGER+treebank%2E)
6. Danqi Chen and Christopher D. Manning. 2014. A fast and accurate dependency parser using neural networks. In Proc. EMNLP. [find](https://scholar.google.com/scholar?q=Danqi+Chen+and+Christopher+D%2E+Manning%2E+2014%2E+A+fast+and+accurate+dependency+parser+using+neural+networks%2E+In+Proc%2E+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/missing-2014-chen-fast-accurate-dependency-parser-neural/)
7. Xinxiong Chen, Lei Xu, Zhiyuan Liu, Maosong Sun, and Huanbo Luan. 2015. Joint learning of character and word embeddings. [find](https://scholar.google.com/scholar?q=Xinxiong+Chen%2C+Lei+Xu%2C+Zhiyuan+Liu%2C+Maosong+Sun%2C+and+Huanbo+Luan%2E+2015%2E+Joint+learning+of+character+and+word+embeddings%2E)
8. Kyunghyun Cho, Bart van Merrienboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. 2014. Learning phrase representations using RNN encoder–decoder for statistical machine translation. In Proc. EMNLP. [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2C+Bart+van+Merrienboer%2C+Caglar+Gulcehre%2C+Dzmitry+Bahdanau%2C+Fethi+Bougares%2C+Holger+Schwenk%2C+and+Yoshua+Bengio%2E+2014%2E+Learning+phrase+representations+using+RNN+encoder%E2%80%93decoder+for+statistic)
9. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. JMLR. [find](https://scholar.google.com/scholar?q=Ronan+Collobert%2C+Jason+Weston%2C+Le%CC%81on+Bottou%2C+Michael+Karlen%2C+Koray+Kavukcuoglu%2C+and+Pavel+Kuksa%2E+2011%2E+Natural+language+processing+%28almost%29+from+scratch%2E+JMLR%2E) [our card](https://md.research.elamur.ai/free/en/papers/collobert-2011-natural-language-processing-almost-scratch/)
10. Ferdinand de Saussure. Course in General Linguistics. 1916. [find](https://scholar.google.com/scholar?q=Ferdinand+de+Saussure%2E+Course+in+General+Linguistics%2E+1916%2E)
11. Alex Graves and Jürgen Schmidhuber. 2005. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Networks. [find](https://scholar.google.com/scholar?q=Alex+Graves+and+Ju%CC%88rgen+Schmidhuber%2E+2005%2E+Framewise+phoneme+classification+with+bidirectional+LSTM+and+other+neural+network+architectures%2E+Neural+Networks%2E)
12. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Comput., 9(8). [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+Comput%2E%2C+9%288%29%2E)
13. Nal Kalchbrenner and Phil Blunsom. 2013. Recurrent continuous translation models. In Proc. EMNLP. [find](https://scholar.google.com/scholar?q=Nal+Kalchbrenner+and+Phil+Blunsom%2E+2013%2E+Recurrent+continuous+translation+models%2E+In+Proc%2E+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2013-recurrent-continuous-translation-models/)
14. Byung-Ju Kang and Key-Sun Choi. 2000. Automatic transliteration and back-transliteration by decision tree learning. In LREC. [find](https://scholar.google.com/scholar?q=Byung%2DJu+Kang+and+Key%2DSun+Choi%2E+2000%2E+Automatic+transliteration+and+back%2Dtransliteration+by+decision+tree+learning%2E+In+LREC%2E)
15. Jiwei Li, Dan Jurafsky, and Eduard H. Hovy. 2015. When are tree structures necessary for deep learning of representations? CoRR, abs/1503.00185. [find](https://scholar.google.com/scholar?q=Jiwei+Li%2C+Dan+Jurafsky%2C+and+Eduard+H%2E+Hovy%2E+2015%2E+When+are+tree+structures+necessary+for+deep+learning+of+representations%3F+CoRR%2C+abs%2F1503%2E00185%2E)
16. Wang Ling, João Graça, Isabel Trancoso, and Alan Black. 2012. Entropy-based pruning for phrase-based machine translation. In Proc. EMNLP. [find](https://scholar.google.com/scholar?q=Wang+Ling%2C+Joa%CC%83o+Grac%CC%A7a%2C+Isabel+Trancoso%2C+and+Alan+Black%2E+2012%2E+Entropy%2Dbased+pruning+for+phrase%2Dbased+machine+translation%2E+In+Proc%2E+EMNLP%2E)
17. Wang Ling, Chris Dyer, Alan W Black, and Isabel Trancoso. 2013. Paraphrasing 4 microblog normalization. In Proc. EMNLP. [find](https://scholar.google.com/scholar?q=Wang+Ling%2C+Chris+Dyer%2C+Alan+W+Black%2C+and+Isabel+Trancoso%2E+2013%2E+Paraphrasing+4+microblog+normalization%2E+In+Proc%2E+EMNLP%2E)
18. Wang Ling, Chris Dyer, Alan Black, and Isabel Trancoso. 2015. Two/too simple adaptations of word2vec for syntax problems. In Proc. NAACL. [find](https://scholar.google.com/scholar?q=Wang+Ling%2C+Chris+Dyer%2C+Alan+Black%2C+and+Isabel+Trancoso%2E+2015%2E+Two%2Ftoo+simple+adaptations+of+word2vec+for+syntax+problems%2E+In+Proc%2E+NAACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/ling-2015-two-too-simple-adaptations-word2vec/)
19. Shujie Liu, Nan Yang, Mu Li, and Ming Zhou. 2014. A recursive recurrent neural network for statistical machine translation. In Proc. ACL. [find](https://scholar.google.com/scholar?q=Shujie+Liu%2C+Nan+Yang%2C+Mu+Li%2C+and+Ming+Zhou%2E+2014%2E+A+recursive+recurrent+neural+network+for+statistical+machine+translation%2E+In+Proc%2E+ACL%2E)
20. Thang Luong, Richard Socher, and Christopher Manning. 2013. Better word representations with recursive neural networks for morphology. In Proc. CoNLL. [find](https://scholar.google.com/scholar?q=Thang+Luong%2C+Richard+Socher%2C+and+Christopher+Manning%2E+2013%2E+Better+word+representations+with+recursive+neural+networks+for+morphology%2E+In+Proc%2E+CoNLL%2E) [our card](https://md.research.elamur.ai/free/en/papers/luong-2013-better-word-representations-recursive-neural/)
21. Christopher D. Manning. 2011. Part-of-speech tagging from 97% to 100%: Is it time for some linguistics? In Proc. CICLing. [find](https://scholar.google.com/scholar?q=Christopher+D%2E+Manning%2E+2011%2E+Part%2Dof%2Dspeech+tagging+from+97%25+to+100%25%3A+Is+it+time+for+some+linguistics%3F+In+Proc%2E+CICLing%2E) [our card](https://md.research.elamur.ai/free/en/papers/enud-2011-manning-part-speech-tagging-97-100/)
22. Mitchell P. Marcus, Mary Ann Marcinkiewicz, and Beatrice Santorini. 1993. Building a large annotated corpus of English: The Penn treebank. Comput. Linguist. [find](https://scholar.google.com/scholar?q=Mitchell+P%2E+Marcus%2C+Mary+Ann+Marcinkiewicz%2C+and+Beatrice+Santorini%2E+1993%2E+Building+a+large+annotated+corpus+of+English%3A+The+Penn+treebank%2E+Comput%2E+Linguist%2E) [our card](https://md.research.elamur.ai/free/en/papers/bank-1993-marcus-building-large-annotated-corpus-english/)
23. William Marslen-Wilson and Lorraine K. Tyler. 1998. Rules, representations, and the English past tense. Trends in Cognitive Science, 2(11). [find](https://scholar.google.com/scholar?q=William+Marslen%2DWilson+and+Lorraine+K%2E+Tyler%2E+1998%2E+Rules%2C+representations%2C+and+the+English+past+tense%2E+Trends+in+Cognitive+Science%2C+2%2811%29%2E)
24. M. Antonia Martı́, Mariona Taulé, Lluı́s Márquez, and Manuel Bertran. 2007. CESS-ECE: A multilingual and multilevel annotated corpus. In Proc. LREC. [find](https://scholar.google.com/scholar?q=M%2E+Antonia+Mart%C4%B1%CC%81%2C+Mariona+Taule%CC%81%2C+Llu%C4%B1%CC%81s+Ma%CC%81rquez%2C+and+Manuel+Bertran%2E+2007%2E+CESS%2DECE%3A+A+multilingual+and+multilevel+annotated+corpus%2E+In+Proc%2E+LREC%2E)
25. Tomas Mikolov, Martin Karafiát, Lukas Burget, Jan Cernockỳ, and Sanjeev Khudanpur. 2010. Recurrent neural network based language model. In Proc. Interspeech. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Martin+Karafia%CC%81t%2C+Lukas+Burget%2C+Jan+Cernocky%CC%80%2C+and+Sanjeev+Khudanpur%2E+2010%2E+Recurrent+neural+network+based+language+model%2E+In+Proc%2E+Interspeech%2E)
26. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S. Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proc. NIPS. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Ilya+Sutskever%2C+Kai+Chen%2C+Greg+S%2E+Corrado%2C+and+Jeff+Dean%2E+2013%2E+Distributed+representations+of+words+and+phrases+and+their+compositionality%2E+In+Proc%2E+NIPS%2E)
27. Robert C. Moore and Chris Quirk. 2009. Less is more: significance-based ngram selection for smaller, better language models. In Proc. EMNLP. [find](https://scholar.google.com/scholar?q=Robert+C%2E+Moore+and+Chris+Quirk%2E+2009%2E+Less+is+more%3A+significance%2Dbased+ngram+selection+for+smaller%2C+better+language+models%2E+In+Proc%2E+EMNLP%2E)
28. Thomas Mueller, Helmut Schmid, and Hinrich Schütze. 2013. Efficient higher-order CRFs for morphological tagging. In Proc. EMNLP. [find](https://scholar.google.com/scholar?q=Thomas+Mueller%2C+Helmut+Schmid%2C+and+Hinrich+Schu%CC%88tze%2E+2013%2E+Efficient+higher%2Dorder+CRFs+for+morphological+tagging%2E+In+Proc%2E+EMNLP%2E)
29. Tetsuji Nakagawa, Taku Kudoh, and Yuji Matsumoto. 2001. Unknown word guessing and part-of-speech tagging using support vector machines. In In Proc. the Sixth Natural Language Processing Pacific Rim Symposium. [find](https://scholar.google.com/scholar?q=Tetsuji+Nakagawa%2C+Taku+Kudoh%2C+and+Yuji+Matsumoto%2E+2001%2E+Unknown+word+guessing+and+part%2Dof%2Dspeech+tagging+using+support+vector+machines%2E+In+In+Proc%2E+the+Sixth+Natural+Language+Processing+Pacific+Rim+Sy)
30. Cicero D. Santos and Bianca Zadrozny. 2014. Learning character-level representations for part-of-speech tagging. In Proc. ICML. [find](https://scholar.google.com/scholar?q=Cicero+D%2E+Santos+and+Bianca+Zadrozny%2E+2014%2E+Learning+character%2Dlevel+representations+for+part%2Dof%2Dspeech+tagging%2E+In+Proc%2E+ICML%2E)
31. Kristie Seymore and Ronald Rosenfeld. 1996. Scalable backoff language models. In Proc. ICSLP. [find](https://scholar.google.com/scholar?q=Kristie+Seymore+and+Ronald+Rosenfeld%2E+1996%2E+Scalable+backoff+language+models%2E+In+Proc%2E+ICSLP%2E)
32. Anders Søgaard. 2011. Semisupervised condensed nearest neighbor for part-of-speech tagging. In Proc. ACL. [find](https://scholar.google.com/scholar?q=Anders+S%C3%B8gaard%2E+2011%2E+Semisupervised+condensed+nearest+neighbor+for+part%2Dof%2Dspeech+tagging%2E+In+Proc%2E+ACL%2E)
33. Radu Soricut and Franz Och. 2015. Unsupervised morphology induction using word embeddings. In Proc. NAACL. [find](https://scholar.google.com/scholar?q=Radu+Soricut+and+Franz+Och%2E+2015%2E+Unsupervised+morphology+induction+using+word+embeddings%2E+In+Proc%2E+NAACL%2E)
34. Drahomı́ra Spoustová, Jan Hajič, Jan Raab, and Miroslav Spousta. 2009. Semi-supervised training for the averaged perceptron POS tagger. In Proc. EACL. [find](https://scholar.google.com/scholar?q=Drahom%C4%B1%CC%81ra+Spoustova%CC%81%2C+Jan+Hajic%CC%8C%2C+Jan+Raab%2C+and+Miroslav+Spousta%2E+2009%2E+Semi%2Dsupervised+training+for+the+averaged+perceptron+POS+tagger%2E+In+Proc%2E+EACL%2E)
35. Andreas Stolcke. 1998. Entropy-based pruning of backoff language models. In In Proc. DARPA Broadcast News Transcription and Understanding Workshop. [find](https://scholar.google.com/scholar?q=Andreas+Stolcke%2E+1998%2E+Entropy%2Dbased+pruning+of+backoff+language+models%2E+In+In+Proc%2E+DARPA+Broadcast+News+Transcription+and+Understanding+Workshop%2E)
36. Xu Sun. 2014. Structure regularization for structured prediction: Theories and experiments. CoRR, abs/1411.6243. — [arXiv:1411.6243](https://arxiv.org/abs/1411.6243) [find](https://scholar.google.com/scholar?q=Xu+Sun%2E+2014%2E+Structure+regularization+for+structured+prediction%3A+Theories+and+experiments%2E+CoRR%2C+abs%2F1411%2E6243%2E+%E2%80%94+%5BarXiv%3A1411%2E6243%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1411%2E6243%29)
37. Martin Sundermeyer, Ralf Schlüter, and Hermann Ney. 2012. LSTM neural networks for language modeling. In Proc. Interspeech. [find](https://scholar.google.com/scholar?q=Martin+Sundermeyer%2C+Ralf+Schlu%CC%88ter%2C+and+Hermann+Ney%2E+2012%2E+LSTM+neural+networks+for+language+modeling%2E+In+Proc%2E+Interspeech%2E)
38. Wei Xu, Alan Ritter, and Ralph Grishman. 2013. Gathering and generating paraphrases from twitter with application to normalization. In Proceedings of the Sixth Workshop on Building and Using Comparable Corpora. [find](https://scholar.google.com/scholar?q=Wei+Xu%2C+Alan+Ritter%2C+and+Ralph+Grishman%2E+2013%2E+Gathering+and+generating+paraphrases+from+twitter+with+application+to+normalization%2E+In+Proceedings+of+the+Sixth+Workshop+on+Building+and+Using+Comparabl)
39. Richard Zens, Daisy Stanton, and Peng Xu. 2012. A systematic comparison of phrase table pruning techniques. In Proc. EMNLP. [find](https://scholar.google.com/scholar?q=Richard+Zens%2C+Daisy+Stanton%2C+and+Peng+Xu%2E+2012%2E+A+systematic+comparison+of+phrase+table+pruning+techniques%2E+In+Proc%2E+EMNLP%2E)