--- title: "Deep Convolutional Neural Networks for — Santos 2014" description: "The paper addresses sentiment classification of very short texts, such as single sentences and tweets, where little context is available and bag-of-words…" authors: ["Cícero dos Santos", "Maíra Gatti"] year: 2014 venue: "Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers" source: "https://aclanthology.org/C14-1008/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/santos-2014-deep-convolutional-neural-networks-sentiment/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/santos-2014-deep-convolutional-neural-networks-sentiment/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2014](https://md.research.elamur.ai/free/en/papers/2014/) / Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts # Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts **Authors:** [Cícero dos Santos](https://md.research.elamur.ai/free/en/authors/cicero-nogueira-dos-santos/), Maíra Gatti · **Institutions:** [IBM](https://md.research.elamur.ai/free/en/companies/ibm/) · **Year:** 2014 · **Venue:** Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/C14-1008/) · [PDF](https://aclanthology.org/C14-1008.pdf) · [OpenAlex](https://openalex.org/W2252215182) **Citations:** 1163 citations (OpenAlex, 2026-10-14) · [106](https://md.research.elamur.ai/free/en/papers/santos-2014-deep-convolutional-neural-networks-sentiment/cited-by/) in our library, of the 140884 papers whose bibliographies we read ## Summary The paper addresses sentiment classification of very short texts, such as single sentences and tweets, where little context is available and bag-of-words features fall short. The authors introduce CharSCNN (Character to Sentence Convolutional Neural Network). It is a feed-forward model with two convolutional layers. The first builds a word representation from its characters, and the second combines word windows into a sentence vector. Both layers use max pooling, so inputs of any length map to fixed-size vectors. Character-derived features are concatenated with word embeddings, which can be initialized from unsupervised pre-training in the word2vec style (Mikolov et al., 2013). The model is evaluated on two corpora: the Stanford Sentiment Treebank (SSTb, movie-review sentences) and the Stanford Twitter Sentiment corpus (STS). The authors report state-of-the-art accuracy on both: 85.7% binary and 48.3% fine-grained on SSTb, and 86.4% on STS. The work matters because it matches or beats the parser-dependent RNTN without needing any syntactic input. ## Key points - On SSTb fine-grained (five-class) prediction, CharSCNN improves on RNTN (Socher et al., 2013b) by 2.6 absolute points. Binary accuracy is about the same for CharSCNN, SCNN (the word-only variant) and RNTN. - On STS, CharSCNN reaches 86.4 with pre-trained embeddings, compared with: - SCNN: 85.2 - LProp (Speriosu et al., 2011): 84.7 - MaxEnt, NB and SVM from Go et al. (2009): 83.0, 82.7 and 82.2 - Unsupervised pre-training of word embeddings adds about 1.5 accuracy points on SSTb and about 4.5 on STS, compared with random initialization. - Character-level features help most on tweets, adding 1.2 points over SCNN on STS when pre-training is used. - Implementation and cost: - The model is built in Theano, with backpropagation done through automatic differentiation. - Training uses SGD on negative log-likelihood with a softmax output. - One epoch on SSTb (all phrases, five classes) takes roughly 10 minutes on a 4-thread Xeon machine. ## Interesting ideas - **Character convolution with max pooling as a word encoder.** Running a convolution over character windows and keeping the maximum per filter gives a fixed-length vector for any word. This captures affixes like "-ly" and pieces of hashtags such as "#SoSad". It could help morphologically rich languages like Ukrainian, or noisy user text full of rare forms. - **Reusing one pooling pattern at two levels.** The same convolution-plus-max idea is applied first to characters within words and then to words within sentences. This gives a simple, parser-free way to handle variable-length input at every level. It is attractive where good syntactic parsers are unavailable. - **Interpreting the model by counting pooling winners.** The authors count how many of the 300 max-pooled sentence features come from each word. This reveals what the model focuses on: in their example, "definitely not dull" accounts for 77% of the features. It is a cheap diagnostic for checking negation handling or other phenomena in any max-pooled CNN. - **Running ablations along two axes.** Comparing with and without character features, and with pre-trained versus random embeddings, cleanly separates the contribution of each component per domain. The ablations show both components matter much more for tweets, which is a useful evaluation template for new datasets. - **Training on labeled phrases, not just full sentences.** The authors suggest that sentiment-labeled phrases teach the model how parts combine into a sentence's sentiment, though they say this still needs confirmation. The idea may be worth testing wherever treebank-style phrase annotations exist. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) Topics: [Sentiment & emotion](https://md.research.elamur.ai/free/en/topics/sentiment-emotion/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{santos2014deep, title = {{Deep Convolutional Neural Networks for Sentiment Analysis of Short Texts}}, author = {Santos, Cícero dos and Gatti, Maíra}, booktitle = {Proceedings of COLING 2014, the 25th International Conference on Computational Linguistics: Technical Papers}, year = {2014}, url = {https://aclanthology.org/C14-1008/} } ``` - [← previous ReferItGame: Referring to Objects in Photographs of Natural Scenes](https://md.research.elamur.ai/free/en/papers/kazemzadeh-2014-referitgame-referring-objects-photographs-natural/) - [next → Adaptive Recursive Neural Network for Target-dependent Twitter Sentiment Classification](https://md.research.elamur.ai/free/en/papers/dong-2014-adaptive-recursive-neural-network-target/) - [↑ back to papers of 2014](https://md.research.elamur.ai/free/en/papers/2014/) ## References 7 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Andrei Alexandrescu and Katrin Kirchhoff. 2006. Factored neural language models. In Proceedings of the Human Language Technology Conference of the NAACL, pages 1–4, New York City, USA, June. [find](https://scholar.google.com/scholar?q=Andrei+Alexandrescu+and+Katrin+Kirchhoff%2E+2006%2E+Factored+neural+language+models%2E+In+Proceedings+of+the+Human+Language+Technology+Conference+of+the+NAACL%2C+pages+1%E2%80%934%2C+New+York+City%2C+USA%2C+June%2E) 2. Luciano Barbosa and Junlan Feng. 2010. Robust sentiment detection on twitter from biased and noisy data. In Proceedings of the 23rd International Conference on Computational Linguistics, pages 36–44. [find](https://scholar.google.com/scholar?q=Luciano+Barbosa+and+Junlan+Feng%2E+2010%2E+Robust+sentiment+detection+on+twitter+from+biased+and+noisy+data%2E+In+Proceedings+of+the+23rd+International+Conference+on+Computational+Linguistics%2C+pages+36%E2%80%9344%2E) [our card](https://md.research.elamur.ai/free/en/papers/barbosa-2010-robust-sentiment-detection-twitter-biased/) 3. James Bergstra, Olivier Breuleux, Frédéric Bastien, Pascal Lamblin, Razvan Pascanu, Guillaume Desjardins, Joseph Turian, David Warde-Farley, and Yoshua Bengio. 2010. Theano: a CPU and GPU math expression compiler. In Proceedings of the Python for Scientific Computing Conference (SciPy). [find](https://scholar.google.com/scholar?q=James+Bergstra%2C+Olivier+Breuleux%2C+Fre%CC%81de%CC%81ric+Bastien%2C+Pascal+Lamblin%2C+Razvan+Pascanu%2C+Guillaume+Desjardins%2C+Joseph+Turian%2C+David+Warde%2DFarley%2C+and+Yoshua+Bengio%2E+2010%2E+Theano%3A+a+CPU+and+GPU+math+expre) 4. Grzegorz Chrupala. 2013. Text segmentation with character-level text embeddings. In Proceedings of the ICML workshop on Deep Learning for Audio, Speech and Language Processing. [find](https://scholar.google.com/scholar?q=Grzegorz+Chrupala%2E+2013%2E+Text+segmentation+with+character%2Dlevel+text+embeddings%2E+In+Proceedings+of+the+ICML+workshop+on+Deep+Learning+for+Audio%2C+Speech+and+Language+Processing%2E) 5. R. Collobert, J. Weston, L. Bottou, M. Karlen, K. Kavukcuoglu, and P. Kuksa. 2011. Natural language processing (almost) from scratch. Journal of Machine Learning Research, 12:2493–2537. [find](https://scholar.google.com/scholar?q=R%2E+Collobert%2C+J%2E+Weston%2C+L%2E+Bottou%2C+M%2E+Karlen%2C+K%2E+Kavukcuoglu%2C+and+P%2E+Kuksa%2E+2011%2E+Natural+language+processing+%28almost%29+from+scratch%2E+Journal+of+Machine+Learning+Research%2C+12%3A2493%E2%80%932537%2E) [our card](https://md.research.elamur.ai/free/en/papers/collobert-2011-natural-language-processing-almost-scratch/) 6. R. Collobert. 2011. Deep learning for efficient discriminative parsing. In Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics (AISTATS), pages 224–232. [find](https://scholar.google.com/scholar?q=R%2E+Collobert%2E+2011%2E+Deep+learning+for+efficient+discriminative+parsing%2E+In+Proceedings+of+the+Fourteenth+International+Conference+on+Artificial+Intelligence+and+Statistics+%28AISTATS%29%2C+pages+224%E2%80%93232%2E) 7. Cı́cero Nogueira dos Santos and Bianca Zadrozny. 2014. Learning character-level representations for part-of-speech tagging. In Proceedings of the 31st International Conference on Machine Learning, JMLR: W&CP volume 32, Beijing, China. . m fil s i th of e ut in m e gl n si a e lik t n’ d di [find](https://scholar.google.com/scholar?q=C%C4%B1%CC%81cero+Nogueira+dos+Santos+and+Bianca+Zadrozny%2E+2014%2E+Learning+character%2Dlevel+representations+for+part%2Dof%2Dspeech+tagging%2E+In+Proceedings+of+the+31st+International+Conference+on+Machine+Learning%2C+JML)