---
title: "Baseline Needs More Love — Shen 2018"
description: "Text-encoding research often relies on recurrent or convolutional compositional functions."
authors: ["Dinghan Shen", "Guoyin Wang", "Wenlin Wang", "Martin Renqiang Min", "Qinliang Su", "Yizhe Zhang", "Chunyuan Li", "Ricardo Henao", "Lawrence Carin"]
year: 2018
venue: "Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/P18-1041/"
doi: "10.18653/v1/p18-1041"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shen-2018-baseline-needs-more-love-simple/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shen-2018-baseline-needs-more-love-simple/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/) / Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms

# Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms

**Authors:** [Dinghan Shen](https://md.research.elamur.ai/free/en/authors/dinghan-shen/), [Guoyin Wang](https://md.research.elamur.ai/free/en/authors/guoyin-wang/), Wenlin Wang, Martin Renqiang Min, Qinliang Su, [Yizhe Zhang](https://md.research.elamur.ai/free/en/authors/yizhe-zhang/), [Chunyuan Li](https://md.research.elamur.ai/free/en/authors/chunyuan-li/), Ricardo Henao, [Lawrence Carin](https://md.research.elamur.ai/free/en/authors/lawrence-carin/) · **Year:** 2018 · **Venue:** Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P18-1041/) · [PDF](https://aclanthology.org/P18-1041.pdf) · [DOI 10.18653/v1/p18-1041](https://doi.org/10.18653/v1/p18-1041) · [OpenAlex](https://openalex.org/W2964301648)

**Citations:** 313 citations (openalex-snapshot · 2026-10-14) · [70](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Text-encoding research often relies on recurrent or convolutional compositional functions. These add many parameters and slow down training, yet their real benefit over plain word embeddings had not been measured carefully. The authors run a task-by-task comparison between LSTM and CNN encoders and Simple Word-Embedding-based Models (SWEMs), which have no compositional parameters and only pool over word vectors. Besides plain averaging, they add three variants: max-pooling (SWEM-max), a concatenation of average and max features (SWEM-concat), and hierarchical pooling (SWEM-hier), which keeps local n-gram order. The evaluation covers 17 datasets in three groups: long-document classification, sentence matching (e.g., SNLI, WikiQA) and short-sentence classification/tagging (e.g., SST, TREC). In most settings SWEMs match or beat the heavier models. They fall behind mainly on short sentiment sentences, and hierarchical pooling closes much of that gap. A subspace-training analysis further suggests that the simple models need fewer effective parameters to reach good solutions. The result is a cheap, fast and interpretable baseline that benchmarks often omit, plus practical guidance on when extra architectural complexity is worth its cost.

## Key points
- The four SWEM variants (SWEM-aver, SWEM-max, SWEM-concat, SWEM-hier) are compared with LSTM and CNN encoders on 17 datasets. The datasets include Yahoo, Yelp, AG News, SNLI, WikiQA, MR, SST-1, SST-2, Subj and TREC.
- **Cost:**
  - SWEM has zero compositional parameters, while CNN has n·K·d and LSTM has 4·d·(K+d).
  - SWEM's computational complexity is O(L·K).
  - SWEM needs O(1) sequential steps, unlike the O(L) steps of an LSTM, so it parallelizes well.
- **Where SWEMs do well and poorly:**
  - Pooling works especially well for long documents with hundreds of words and for sentence matching.
  - Sentiment tasks depend more on word order than topic categorization does, which is where SWEM-hier helps.
- **Linear classifier on SWEM features:** replacing the MLP with a linear classifier costs very little accuracy (Yahoo! Ans. 73.53% → 73.18%; Yelp P. 93.76% → 93.66%).
- **Sogou news (Chinese in Pinyin):**
  - SWEM-hier with a window of 5 reaches 96.2%, versus 91.3% for SWEM-concat.
  - This is on par with the reported CNN (95.6%) and LSTM (95.2%) results.

## Interesting ideas
- **Take the maximum per embedding dimension.** Pooling the maximum value of each dimension lets a few salient words dominate the representation while filler words fade out. The authors also find that single dimensions then group thematically related words, which gives interpretability almost for free.
- **Concatenate average and max pooling.** The two pooled vectors capture complementary information, so joining them is a nearly free gain for any embedding-based classifier or retrieval encoder.
- **Pool hierarchically.** Pooling first inside local windows and then across the whole sequence retains some word-order information without any learned compositional weights. This is useful for order-sensitive tasks, and the Sogou result suggests it may matter more for some languages than for English.
- **Use subspace training to compare models.** Restricting training to a low-dimensional parameter subspace, and measuring the intrinsic dimension needed to reach an accuracy threshold, shows how much capacity a task really demands. This is a reusable way to justify picking a simpler model.
- **Probe representations with a linear classifier.** Swapping the non-linear head for a linear one checks whether the encoder itself produces informative features. This sanity check transfers easily to evaluating other embeddings.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

70 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 69 not yet.

- [Knowledge-Enriched Transformer for Emotion Detection in Textual Conversations](https://md.research.elamur.ai/free/en/papers/zhong-2019-knowledge-enriched-transformer-emotion-detection/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{shen2018baseline,
  title = {{Baseline Needs More Love: On Simple Word-Embedding-Based Models and Associated Pooling Mechanisms}},
  author = {Shen, Dinghan and Wang, Guoyin and Wang, Wenlin and Min, Martin Renqiang and Su, Qinliang and Zhang, Yizhe and Li, Chunyuan and Henao, Ricardo and Carin, Lawrence},
  booktitle = {Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2018},
  doi = {10.18653/v1/p18-1041},
  url = {https://doi.org/10.18653/v1/p18-1041}
}
```

- [← previous Averaging Weights Leads to Wider Optima and Better Generalization](https://md.research.elamur.ai/free/en/papers/izmailov-2018-averaging-weights-leads-wider-optima/)
- [next → BASHI: A Corpus of Wall Street Journal Articles Annotated with Bridging Links](https://md.research.elamur.ai/free/en/papers/rosiger-2018-bashi-corpus-wall-street-journal/)
- [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/)

## References

38 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Diederik Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. [find](https://scholar.google.com/scholar?q=Diederik+Kingma+and+Jimmy+Ba%2E+2014%2E+Adam%3A+A+method+for+stochastic+optimization%2E+arXiv+preprint+arXiv%3A1412%2E6980%2E)
2. Yossi Adi, Einat Kermany, Yonatan Belinkov, Ofer Lavi, and Yoav Goldberg. 2016. Fine-grained analysis of sentence embeddings using auxiliary prediction tasks. ICLR. [find](https://scholar.google.com/scholar?q=Yossi+Adi%2C+Einat+Kermany%2C+Yonatan+Belinkov%2C+Ofer+Lavi%2C+and+Yoav+Goldberg%2E+2016%2E+Fine%2Dgrained+analysis+of+sentence+embeddings+using+auxiliary+prediction+tasks%2E+ICLR%2E) [our card](https://md.research.elamur.ai/free/en/papers/adi-2016-fine-grained-analysis-sentence-embeddings/)
3. Chunyuan Li, Heerad Farkhoor, Rosanne Liu, and Jason Yosinski. 2018. Measuring the intrinsic dimension of objective landscapes. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Chunyuan+Li%2C+Heerad+Farkhoor%2C+Rosanne+Liu%2C+and+Jason+Yosinski%2E+2018%2E+Measuring+the+intrinsic+dimension+of+objective+landscapes%2E+In+International+Conference+on+Learning+Representations%2E)
4. Sanjeev Arora, Yingyu Liang, and Tengyu Ma. 2016. A simple but tough-to-beat baseline for sentence embeddings. In ICLR. [find](https://scholar.google.com/scholar?q=Sanjeev+Arora%2C+Yingyu+Liang%2C+and+Tengyu+Ma%2E+2016%2E+A+simple+but+tough%2Dto%2Dbeat+baseline+for+sentence+embeddings%2E+In+ICLR%2E)
5. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In NIPS, pages 3111–3119. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Ilya+Sutskever%2C+Kai+Chen%2C+Greg+S+Corrado%2C+and+Jeff+Dean%2E+2013%2E+Distributed+representations+of+words+and+phrases+and+their+compositionality%2E+In+NIPS%2C+pages+3111%E2%80%933119%2E)
6. Yoshua Bengio, Réjean Ducharme, Pascal Vincent, and Christian Jauvin. 2003. A neural probabilistic language model. JMLR, 3(Feb):1137–1155. [find](https://scholar.google.com/scholar?q=Yoshua+Bengio%2C+Re%CC%81jean+Ducharme%2C+Pascal+Vincent%2C+and+Christian+Jauvin%2E+2003%2E+A+neural+probabilistic+language+model%2E+JMLR%2C+3%28Feb%29%3A1137%E2%80%931155%2E)
7. Jeff Mitchell and Mirella Lapata. 2010. Composition in distributional models of semantics. Cognitive science, 34(8):1388–1429. [find](https://scholar.google.com/scholar?q=Jeff+Mitchell+and+Mirella+Lapata%2E+2010%2E+Composition+in+distributional+models+of+semantics%2E+Cognitive+science%2C+34%288%29%3A1388%E2%80%931429%2E)
8. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. JMLR, 12(Aug):2493–2537. [find](https://scholar.google.com/scholar?q=Ronan+Collobert%2C+Jason+Weston%2C+Le%CC%81on+Bottou%2C+Michael+Karlen%2C+Koray+Kavukcuoglu%2C+and+Pavel+Kuksa%2E+2011%2E+Natural+language+processing+%28almost%29+from+scratch%2E+JMLR%2C+12%28Aug%29%3A2493%E2%80%932537%2E) [our card](https://md.research.elamur.ai/free/en/papers/collobert-2011-natural-language-processing-almost-scratch/)
9. Yixin Nie and Mohit Bansal. 2017. Shortcut-stacked sentence encoders for multi-domain inference. arXiv preprint arXiv:1708.02312. [find](https://scholar.google.com/scholar?q=Yixin+Nie+and+Mohit+Bansal%2E+2017%2E+Shortcut%2Dstacked+sentence+encoders+for+multi%2Ddomain+inference%2E+arXiv+preprint+arXiv%3A1708%2E02312%2E)
10. Alexis Conneau, Douwe Kiela, Holger Schwenk, Loic Barrault, and Antoine Bordes. 2017. Supervised learning of universal sentence representations from natural language inference data. EMNLP. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Douwe+Kiela%2C+Holger+Schwenk%2C+Loic+Barrault%2C+and+Antoine+Bordes%2E+2017%2E+Supervised+learning+of+universal+sentence+representations+from+natural+language+inference+data%2E+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/conneau-2017-supervised-learning-universal-sentence-representations/)
11. Bo Pang, Lillian Lee, and Shivakumar Vaithyanathan. 2002. Thumbs up?: sentiment classification using machine learning techniques. In EMNLP, pages 79– 86. ACL. [find](https://scholar.google.com/scholar?q=Bo+Pang%2C+Lillian+Lee%2C+and+Shivakumar+Vaithyanathan%2E+2002%2E+Thumbs+up%3F%3A+sentiment+classification+using+machine+learning+techniques%2E+In+EMNLP%2C+pages+79%E2%80%93+86%2E+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/pang-2002-thumbs-up-sentiment-classification-machine/)
12. Alexis Conneau, Holger Schwenk, Loı̈c Barrault, and Yann Lecun. 2016. Very deep convolutional networks for natural language processing. arXiv preprint arXiv:1606.01781. — [arXiv:1606.01781](https://arxiv.org/abs/1606.01781) [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Holger+Schwenk%2C+Lo%C4%B1%CC%88c+Barrault%2C+and+Yann+Lecun%2E+2016%2E+Very+deep+convolutional+networks+for+natural+language+processing%2E+arXiv+preprint+arXiv%3A1606%2E01781%2E+%E2%80%94+%5BarXiv%3A1606%2E01781%5D%28https%3A%2F%2Far)
13. Ankur P Parikh, Oscar Täckström, Dipanjan Das, and Jakob Uszkoreit. 2016. A decomposable attention model for natural language inference. EMNLP. [find](https://scholar.google.com/scholar?q=Ankur+P+Parikh%2C+Oscar+Ta%CC%88ckstro%CC%88m%2C+Dipanjan+Das%2C+and+Jakob+Uszkoreit%2E+2016%2E+A+decomposable+attention+model+for+natural+language+inference%2E+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/parikh-2016-decomposable-attention-model-natural-language/)
14. Zhe Gan, Yunchen Pu, Ricardo Henao, Chunyuan Li, Xiaodong He, and Lawrence Carin. 2017. Learning generic sentence representations using convolutional neural networks. In EMNLP, pages 2380–2390. [find](https://scholar.google.com/scholar?q=Zhe+Gan%2C+Yunchen+Pu%2C+Ricardo+Henao%2C+Chunyuan+Li%2C+Xiaodong+He%2C+and+Lawrence+Carin%2E+2017%2E+Learning+generic+sentence+representations+using+convolutional+neural+networks%2E+In+EMNLP%2C+pages+2380%E2%80%932390%2E)
15. Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. Glove: Global vectors for word representation. In EMNLP, pages 1532–1543. [find](https://scholar.google.com/scholar?q=Jeffrey+Pennington%2C+Richard+Socher%2C+and+Christopher+Manning%2E+2014%2E+Glove%3A+Global+vectors+for+word+representation%2E+In+EMNLP%2C+pages+1532%E2%80%931543%2E)
16. Alex Graves, Navdeep Jaitly, and Abdel-rahman Mohamed. 2013. Hybrid speech recognition with deep bidirectional lstm. In Automatic Speech Recognition and Understanding (ASRU), 2013 IEEE Workshop on, pages 273–278. IEEE. [find](https://scholar.google.com/scholar?q=Alex+Graves%2C+Navdeep+Jaitly%2C+and+Abdel%2Drahman+Mohamed%2E+2013%2E+Hybrid+speech+recognition+with+deep+bidirectional+lstm%2E+In+Automatic+Speech+Recognition+and+Understanding+%28ASRU%29%2C+2013+IEEE+Workshop+on%2C+pa)
17. Dinghan Shen, Martin Renqiang Min, Yitong Li, and Lawrence Carin. 2017. Adaptive convolutional filter generation for natural language understanding. arXiv preprint arXiv:1709.08294. — [arXiv:1709.08294](https://arxiv.org/abs/1709.08294) [find](https://scholar.google.com/scholar?q=Dinghan+Shen%2C+Martin+Renqiang+Min%2C+Yitong+Li%2C+and+Lawrence+Carin%2E+2017%2E+Adaptive+convolutional+filter+generation+for+natural+language+understanding%2E+arXiv+preprint+arXiv%3A1709%2E08294%2E+%E2%80%94+%5BarXiv%3A1709%2E0829)
18. Dinghan Shen, Yizhe Zhang, Ricardo Henao, Qinliang Su, and Lawrence Carin. 2018. Deconvolutional latent-variable model for text sequence matching. AAAI. [find](https://scholar.google.com/scholar?q=Dinghan+Shen%2C+Yizhe+Zhang%2C+Ricardo+Henao%2C+Qinliang+Su%2C+and+Lawrence+Carin%2E+2018%2E+Deconvolutional+latent%2Dvariable+model+for+text+sequence+matching%2E+AAAI%2E)
19. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+computation%2C+9%288%29%3A1735%E2%80%931780%2E)
20. Richard Socher, Brody Huval, Christopher D Manning, and Andrew Y Ng. 2012. Semantic compositionality through recursive matrix-vector spaces. In EMNLP, pages 1201–1211. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Brody+Huval%2C+Christopher+D+Manning%2C+and+Andrew+Y+Ng%2E+2012%2E+Semantic+compositionality+through+recursive+matrix%2Dvector+spaces%2E+In+EMNLP%2C+pages+1201%E2%80%931211%2E+Association+for+Computational+Li) [our card](https://md.research.elamur.ai/free/en/papers/socher-2012-semantic-compositionality-through-recursive-matrix/)
21. Baotian Hu, Zhengdong Lu, Hang Li, and Qingcai Chen. 2014. Convolutional neural network architectures for matching natural language sentences. In NIPS, pages 2042–2050. [find](https://scholar.google.com/scholar?q=Baotian+Hu%2C+Zhengdong+Lu%2C+Hang+Li%2C+and+Qingcai+Chen%2E+2014%2E+Convolutional+neural+network+architectures+for+matching+natural+language+sentences%2E+In+NIPS%2C+pages+2042%E2%80%932050%2E)
22. Mohit Iyyer, Varun Manjunatha, Jordan Boyd-Graber, and Hal Daumé III. 2015. Deep unordered composition rivals syntactic methods for text classification. In ACL, volume 1, pages 16 81–1691. [find](https://scholar.google.com/scholar?q=Mohit+Iyyer%2C+Varun+Manjunatha%2C+Jordan+Boyd%2DGraber%2C+and+Hal+Daume%CC%81+III%2E+2015%2E+Deep+unordered+composition+rivals+syntactic+methods+for+text+classification%2E+In+ACL%2C+volume+1%2C+pages+16+81%E2%80%931691%2E) [our card](https://md.research.elamur.ai/free/en/papers/iyyer-2015-deep-unordered-composition-rivals-syntactic/)
23. Richard Socher, Cliff C Lin, Chris Manning, and Andrew Y Ng. 2011a. Parsing natural scenes and natural language with recursive neural networks. In ICML, pages 129–136. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Cliff+C+Lin%2C+Chris+Manning%2C+and+Andrew+Y+Ng%2E+2011a%2E+Parsing+natural+scenes+and+natural+language+with+recursive+neural+networks%2E+In+ICML%2C+pages+129%E2%80%93136%2E)
24. Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. 2016. Bag of tricks for efficient text classification. arXiv preprint arXiv:1607.01759. [find](https://scholar.google.com/scholar?q=Armand+Joulin%2C+Edouard+Grave%2C+Piotr+Bojanowski%2C+and+Tomas+Mikolov%2E+2016%2E+Bag+of+tricks+for+efficient+text+classification%2E+arXiv+preprint+arXiv%3A1607%2E01759%2E)
25. Richard Socher, Jeffrey Pennington, Eric H Huang, Andrew Y Ng, and Christopher D Manning. 2011b. Semi-supervised recursive autoencoders for predicting sentiment distributions. In EMNLP, pages 151– 161. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Jeffrey+Pennington%2C+Eric+H+Huang%2C+Andrew+Y+Ng%2C+and+Christopher+D+Manning%2E+2011b%2E+Semi%2Dsupervised+recursive+autoencoders+for+predicting+sentiment+distributions%2E+In+EMNLP%2C+pages+151%E2%80%93+161) [our card](https://md.research.elamur.ai/free/en/papers/socher-2011-semi-supervised-recursive-autoencoders-predicting/)
26. Nal Kalchbrenner, Edward Grefenstette, and Phil Blunsom. 2014. A convolutional neural network for modelling sentences. arXiv preprint arXiv:1404.2188. [find](https://scholar.google.com/scholar?q=Nal+Kalchbrenner%2C+Edward+Grefenstette%2C+and+Phil+Blunsom%2E+2014%2E+A+convolutional+neural+network+for+modelling+sentences%2E+arXiv+preprint+arXiv%3A1404%2E2188%2E) [our card](https://md.research.elamur.ai/free/en/papers/kalchbrenner-2014-convolutional-neural-network-modelling-sentences/)
27. Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. Dropout: a simple way to prevent neural networks from overfitting. JMLR, 15(1):1929–1958. [find](https://scholar.google.com/scholar?q=Nitish+Srivastava%2C+Geoffrey+E+Hinton%2C+Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Ruslan+Salakhutdinov%2E+2014%2E+Dropout%3A+a+simple+way+to+prevent+neural+networks+from+overfitting%2E+JMLR%2C+15%281%29%3A1929%E2%80%931958%2E)
28. Yoon Kim. 2014. Convolutional neural networks for sentence classification. EMNLP. [find](https://scholar.google.com/scholar?q=Yoon+Kim%2E+2014%2E+Convolutional+neural+networks+for+sentence+classification%2E+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/kim-2014-convolutional-neural-networks-sentence-classification/)
29. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In NIPS, pages 3104–3112. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+NIPS%2C+pages+3104%E2%80%933112%2E)
30. Kai Sheng Tai, Richard Socher, and Christopher D Manning. 2015. Improved semantic representations from tree-structured long short-term memory networks. arXiv preprint arXiv:1503.00075. [find](https://scholar.google.com/scholar?q=Kai+Sheng+Tai%2C+Richard+Socher%2C+and+Christopher+D+Manning%2E+2015%2E+Improved+semantic+representations+from+tree%2Dstructured+long+short%2Dterm+memory+networks%2E+arXiv+preprint+arXiv%3A1503%2E00075%2E)
31. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. NIPS. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+Lukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+NIPS%2E)
32. John Wieting, Mohit Bansal, Kevin Gimpel, and Karen Livescu. 2015. Towards universal paraphrastic sentence embeddings. ICLR. [find](https://scholar.google.com/scholar?q=John+Wieting%2C+Mohit+Bansal%2C+Kevin+Gimpel%2C+and+Karen+Livescu%2E+2015%2E+Towards+universal+paraphrastic+sentence+embeddings%2E+ICLR%2E) [our card](https://md.research.elamur.ai/free/en/papers/wieting-2015-towards-universal-paraphrastic-sentence-embeddings/)
33. Adina Williams, Nikita Nangia, and Samuel R Bowman. 2017. A broad-coverage challenge corpus for sentence understanding through inference. arXiv preprint arXiv:1704.05426. [find](https://scholar.google.com/scholar?q=Adina+Williams%2C+Nikita+Nangia%2C+and+Samuel+R+Bowman%2E+2017%2E+A+broad%2Dcoverage+challenge+corpus+for+sentence+understanding+through+inference%2E+arXiv+preprint+arXiv%3A1704%2E05426%2E)
34. Shiliang Zhang, Hui Jiang, Mingbin Xu, Junfeng Hou, and Lirong Dai. 2015a. The fixed-size ordinally-forgetting encoding method for neural network language models. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on… [find](https://scholar.google.com/scholar?q=Shiliang+Zhang%2C+Hui+Jiang%2C+Mingbin+Xu%2C+Junfeng+Hou%2C+and+Lirong+Dai%2E+2015a%2E+The+fixed%2Dsize+ordinally%2Dforgetting+encoding+method+for+neural+network+language+models%2E+In+Proceedings+of+the+53rd+Annual+Mee)
35. Xiang Zhang, Junbo Zhao, and Yann LeCun. 2015b. Character-level convolutional networks for text classification. In NIPS, pages 649–657. [find](https://scholar.google.com/scholar?q=Xiang+Zhang%2C+Junbo+Zhao%2C+and+Yann+LeCun%2E+2015b%2E+Character%2Dlevel+convolutional+networks+for+text+classification%2E+In+NIPS%2C+pages+649%E2%80%93657%2E) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/)
36. Yizhe Zhang, Zhe Gan, Kai Fan, Zhi Chen, Ricardo Henao, Dinghan Shen, and Lawrence Carin. 2017a. Adversarial feature matching for text generation. In ICML. [find](https://scholar.google.com/scholar?q=Yizhe+Zhang%2C+Zhe+Gan%2C+Kai+Fan%2C+Zhi+Chen%2C+Ricardo+Henao%2C+Dinghan+Shen%2C+and+Lawrence+Carin%2E+2017a%2E+Adversarial+feature+matching+for+text+generation%2E+In+ICML%2E)
37. Yizhe Zhang, Dinghan Shen, Guoyin Wang, Zhe Gan, Ricardo Henao, and Lawrence Carin. 2017b. Deconvolutional paragraph representation learning. NIPS. [find](https://scholar.google.com/scholar?q=Yizhe+Zhang%2C+Dinghan+Shen%2C+Guoyin+Wang%2C+Zhe+Gan%2C+Ricardo+Henao%2C+and+Lawrence+Carin%2E+2017b%2E+Deconvolutional+paragraph+representation+learning%2E+NIPS%2E)
38. Han Zhao, Zhengdong Lu, and Pascal Poupart. 2015. Self-adaptive hierarchical sentence model. In IJCAI, pages 4069–4076. [find](https://scholar.google.com/scholar?q=Han+Zhao%2C+Zhengdong+Lu%2C+and+Pascal+Poupart%2E+2015%2E+Self%2Dadaptive+hierarchical+sentence+model%2E+In+IJCAI%2C+pages+4069%E2%80%934076%2E)