---
title: "Harnessing Deep Neural Networks with Logic Rules — Hu 2016"
description: "Deep neural networks learn mostly from labeled examples."
authors: ["Zhiting Hu", "Xuezhe Ma", "Zhengzhong Liu", "Eduard Hovy", "Eric Xing"]
year: 2016
venue: "Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/P16-1228/"
doi: "10.18653/v1/p16-1228"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hu-2016-harnessing-deep-neural-networks-logic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hu-2016-harnessing-deep-neural-networks-logic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/) / Harnessing Deep Neural Networks with Logic Rules

# Harnessing Deep Neural Networks with Logic Rules

**Authors:** [Zhiting Hu](https://md.research.elamur.ai/free/en/authors/zhiting-hu/), [Xuezhe Ma](https://md.research.elamur.ai/free/en/authors/xuezhe-ma/), [Zhengzhong Liu](https://md.research.elamur.ai/free/en/authors/zhengzhong-liu/), [Eduard Hovy](https://md.research.elamur.ai/free/en/authors/eduard-hovy/), [Eric Xing](https://md.research.elamur.ai/free/en/authors/eric-xing/) · **Institutions:** [Carnegie Mellon University](https://md.research.elamur.ai/free/en/universities/carnegie-mellon-university/) · **Year:** 2016 · **Venue:** Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P16-1228/) · [PDF](https://aclanthology.org/P16-1228.pdf) · [DOI 10.18653/v1/p16-1228](https://doi.org/10.18653/v1/p16-1228) · [OpenAlex](https://openalex.org/W2963687836)

**Citations:** 545 citations (OpenAlex, 2026-10-14) · [74](https://md.research.elamur.ai/free/en/papers/hu-2016-harnessing-deep-neural-networks-logic/cited-by/) in our library, of the 140884 papers whose bibliographies we read

## Summary
Deep neural networks learn mostly from labeled examples. This makes them data-hungry, hard to interpret, and hard to steer with human knowledge. Hu et al. propose a general way to inject declarative first-order logic rules into arbitrary architectures such as CNNs and RNNs. Rules are written in soft logic, so truth values are continuous, and each rule has a confidence weight. At every training step, a "teacher" distribution is built in closed form by projecting the network's predictions onto a rule-constrained space, following the posterior regularization principle. The "student" network is then trained to match both the gold labels and the teacher's soft outputs. Teacher and student evolve together, and the rule knowledge ends up in the student's weights, so the student can predict without evaluating rules. Unlabeled data can also feed the imitation term, which gives semi-supervised learning. The authors test a CNN on sentiment analysis and a BLSTM on CoNLL-2003 NER. With only one or two intuitive rules, they reach results comparable to the best systems of the time, without adding model parameters.

## Key points
- The method combines knowledge distillation with posterior regularization (Ganchev et al., 2010). Unlike classic distillation, the teacher is not pre-trained; both models are updated iteratively.
- On CoNLL-2003 NER, the base BLSTM scores 89.55 F1. Adding bigram tag-transition rules lifts the teacher q to 91.11 F1. Adding a rule for entities in lists brings it to 91.18 F1 (student p: 89.93). This nearly matches Joint-NER-EL (91.20) and BLSTM-CRF of Ma and Hovy (91.21).
- In low-resource sentiment experiments, the rule-enhanced teacher beats the base CNN by 2.6% with 5% of the training data. The gains are 2.3% at 10% and 2.0% at 30%. Adding unlabeled data improves accuracy further.
- The iterative distillation clearly outperforms plain semi-supervised posterior regularization, which applies the rules only through unlabeled data. It also beats projecting a network onto the rules after training.
- The NER setup uses the BIOES tagging scheme, GloVe initialization and Adadelta. The gap between teacher and student is larger for NER than for sentiment, because the NER rules link adjacent positions and multiple instances, which requires joint inference.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{hu2016harnessing,
  title = {{Harnessing Deep Neural Networks with Logic Rules}},
  author = {Hu, Zhiting and Ma, Xuezhe and Liu, Zhengzhong and Hovy, Eduard and Xing, Eric},
  booktitle = {Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2016},
  doi = {10.18653/v1/p16-1228},
  url = {https://doi.org/10.18653/v1/p16-1228}
}
```

- [← previous Pointing the Unknown Words](https://md.research.elamur.ai/free/en/papers/gulcehre-2016-pointing-unknown-words/)
- [next → Fine-grained Analysis of Sentence Embeddings Using Auxiliary Prediction Tasks](https://md.research.elamur.ai/free/en/papers/adi-2016-fine-grained-analysis-sentence-embeddings/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/)

## References

45 references, 16 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Stephen H Bach, Matthias Broecheler, Bert Huang, and Lise Getoor. 2015. Hinge-loss Markov random fields and probabilistic soft logic. arXiv preprint arXiv:1505.04406. — [arXiv:1505.04406](https://arxiv.org/abs/1505.04406) [find](https://scholar.google.com/scholar?q=Stephen+H+Bach%2C+Matthias+Broecheler%2C+Bert+Huang%2C+and+Lise+Getoor%2E+2015%2E+Hinge%2Dloss+Markov+random+fields+and+probabilistic+soft+logic%2E+arXiv+preprint+arXiv%3A1505%2E04406%2E+%E2%80%94+%5BarXiv%3A1505%2E04406%5D%28https%3A%2F%2Farxi)
2. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2014. Neural machine translation by jointly learning to align and translate. Proc. of ICLR. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2014%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+Proc%2E+of+ICLR%2E) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
3. Tejas D Kulkarni, William F Whitney, Pushmeet Kohli, and Josh Tenenbaum. 2015. Deep convolutional inverse graphics network. In Proc. of NIPS, pages 2530–2538. [find](https://scholar.google.com/scholar?q=Tejas+D+Kulkarni%2C+William+F+Whitney%2C+Pushmeet+Kohli%2C+and+Josh+Tenenbaum%2E+2015%2E+Deep+convolutional+inverse+graphics+network%2E+In+Proc%2E+of+NIPS%2C+pages+2530%E2%80%932538%2E)
4. Brenden M Lake, Ruslan Salakhutdinov, and Joshua B Tenenbaum. 2015. Human-level concept learning through probabilistic program induction. Science, 350(6266):1332– 1338. [find](https://scholar.google.com/scholar?q=Brenden+M+Lake%2C+Ruslan+Salakhutdinov%2C+and+Joshua+B+Tenenbaum%2E+2015%2E+Human%2Dlevel+concept+learning+through+probabilistic+program+induction%2E+Science%2C+350%286266%29%3A1332%E2%80%93+1338%2E)
5. Cristian Bucilu, Rich Caruana, and Alexandru Niculescu-Mizil. 2006. Model compression. In Proc. of KDD, pages 535–541. ACM. [find](https://scholar.google.com/scholar?q=Cristian+Bucilu%2C+Rich+Caruana%2C+and+Alexandru+Niculescu%2DMizil%2E+2006%2E+Model+compression%2E+In+Proc%2E+of+KDD%2C+pages+535%E2%80%93541%2E+ACM%2E)
6. Guillaume Lample, Miguel Ballesteros, Sandeep Subramanian, Kazuya Kawakami, and Chris Dyer. 2016. Neural architectures for named entity recognition. In Proc. of NAACL. [find](https://scholar.google.com/scholar?q=Guillaume+Lample%2C+Miguel+Ballesteros%2C+Sandeep+Subramanian%2C+Kazuya+Kawakami%2C+and+Chris+Dyer%2E+2016%2E+Neural+architectures+for+named+entity+recognition%2E+In+Proc%2E+of+NAACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/lample-2016-neural-architectures-named-entity-recognition/)
7. Jason PC Chiu and Eric Nichols. 2015. Named entity recognition with bidirectional LSTM-CNNs. arXiv preprint arXiv:1511.08308. [find](https://scholar.google.com/scholar?q=Jason+PC+Chiu+and+Eric+Nichols%2E+2015%2E+Named+entity+recognition+with+bidirectional+LSTM%2DCNNs%2E+arXiv+preprint+arXiv%3A1511%2E08308%2E) [our card](https://md.research.elamur.ai/free/en/papers/chiu-2015-named-entity-recognition-bidirectional-lstm/)
8. Quoc V Le and Tomas Mikolov. 2014. Distributed representations of sentences and documents. Proc. of ICML. [find](https://scholar.google.com/scholar?q=Quoc+V+Le+and+Tomas+Mikolov%2E+2014%2E+Distributed+representations+of+sentences+and+documents%2E+Proc%2E+of+ICML%2E) [our card](https://md.research.elamur.ai/free/en/papers/le-2014-distributed-representations-sentences-documents/)
9. Ronan Collobert, Jason Weston, Léon Bottou, Michael Karlen, Koray Kavukcuoglu, and Pavel Kuksa. 2011. Natural language processing (almost) from scratch. JMLR, 12:2493– 2537. [find](https://scholar.google.com/scholar?q=Ronan+Collobert%2C+Jason+Weston%2C+Le%CC%81on+Bottou%2C+Michael+Karlen%2C+Koray+Kavukcuoglu%2C+and+Pavel+Kuksa%2E+2011%2E+Natural+language+processing+%28almost%29+from+scratch%2E+JMLR%2C+12%3A2493%E2%80%93+2537%2E) [our card](https://md.research.elamur.ai/free/en/papers/collobert-2011-natural-language-processing-almost-scratch/)
10. James Foulds, Shachi Kumar, and Lise Getoor. 2015. Latent topic networks: A versatile probabilistic programming framework for topic models. In Proc. of ICML, pages 777–786. [find](https://scholar.google.com/scholar?q=James+Foulds%2C+Shachi+Kumar%2C+and+Lise+Getoor%2E+2015%2E+Latent+topic+networks%3A+A+versatile+probabilistic+programming+framework+for+topic+models%2E+In+Proc%2E+of+ICML%2C+pages+777%E2%80%93786%2E)
11. Manoel VM França, Gerson Zaverucha, and Artur S dAvila Garcez. 2014. Fast relational learning using bottom clause propositionalization with artificial neural networks. Machine learning, 94(1):81–104. [find](https://scholar.google.com/scholar?q=Manoel+VM+Franc%CC%A7a%2C+Gerson+Zaverucha%2C+and+Artur+S+dAvila+Garcez%2E+2014%2E+Fast+relational+learning+using+bottom+clause+propositionalization+with+artificial+neural+networks%2E+Machine+learning%2C+94%281%29%3A81%E2%80%93104%2E)
12. Kuzman Ganchev, Joao Graça, Jennifer Gillenwater, and Ben Taskar. 2010. Posterior regularization for structured latent variable models. JMLR, 11:2001–2049. [find](https://scholar.google.com/scholar?q=Kuzman+Ganchev%2C+Joao+Grac%CC%A7a%2C+Jennifer+Gillenwater%2C+and+Ben+Taskar%2E+2010%2E+Posterior+regularization+for+structured+latent+variable+models%2E+JMLR%2C+11%3A2001%E2%80%932049%2E)
13. Artur S d’Avila Garcez, Krysia Broda, and Dov M Gabbay. 2012. Neural-symbolic learning systems: foundations and applications. Springer Science & Business Media. [find](https://scholar.google.com/scholar?q=Artur+S+d%E2%80%99Avila+Garcez%2C+Krysia+Broda%2C+and+Dov+M+Gabbay%2E+2012%2E+Neural%2Dsymbolic+learning+systems%3A+foundations+and+applications%2E+Springer+Science+%26+Business+Media%2E)
14. Geoffrey Hinton, Li Deng, Dong Yu, George E Dahl, Abdelrahman Mohamed, Navdeep Jaitly, Andrew Senior, Vincent Vanhoucke, Patrick Nguyen, Tara N Sainath, et al. 2012. Deep neural networks for acoustic modeling in speech recognition: The shared views of four research groups. Signal Processing Magazine… [find](https://scholar.google.com/scholar?q=Geoffrey+Hinton%2C+Li+Deng%2C+Dong+Yu%2C+George+E+Dahl%2C+Abdelrahman+Mohamed%2C+Navdeep+Jaitly%2C+Andrew+Senior%2C+Vincent+Vanhoucke%2C+Patrick+Nguyen%2C+Tara+N+Sainath%2C+et+al%2E+2012%2E+Deep+neural+networks+for+acoustic+)
15. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. 2015. Distilling the knowledge in a neural network. arXiv preprint arXiv:1503.02531. [find](https://scholar.google.com/scholar?q=Geoffrey+Hinton%2C+Oriol+Vinyals%2C+and+Jeff+Dean%2E+2015%2E+Distilling+the+knowledge+in+a+neural+network%2E+arXiv+preprint+arXiv%3A1503%2E02531%2E) [our card](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/)
16. Percy Liang, Hal Daumé III, and Dan Klein. 2008. Structure compilation: trading structure for features. In Proc. of ICML, pages 592–599. ACM. [find](https://scholar.google.com/scholar?q=Percy+Liang%2C+Hal+Daume%CC%81+III%2C+and+Dan+Klein%2E+2008%2E+Structure+compilation%3A+trading+structure+for+features%2E+In+Proc%2E+of+ICML%2C+pages+592%E2%80%93599%2E+ACM%2E)
17. Percy Liang, Michael I Jordan, and Dan Klein. 2009. Learning from measurements in exponential families. In Proc. of ICML, pages 641–648. ACM. [find](https://scholar.google.com/scholar?q=Percy+Liang%2C+Michael+I+Jordan%2C+and+Dan+Klein%2E+2009%2E+Learning+from+measurements+in+exponential+families%2E+In+Proc%2E+of+ICML%2C+pages+641%E2%80%93648%2E+ACM%2E)
18. David Lopez-Paz, Léon Bottou, Bernhard Schölkopf, and Vladimir Vapnik. 2016. Unifying distillation and privileged information. Prof. of ICLR. [find](https://scholar.google.com/scholar?q=David+Lopez%2DPaz%2C+Le%CC%81on+Bottou%2C+Bernhard+Scho%CC%88lkopf%2C+and+Vladimir+Vapnik%2E+2016%2E+Unifying+distillation+and+privileged+information%2E+Prof%2E+of+ICLR%2E)
19. Gang Luo, Xiaojiang Huang, Chin-Yew Lin, and Zaiqing Nie. 2015. Joint named entity recognition and disambiguation. In Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Gang+Luo%2C+Xiaojiang+Huang%2C+Chin%2DYew+Lin%2C+and+Zaiqing+Nie%2E+2015%2E+Joint+named+entity+recognition+and+disambiguation%2E+In+Proc%2E+of+EMNLP%2E)
20. Xuezhe Ma and Eduard Hovy. 2016. End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Xuezhe+Ma+and+Eduard+Hovy%2E+2016%2E+End%2Dto%2Dend+sequence+labeling+via+bi%2Ddirectional+LSTM%2DCNNs%2DCRF%2E+In+Proc%2E+of+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/ma-2016-end-end-sequence-labeling-bi/)
21. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Proc. of NIPS, pages 3111–3119. [find](https://scholar.google.com/scholar?q=Tomas+Mikolov%2C+Ilya+Sutskever%2C+Kai+Chen%2C+Greg+S+Corrado%2C+and+Jeff+Dean%2E+2013%2E+Distributed+representations+of+words+and+phrases+and+their+compositionality%2E+In+Proc%2E+of+NIPS%2C+pages+3111%E2%80%933119%2E) [our card](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/)
22. Marvin Minksy. 1980. Learning meaning. Technical Report AI Lab Memo. Project MAC. MIT. [find](https://scholar.google.com/scholar?q=Marvin+Minksy%2E+1980%2E+Learning+meaning%2E+Technical+Report+AI+Lab+Memo%2E+Project+MAC%2E+MIT%2E)
23. Anh Nguyen, Jason Yosinski, and Jeff Clune. 2015. Deep neural networks are easily fooled: High confidence predictions for unrecognizable images. In Proc. of CVPR, pages 427–436. IEEE. [find](https://scholar.google.com/scholar?q=Anh+Nguyen%2C+Jason+Yosinski%2C+and+Jeff+Clune%2E+2015%2E+Deep+neural+networks+are+easily+fooled%3A+High+confidence+predictions+for+unrecognizable+images%2E+In+Proc%2E+of+CVPR%2C+pages+427%E2%80%93436%2E+IEEE%2E)
24. Minqing Hu and Bing Liu. 2004. Mining and summarizing customer reviews. In Proc. of KDD, pages 168–177. ACM. [find](https://scholar.google.com/scholar?q=Minqing+Hu+and+Bing+Liu%2E+2004%2E+Mining+and+summarizing+customer+reviews%2E+In+Proc%2E+of+KDD%2C+pages+168%E2%80%93177%2E+ACM%2E)
25. Bo Pang and Lillian Lee. 2005. Seeing stars: Exploiting class relationships for sentiment categorization with respect to rating scales. In Proc. of ACL, pages 115–124. [find](https://scholar.google.com/scholar?q=Bo+Pang+and+Lillian+Lee%2E+2005%2E+Seeing+stars%3A+Exploiting+class+relationships+for+sentiment+categorization+with+respect+to+rating+scales%2E+In+Proc%2E+of+ACL%2C+pages+115%E2%80%93124%2E) [our card](https://md.research.elamur.ai/free/en/papers/pang-2005-seeing-stars-exploiting-class-relationships/)
26. Matthew J Johnson, David Duvenaud, Alexander B Wiltschko, Sandeep R Datta, and Ryan P Adams. 2016. Structured VAEs: Composing probabilistic graphical models and variational autoencoders. arXiv preprint arXiv:1603.06277. — [arXiv:1603.06277](https://arxiv.org/abs/1603.06277) [find](https://scholar.google.com/scholar?q=Matthew+J+Johnson%2C+David+Duvenaud%2C+Alexander+B+Wiltschko%2C+Sandeep+R+Datta%2C+and+Ryan+P+Adams%2E+2016%2E+Structured+VAEs%3A+Composing+probabilistic+graphical+models+and+variational+autoencoders%2E+arXiv+preprin)
27. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proc. of EMNLP, volume 14, pages 1532–1543. [find](https://scholar.google.com/scholar?q=Jeffrey+Pennington%2C+Richard+Socher%2C+and+Christopher+D+Manning%2E+2014%2E+Glove%3A+Global+vectors+for+word+representation%2E+In+Proc%2E+of+EMNLP%2C+volume+14%2C+pages+1532%E2%80%931543%2E) [our card](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/)
28. Theofanis Karaletsos, Serge Belongie, Cornell Tech, and Gunnar Rätsch. 2016. Bayesian representation learning with oracle constraints. In Proc. of ICLR. [find](https://scholar.google.com/scholar?q=Theofanis+Karaletsos%2C+Serge+Belongie%2C+Cornell+Tech%2C+and+Gunnar+Ra%CC%88tsch%2E+2016%2E+Bayesian+representation+learning+with+oracle+constraints%2E+In+Proc%2E+of+ICLR%2E)
29. Yoon Kim. 2014. Convolutional neural networks for sentence classification. Proc. of EMNLP. [find](https://scholar.google.com/scholar?q=Yoon+Kim%2E+2014%2E+Convolutional+neural+networks+for+sentence+classification%2E+Proc%2E+of+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/kim-2014-convolutional-neural-networks-sentence-classification/)
30. Diederik P Kingma and Max Welling. 2014. Auto-encoding variational Bayes. In Proc. of ICLR. [find](https://scholar.google.com/scholar?q=Diederik+P+Kingma+and+Max+Welling%2E+2014%2E+Auto%2Dencoding+variational+Bayes%2E+In+Proc%2E+of+ICLR%2E) [our card](https://md.research.elamur.ai/free/en/papers/kingma-2013-auto-encoding-variational-bayes/)
31. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Proc. of NIPS, pages 1097–1105. [find](https://scholar.google.com/scholar?q=Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Geoffrey+E+Hinton%2E+2012%2E+Imagenet+classification+with+deep+convolutional+neural+networks%2E+In+Proc%2E+of+NIPS%2C+pages+1097%E2%80%931105%2E)
32. Danilo Jimenez Rezende, Shakir Mohamed, and Daan Wierstra. 2014. Stochastic backpropagation and approximate inference in deep generative models. Proc. of ICML. [find](https://scholar.google.com/scholar?q=Danilo+Jimenez+Rezende%2C+Shakir+Mohamed%2C+and+Daan+Wierstra%2E+2014%2E+Stochastic+backpropagation+and+approximate+inference+in+deep+generative+models%2E+Proc%2E+of+ICML%2E) [our card](https://md.research.elamur.ai/free/en/papers/rezende-2014-stochastic-backpropagation-approximate-inference-deep/)
33. Danilo Jimenez Rezende, Shakir Mohamed, Ivo Danihelka, Karol Gregor, and Daan Wierstra. 2016. One-shot generalization in deep generative models. arXiv preprint arXiv:1603.05106. — [arXiv:1603.05106](https://arxiv.org/abs/1603.05106) [find](https://scholar.google.com/scholar?q=Danilo+Jimenez+Rezende%2C+Shakir+Mohamed%2C+Ivo+Danihelka%2C+Karol+Gregor%2C+and+Daan+Wierstra%2E+2016%2E+One%2Dshot+generalization+in+deep+generative+models%2E+arXiv+preprint+arXiv%3A1603%2E05106%2E+%E2%80%94+%5BarXiv%3A1603%2E05106%5D%28h)
34. Matthew Richardson and Pedro Domingos. 2006. Markov logic networks. Machine learning, 62(1-2):107–136. [find](https://scholar.google.com/scholar?q=Matthew+Richardson+and+Pedro+Domingos%2E+2006%2E+Markov+logic+networks%2E+Machine+learning%2C+62%281%2D2%29%3A107%E2%80%93136%2E)
35. David Silver, Aja Huang, Chris J Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Veda Panneershelvam, Marc Lanctot, et al. 2016. Mastering the game of go with deep neural networks and tree search. Nature, 529(7587):484–489. [find](https://scholar.google.com/scholar?q=David+Silver%2C+Aja+Huang%2C+Chris+J+Maddison%2C+Arthur+Guez%2C+Laurent+Sifre%2C+George+van+den+Driessche%2C+Julian+Schrittwieser%2C+Ioannis+Antonoglou%2C+Veda+Panneershelvam%2C+Marc+Lanctot%2C+et+al%2E+2016%2E+Mastering+the)
36. Richard Socher, Alex Perelygin, Jean Y Wu, Jason Chuang, Christopher D Manning, Andrew Y Ng, and Christopher Potts. 2013. Recursive deep models for semantic compositionality over a sentiment treebank. In Proc. of EMNLP, volume 1631, page 1642. Citeseer. [find](https://scholar.google.com/scholar?q=Richard+Socher%2C+Alex+Perelygin%2C+Jean+Y+Wu%2C+Jason+Chuang%2C+Christopher+D+Manning%2C+Andrew+Y+Ng%2C+and+Christopher+Potts%2E+2013%2E+Recursive+deep+models+for+semantic+compositionality+over+a+sentiment+treebank%2E)
37. Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, Dumitru Erhan, Ian Goodfellow, and Rob Fergus. 2014. Intriguing properties of neural networks. Proc. of ICLR. [find](https://scholar.google.com/scholar?q=Christian+Szegedy%2C+Wojciech+Zaremba%2C+Ilya+Sutskever%2C+Joan+Bruna%2C+Dumitru+Erhan%2C+Ian+Goodfellow%2C+and+Rob+Fergus%2E+2014%2E+Intriguing+properties+of+neural+networks%2E+Proc%2E+of+ICLR%2E) [our card](https://md.research.elamur.ai/free/en/papers/szegedy-2013-intriguing-properties-neural-networks/)
38. Erik F Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the conll-2003 shared task: Language-independent named entity recognition. In Proc. of CoNLL, pages 142– 147. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Erik+F+Tjong+Kim+Sang+and+Fien+De+Meulder%2E+2003%2E+Introduction+to+the+conll%2D2003+shared+task%3A+Language%2Dindependent+named+entity+recognition%2E+In+Proc%2E+of+CoNLL%2C+pages+142%E2%80%93+147%2E+Association+for+Computati) [our card](https://md.research.elamur.ai/free/en/papers/sang-2003-introduction-conll-2003-shared-task/)
39. Geoffrey G Towell, Jude W Shavlik, and Michiel O No-ordewier. 1990. Refinement of approximate domain theories by knowledge-based neural networks. In Proceedings of the eighth National conference on Artificial intelligence, pages 861–866. Boston, MA. [find](https://scholar.google.com/scholar?q=Geoffrey+G+Towell%2C+Jude+W+Shavlik%2C+and+Michiel+O+No%2Dordewier%2E+1990%2E+Refinement+of+approximate+domain+theories+by+knowledge%2Dbased+neural+networks%2E+In+Proceedings+of+the+eighth+National+conference+on+Ar)
40. Sida Wang and Christopher Manning. 2013. Fast dropout training. In Proc. of ICML, pages 118–126. [find](https://scholar.google.com/scholar?q=Sida+Wang+and+Christopher+Manning%2E+2013%2E+Fast+dropout+training%2E+In+Proc%2E+of+ICML%2C+pages+118%E2%80%93126%2E)
41. Bishan Yang and Claire Cardie. 2014. Context-aware learning for sentence-level sentiment analysis with posterior regularization. In Proc. of ACL, pages 325–335. [find](https://scholar.google.com/scholar?q=Bishan+Yang+and+Claire+Cardie%2E+2014%2E+Context%2Daware+learning+for+sentence%2Dlevel+sentiment+analysis+with+posterior+regularization%2E+In+Proc%2E+of+ACL%2C+pages+325%E2%80%93335%2E)
42. Wenpeng Yin and Hinrich Schutze. 2015. Multichannel variable-size convolution for sentence classification. Proc. of CONLL. [find](https://scholar.google.com/scholar?q=Wenpeng+Yin+and+Hinrich+Schutze%2E+2015%2E+Multichannel+variable%2Dsize+convolution+for+sentence+classification%2E+Proc%2E+of+CONLL%2E)
43. Matthew D Zeiler. 2012. Adadelta: an adaptive learning rate method. arXiv preprint arXiv:1212.5701. [find](https://scholar.google.com/scholar?q=Matthew+D+Zeiler%2E+2012%2E+Adadelta%3A+an+adaptive+learning+rate+method%2E+arXiv+preprint+arXiv%3A1212%2E5701%2E) [our card](https://md.research.elamur.ai/free/en/papers/zeiler-2012-adadelta-adaptive-learning-rate-method/)
44. Ye Zhang, Stephen Roller, and Byron Wallace. 2016. MGNC-CNN: A simple approach to exploiting multiple word embeddings for sentence classification. Proc. of NAACL. [find](https://scholar.google.com/scholar?q=Ye+Zhang%2C+Stephen+Roller%2C+and+Byron+Wallace%2E+2016%2E+MGNC%2DCNN%3A+A+simple+approach+to+exploiting+multiple+word+embeddings+for+sentence+classification%2E+Proc%2E+of+NAACL%2E)
45. Jun Zhu, Ning Chen, and Eric P Xing. 2014. Bayesian inference with posterior regularization and applications to infinite latent SVMs. JMLR, 15(1):1799–1847. [find](https://scholar.google.com/scholar?q=Jun+Zhu%2C+Ning+Chen%2C+and+Eric+P+Xing%2E+2014%2E+Bayesian+inference+with+posterior+regularization+and+applications+to+infinite+latent+SVMs%2E+JMLR%2C+15%281%29%3A1799%E2%80%931847%2E)