---
title: "End-to-End Differentiable Proving — Rocktäschel 2017"
description: "Rocktäschel and Riedel propose Neural Theorem Provers (NTPs), which bridge the two. 69 references, cited by 75 papers here."
authors: ["Tim Rocktäschel", "Sebastian Riedel"]
year: 2017
venue: "arXiv:1705.11040"
source: "https://arxiv.org/abs/1705.11040"
doi: "10.48550/arxiv.1705.11040"
arxiv: "1705.11040"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "https://arxiv.org/licenses/nonexclusive-distrib/1.0/license.html"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/rocktaschel-2017-end-end-differentiable-proving/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rocktaschel-2017-end-end-differentiable-proving/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/3/) / End-to-End Differentiable Proving

# End-to-End Differentiable Proving

**Authors:** [Tim Rocktäschel](https://md.research.elamur.ai/free/en/authors/tim-rocktaschel/) and [Sebastian Riedel](https://md.research.elamur.ai/free/en/authors/sebastian-riedel/) · **Year:** 2017 · **Venue:** arXiv:1705.11040 · **Licence:** [arXiv — non-exclusive licence to distribute](https://arxiv.org/licenses/nonexclusive-distrib/1.0/license.html)

**Links:** [Publisher's page](https://arxiv.org/abs/1705.11040) · [PDF](https://arxiv.org/pdf/1705.11040v2) · [DOI 10.48550/arxiv.1705.11040](https://doi.org/10.48550/arxiv.1705.11040) · [arXiv 1705.11040](https://arxiv.org/abs/1705.11040) · [OpenAlex](https://openalex.org/W2617995259)

**Citations:** 113 citations (OpenAlex, 2026-10-14) · [75](https://md.research.elamur.ai/free/en/papers/rocktaschel-2017-end-end-differentiable-proving/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Neural link prediction models complete knowledge bases (KBs) by learning symbol embeddings and judging similarity, but they score each fact in isolation. This makes multi-step inference such as transitivity difficult for them. Symbolic provers like Prolog chain inferences easily but cannot generalize across symbols that are similar but not identical. Rocktäschel and Riedel propose Neural Theorem Provers (NTPs), which bridge the two. NTPs follow Prolog's backward chaining algorithm to recursively build a neural network for each query. Variable binding stays symbolic, while exact symbol matching is replaced by an RBF-kernel similarity between embeddings. Because the proof success score is differentiable, gradient descent can learn symbol representations and the parameters of rule templates, such as transitive patterns. The induced rules can then be decoded into readable first-order clauses. The authors evaluate on Countries, Kinship, Nations and UMLS. NTPλ, an NTP trained jointly with ComplEx as an auxiliary loss, beats ComplEx on three of the four KBs. This shows that a differentiable reasoner can be both accurate and interpretable.

## Key points
- **Method:** Unification compares non-variable symbols using an RBF kernel (µ = 1/√2) over embeddings, and partial proof scores are combined with min. The core operations are OR and AND modules modeled on Prolog.
- **Optimizations:** The architecture is made practical through joint training with a neural link predictor (ComplEx), batch proving, and approximate gradient computation.
- **Results:** Vanilla NTPs beat ComplEx only on Countries S3 and Nations. NTPλ leads in most tasks, with AUC-PR gains over ComplEx significant on all Countries tasks (p \< 0.0001).
- **Interpretability:** Induced rules recover the logic needed for the Countries tasks and capture transitivity on UMLS, a pattern that ComplEx struggles to encode.
- **Limitations:** Scalability to large KBs is limited, and function terms are not yet supported. A TensorLog-based differentiable rule learner scales better, but the ComplEx baseline outperformed it on HITS@10 (UMLS 1.00 vs 0.70; Kinship 0.98 vs 0.73).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Oxford](https://md.research.elamur.ai/free/en/universities/university-of-oxford/) — University, Oxford, United Kingdom · 122 works · 160,277 citations
- [University College London](https://md.research.elamur.ai/free/en/universities/university-college-london/) — University, London, United Kingdom · 107 works · 22,924 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

## Cite

Cite as: Tim Rocktäschel and Sebastian Riedel. 2017. End-to-End Differentiable Proving. arXiv:1705.11040.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{rocktaschel2017end,
  title = {{End-to-End Differentiable Proving}},
  author = {Tim Rocktäschel and Sebastian Riedel},
  year = {2017},
  doi = {10.48550/arxiv.1705.11040},
  eprint = {1705.11040},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1705.11040}
}
```

- [← previous Grounded Language Learning in a Simulated 3D World](https://md.research.elamur.ai/free/en/papers/hermann-2017-grounded-language-learning-simulated-3d/)
- [next → Learning Language Representations for Typology Prediction](https://md.research.elamur.ai/free/en/papers/malaviya-2017-learning-language-representations-typology-prediction/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/3/)

## References

69 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. Factorizing YAGO: scalable machine learning for linked data. In Proceedings of the 21st World Wide Web Conference 2012, WWW 2012, Lyon, France, April 16-20, 2012, pages 271–280, 2012. doi: 10.1145/2187836.2187874. — [doi:10.1145/2187836.2187874](https://doi.org/10.1145/2187836.2187874) [find](https://scholar.google.com/scholar?q=Maximilian+Nickel%2C+Volker+Tresp%2C+and+Hans%2DPeter+Kriegel%2E+Factorizing+YAGO%3A+scalable+machine+learning+for+linked+data%2E+In+Proceedings+of+the+21st+World+Wide+Web+Conference+2012%2C+WWW+2012%2C+Lyon%2C+France%2C)
2. Sebastian Riedel, Limin Yao, Andrew McCallum, and Benjamin M. Marlin. Relation extraction with matrix factorization and universal schemas. In Human Language Technologies: Conference of the North American Chapter of the Association of Computational Linguistics, Proceedings, June 9-14, 2013, Westin Pe… [find](https://scholar.google.com/scholar?q=Sebastian+Riedel%2C+Limin+Yao%2C+Andrew+McCallum%2C+and+Benjamin+M%2E+Marlin%2E+Relation+extraction+with+matrix+factorization+and+universal+schemas%2E+In+Human+Language+Technologies%3A+Conference+of+the+North+Ameri)
3. Richard Socher, Danqi Chen, Christopher D. Manning, and Andrew Y. Ng. [Reasoning with neural tensor networks for knowledge base completion](https://md.research.elamur.ai/free/en/papers/socher-2013-reasoning-neural-tensor-networks-knowledge/). In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meeting held Dece…
4. Kai-Wei Chang, Wen-tau Yih, Bishan Yang, and Christopher Meek. Typed tensor decomposition of knowledge bases for relation extraction. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Specia… [find](https://scholar.google.com/scholar?q=Kai%2DWei+Chang%2C+Wen%2Dtau+Yih%2C+Bishan+Yang%2C+and+Christopher+Meek%2E+Typed+tensor+decomposition+of+knowledge+bases+for+relation+extraction%2E+In+Proceedings+of+the+2014+Conference+on+Empirical+Methods+in+Natu)
5. Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. [Embedding entities and relations for learning and inference in knowledge bases](https://md.research.elamur.ai/free/en/papers/yang-2014-embedding-entities-relations-learning-inference/). In International Conference on Learning Representations (ICLR), 2015.
6. Kristina Toutanova, Danqi Chen, Patrick Pantel, Hoifung Poon, Pallavi Choudhury, and Michael Gamon. [Representing text for joint embedding of text and knowledge bases](https://md.research.elamur.ai/free/en/papers/toutanova-2015-representing-text-joint-embedding-text/). In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, EMNLP 2015, Lisbon, Portugal, September 17…
7. Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. [Complex embeddings for simple link prediction](https://md.research.elamur.ai/free/en/papers/trouillon-2016-complex-embeddings-simple-link-prediction/). In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, pages 2071–2080, 2016.
8. Hervé Gallaire and Jack Minker, editors. Logic and Data Bases, Symposium on Logic and Data Bases, Centre d’études et de recherches de Toulouse, 1977, Advances in Data Base Theory, New York, 1978. Plemum Press. ISBN 0-306-40060-X. [find](https://scholar.google.com/scholar?q=Herv%C3%A9+Gallaire+and+Jack+Minker%2C+editors%2E+Logic+and+Data+Bases%2C+Symposium+on+Logic+and+Data+Bases%2C+Centre+d%E2%80%99%C3%A9tudes+et+de+recherches+de+Toulouse%2C+1977%2C+Advances+in+Data+Base+Theory%2C+New+York%2C+1978%2E+Plem)
9. Stephen Muggleton. Inductive logic programming. New Generation Comput., 8(4):295–318, 1991. doi: 10.1007/BF03037089. — [doi:10.1007/bf03037089](https://doi.org/10.1007/bf03037089) [find](https://scholar.google.com/scholar?q=Stephen+Muggleton%2E+Inductive+logic+programming%2E+New+Generation+Comput%2E%2C+8%284%29%3A295%E2%80%93318%2C+1991%2E+doi%3A+10%2E1007%2FBF03037089%2E+%E2%80%94+%5Bdoi%3A10%2E1007%2Fbf03037089%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E1007%2Fbf03037089%29)
10. Stanley Kok and Pedro M. Domingos. Statistical predicate invention. In Machine Learning, Proceedings of the Twenty-Fourth International Conference (ICML 2007), Corvallis, Oregon, USA, June 20-24, 2007, pages 433–440, 2007. doi: 10.1145/1273496.1273551. — [doi:10.1145/1273496.1273551](https://doi.org/10.1145/1273496.1273551) [find](https://scholar.google.com/scholar?q=Stanley+Kok+and+Pedro+M%2E+Domingos%2E+Statistical+predicate+invention%2E+In+Machine+Learning%2C+Proceedings+of+the+Twenty%2DFourth+International+Conference+%28ICML+2007%29%2C+Corvallis%2C+Oregon%2C+USA%2C+June+20%2D24%2C+2007)
11. Matt Gardner, Partha Pratim Talukdar, Bryan Kisiel, and Tom M. Mitchell. Improving learning and inference in a large knowledge-base using latent syntactic cues. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, EMNLP 2013, 18-21 October 2013, Grand Hyatt Seat… [find](https://scholar.google.com/scholar?q=Matt+Gardner%2C+Partha+Pratim+Talukdar%2C+Bryan+Kisiel%2C+and+Tom+M%2E+Mitchell%2E+Improving+learning+and+inference+in+a+large+knowledge%2Dbase+using+latent+syntactic+cues%2E+In+Proceedings+of+the+2013+Conference+o)
12. Matt Gardner, Partha Pratim Talukdar, Jayant Krishnamurthy, and Tom M. Mitchell. Incorporating vector space similarity in random walk inference over knowledge bases. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qata… [find](https://scholar.google.com/scholar?q=Matt+Gardner%2C+Partha+Pratim+Talukdar%2C+Jayant+Krishnamurthy%2C+and+Tom+M%2E+Mitchell%2E+Incorporating+vector+space+similarity+in+random+walk+inference+over+knowledge+bases%2E+In+Proceedings+of+the+2014+Confere)
13. Islam Beltagy, Stephen Roller, Pengxiang Cheng, Katrin Erk, and Raymond J Mooney. [Representing meaning with a combination of logical and distributional models](https://md.research.elamur.ai/free/en/papers/beltagy-2016-representing-meaning-combination-logical-distributional/). Computational Linguistics, 2017.
14. Arvind Neelakantan, Benjamin Roth, and Andrew McCallum. [Compositional vector space models for knowledge base completion](https://md.research.elamur.ai/free/en/papers/neelakantan-2015-compositional-vector-space-models-knowledge/). In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian F…
15. Baolin Peng, Zhengdong Lu, Hang Li, and Kam-Fai Wong. Towards neural network-based reasoning. CoRR, abs/1508.05508, 2015. — [arXiv:1508.05508](https://arxiv.org/abs/1508.05508) [find](https://scholar.google.com/scholar?q=Baolin+Peng%2C+Zhengdong+Lu%2C+Hang+Li%2C+and+Kam%2DFai+Wong%2E+Towards+neural+network%2Dbased+reasoning%2E+CoRR%2C+abs%2F1508%2E05508%2C+2015%2E+%E2%80%94+%5BarXiv%3A1508%2E05508%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1508%2E05508%29)
16. Rajarshi Das, Arvind Neelakantan, David Belanger, and Andrew McCallum. Chains of reasoning over entities, relations, and text using recurrent neural networks. In Conference of the European Chapter of the Association for Computational Linguistics (EACL), 2017. [find](https://scholar.google.com/scholar?q=Rajarshi+Das%2C+Arvind+Neelakantan%2C+David+Belanger%2C+and+Andrew+McCallum%2E+Chains+of+reasoning+over+entities%2C+relations%2C+and+text+using+recurrent+neural+networks%2E+In+Conference+of+the+European+Chapter+of+)
17. Dirk Weissenborn. Separating answers from queries for neural reading comprehension. abs/1607.03316, 2016. — [arXiv:1607.03316](https://arxiv.org/abs/1607.03316) [find](https://scholar.google.com/scholar?q=Dirk+Weissenborn%2E+Separating+answers+from+queries+for+neural+reading+comprehension%2E+abs%2F1607%2E03316%2C+2016%2E+%E2%80%94+%5BarXiv%3A1607%2E03316%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1607%2E03316%29)
18. Yelong Shen, Po-Sen Huang, Jianfeng Gao, and Weizhu Chen. Reasonet: Learning to stop reading in machine comprehension. In Proceedings of the Workshop on Cognitive Computation: Integrating neural and symbolic approaches 2016 co-located with the 30th Annual Conference on Neural Information Processing… [find](https://scholar.google.com/scholar?q=Yelong+Shen%2C+Po%2DSen+Huang%2C+Jianfeng+Gao%2C+and+Weizhu+Chen%2E+Reasonet%3A+Learning+to+stop+reading+in+machine+comprehension%2E+In+Proceedings+of+the+Workshop+on+Cognitive+Computation%3A+Integrating+neural+and+s)
19. Alex Graves, Greg Wayne, and Ivo Danihelka. Neural turing machines. CoRR, abs/1410.5401, 2014. [find](https://scholar.google.com/scholar?q=Alex+Graves%2C+Greg+Wayne%2C+and+Ivo+Danihelka%2E+Neural+turing+machines%2E+CoRR%2C+abs%2F1410%2E5401%2C+2014%2E)
20. Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. CoRR, abs/1410.3916, 2014. [find](https://scholar.google.com/scholar?q=Jason+Weston%2C+Sumit+Chopra%2C+and+Antoine+Bordes%2E+Memory+networks%2E+CoRR%2C+abs%2F1410%2E3916%2C+2014%2E)
21. Edward Grefenstette, Karl Moritz Hermann, Mustafa Suleyman, and Phil Blunsom. Learning to transduce with unbounded memory. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pages 18… [find](https://scholar.google.com/scholar?q=Edward+Grefenstette%2C+Karl+Moritz+Hermann%2C+Mustafa+Suleyman%2C+and+Phil+Blunsom%2E+Learning+to+transduce+with+unbounded+memory%2E+In+Advances+in+Neural+Information+Processing+Systems+28%3A+Annual+Conference+on)
22. Armand Joulin and Tomas Mikolov. Inferring algorithmic patterns with stack-augmented recurrent nets. In Advances in Neural Information Processing Systems 28: Annual Conference on Neural Information Processing Systems 2015, December 7-12, 2015, Montreal, Quebec, Canada, pages 190–198, 2015. [find](https://scholar.google.com/scholar?q=Armand+Joulin+and+Tomas+Mikolov%2E+Inferring+algorithmic+patterns+with+stack%2Daugmented+recurrent+nets%2E+In+Advances+in+Neural+Information+Processing+Systems+28%3A+Annual+Conference+on+Neural+Information+Pr)
23. Arvind Neelakantan, Quoc V. Le, and Ilya Sutskever. Neural programmer: Inducing latent programs with gradient descent. In International Conference on Learning Representations (ICLR), 2016. [find](https://scholar.google.com/scholar?q=Arvind+Neelakantan%2C+Quoc+V%2E+Le%2C+and+Ilya+Sutskever%2E+Neural+programmer%3A+Inducing+latent+programs+with+gradient+descent%2E+In+International+Conference+on+Learning+Representations+%28ICLR%29%2C+2016%2E)
24. Scott E. Reed and Nando de Freitas. Neural programmer-interpreters. In International Conference on Learning Representations (ICLR), 2016. [find](https://scholar.google.com/scholar?q=Scott+E%2E+Reed+and+Nando+de+Freitas%2E+Neural+programmer%2Dinterpreters%2E+In+International+Conference+on+Learning+Representations+%28ICLR%29%2C+2016%2E)
25. Marcin Andrychowicz, Misha Denil, Sergio Gomez Colmenarejo, Matthew W. Hoffman, David Pfau, Tom Schaul, and Nando de Freitas. Learning to learn by gradient descent by gradient descent. In Advances in Neural Information Processing Systems 29: Annual Conference on Neural Information Processing Systems… [find](https://scholar.google.com/scholar?q=Marcin+Andrychowicz%2C+Misha+Denil%2C+Sergio+Gomez+Colmenarejo%2C+Matthew+W%2E+Hoffman%2C+David+Pfau%2C+Tom+Schaul%2C+and+Nando+de+Freitas%2E+Learning+to+learn+by+gradient+descent+by+gradient+descent%2E+In+Advances+in+)
26. Matko Bosnjak, Tim Rocktäschel, Jason Naradowsky, and Sebastian Riedel. Programming with a differentiable forth interpreter. In International Conference on Machine Learning (ICML), 2017. [find](https://scholar.google.com/scholar?q=Matko+Bosnjak%2C+Tim+Rockt%C3%A4schel%2C+Jason+Naradowsky%2C+and+Sebastian+Riedel%2E+Programming+with+a+differentiable+forth+interpreter%2E+In+International+Conference+on+Machine+Learning+%28ICML%29%2C+2017%2E)
27. Stuart J. Russell and Peter Norvig. Artificial Intelligence - A Modern Approach (3. internat. ed.). Pearson Education, 2010. ISBN 978-0-13-207148-2. [find](https://scholar.google.com/scholar?q=Stuart+J%2E+Russell+and+Peter+Norvig%2E+Artificial+Intelligence+%2D+A+Modern+Approach+%283%2E+internat%2E+ed%2E%29%2E+Pearson+Education%2C+2010%2E+ISBN+978%2D0%2D13%2D207148%2D2%2E)
28. Lise Getoor. Introduction to statistical relational learning. MIT press, 2007. [find](https://scholar.google.com/scholar?q=Lise+Getoor%2E+Introduction+to+statistical+relational+learning%2E+MIT+press%2C+2007%2E)
29. Jacob Andreas, Marcus Rohrbach, Trevor Darrell, and Dan Klein. [Learning to compose neural networks for question answering](https://md.research.elamur.ai/free/en/papers/andreas-2016-learning-compose-neural-networks-question/). In NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, Ju…
30. David S Broomhead and David Lowe. Radial basis functions, multi-variable functional interpolation and adaptive networks. Technical report, DTIC Document, 1988. [find](https://scholar.google.com/scholar?q=David+S+Broomhead+and+David+Lowe%2E+Radial+basis+functions%2C+multi%2Dvariable+functional+interpolation+and+adaptive+networks%2E+Technical+report%2C+DTIC+Document%2C+1988%2E)
31. Allen Van Gelder. Efficient loop detection in prolog using the tortoise-and-hare technique. J. Log. Program., 4(1):23–31, 1987. doi: 10.1016/0743-1066(87)90020-3. — [doi:10.1016/0743-1066(87)90020-3](https://doi.org/10.1016/0743-1066%2887%2990020-3) [find](https://scholar.google.com/scholar?q=Allen+Van+Gelder%2E+Efficient+loop+detection+in+prolog+using+the+tortoise%2Dand%2Dhare+technique%2E+J%2E+Log%2E+Program%2E%2C+4%281%29%3A23%E2%80%9331%2C+1987%2E+doi%3A+10%2E1016%2F0743%2D1066%2887%2990020%2D3%2E+%E2%80%94+%5Bdoi%3A10%2E1016%2F0743%2D1066%2887%2990020%2D3%5D%28)
32. J. Ross Quinlan. Learning logical definitions from relations. Machine Learning, 5:239–266, 1990. doi: 10.1007/BF00117105. [find](https://scholar.google.com/scholar?q=J%2E+Ross+Quinlan%2E+Learning+logical+definitions+from+relations%2E+Machine+Learning%2C+5%3A239%E2%80%93266%2C+1990%2E+doi%3A+10%2E1007%2FBF00117105%2E)
33. William Yang Wang and William W. Cohen. Joint information extraction and reasoning: A scalable statistical relational learning approach. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processin… [find](https://scholar.google.com/scholar?q=William+Yang+Wang+and+William+W%2E+Cohen%2E+Joint+information+extraction+and+reasoning%3A+A+scalable+statistical+relational+learning+approach%2E+In+Proceedings+of+the+53rd+Annual+Meeting+of+the+Association+fo)
34. Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. [Translating embeddings for modeling multi-relational data](https://md.research.elamur.ai/free/en/papers/bordes-2013-translating-embeddings-modeling-multi-relational/). In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a meetin…
35. Guillaume Bouchard, Sameer Singh, and Theo Trouillon. On approximate reasoning capabilities of low-rank vector spaces. In Proceedings of the 2015 AAAI Spring Symposium on Knowledge Representation and Reasoning (KRR): Integrating Symbolic and Neural Approaches, 2015. [find](https://scholar.google.com/scholar?q=Guillaume+Bouchard%2C+Sameer+Singh%2C+and+Theo+Trouillon%2E+On+approximate+reasoning+capabilities+of+low%2Drank+vector+spaces%2E+In+Proceedings+of+the+2015+AAAI+Spring+Symposium+on+Knowledge+Representation+and+)
36. Maximilian Nickel, Lorenzo Rosasco, and Tomaso A. Poggio. Holographic embeddings of knowledge graphs. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, February 12-17, 2016, Phoenix, Arizona, USA., pages 1955–1961, 2016. [find](https://scholar.google.com/scholar?q=Maximilian+Nickel%2C+Lorenzo+Rosasco%2C+and+Tomaso+A%2E+Poggio%2E+Holographic+embeddings+of+knowledge+graphs%2E+In+Proceedings+of+the+Thirtieth+AAAI+Conference+on+Artificial+Intelligence%2C+February+12%2D17%2C+2016%2C+)
37. Fan Yang, Zhilin Yang, and William W. Cohen. Differentiable learning of logical rules for knowledge base completion. CoRR, abs/1702.08367, 2017. [find](https://scholar.google.com/scholar?q=Fan+Yang%2C+Zhilin+Yang%2C+and+William+W%2E+Cohen%2E+Differentiable+learning+of+logical+rules+for+knowledge+base+completion%2E+CoRR%2C+abs%2F1702%2E08367%2C+2017%2E)
38. Artur S. d’Avila Garcez, Krysia Broda, and Dov M. Gabbay. Neural-symbolic learning systems: foundations and applications. Springer Science & Business Media, 2012. [find](https://scholar.google.com/scholar?q=Artur+S%2E+d%E2%80%99Avila+Garcez%2C+Krysia+Broda%2C+and+Dov+M%2E+Gabbay%2E+Neural%2Dsymbolic+learning+systems%3A+foundations+and+applications%2E+Springer+Science+%26+Business+Media%2C+2012%2E)
39. Jude W Shavlik and Geoffrey G Towell. An approach to combining explanation-based and neural learning algorithms. Connection Science, 1(3):231–253, 1989. [find](https://scholar.google.com/scholar?q=Jude+W+Shavlik+and+Geoffrey+G+Towell%2E+An+approach+to+combining+explanation%2Dbased+and+neural+learning+algorithms%2E+Connection+Science%2C+1%283%29%3A231%E2%80%93253%2C+1989%2E)
40. Geoffrey G. Towell and Jude W. Shavlik. Knowledge-based artificial neural networks. Artif. Intell., 70 (1-2):119–165, 1994. doi: 10.1016/0004-3702(94)90105-8. — [doi:10.1016/0004-3702(94)90105-8](https://doi.org/10.1016/0004-3702%2894%2990105-8) [find](https://scholar.google.com/scholar?q=Geoffrey+G%2E+Towell+and+Jude+W%2E+Shavlik%2E+Knowledge%2Dbased+artificial+neural+networks%2E+Artif%2E+Intell%2E%2C+70+%281%2D2%29%3A119%E2%80%93165%2C+1994%2E+doi%3A+10%2E1016%2F0004%2D3702%2894%2990105%2D8%2E+%E2%80%94+%5Bdoi%3A10%2E1016%2F0004%2D3702%2894%2990105%2D8%5D%28http)
41. Artur S. d’Avila Garcez and Gerson Zaverucha. The connectionist inductive learning and logic programming system. Appl. Intell., 11(1):59–77, 1999. doi: 10.1023/A:1008328630915. — [doi:10.1023/a:1008328630915](https://doi.org/10.1023/a:1008328630915) [find](https://scholar.google.com/scholar?q=Artur+S%2E+d%E2%80%99Avila+Garcez+and+Gerson+Zaverucha%2E+The+connectionist+inductive+learning+and+logic+programming+system%2E+Appl%2E+Intell%2E%2C+11%281%29%3A59%E2%80%9377%2C+1999%2E+doi%3A+10%2E1023%2FA%3A1008328630915%2E+%E2%80%94+%5Bdoi%3A10%2E1023%2Fa%3A100832)
42. Lokendra Shastri. Neurally motivated constraints on the working memory capacity of a production system for parallel processing: Implications of a connectionist model based on temporal synchrony. In Proceedings of the Fourteenth Annual Conference of the Cognitive Science Society: July 29 to August 1,… [find](https://scholar.google.com/scholar?q=Lokendra+Shastri%2E+Neurally+motivated+constraints+on+the+working+memory+capacity+of+a+production+system+for+parallel+processing%3A+Implications+of+a+connectionist+model+based+on+temporal+synchrony%2E+In+Pr)
43. Liya Ding. Neural prolog-the concepts, construction and mechanism. In Systems, Man and Cybernetics, 1995. Intelligent Systems for the 21st Century., IEEE International Conference on, volume 4, pages 3603–3608. IEEE, 1995. [find](https://scholar.google.com/scholar?q=Liya+Ding%2E+Neural+prolog%2Dthe+concepts%2C+construction+and+mechanism%2E+In+Systems%2C+Man+and+Cybernetics%2C+1995%2E+Intelligent+Systems+for+the+21st+Century%2E%2C+IEEE+International+Conference+on%2C+volume+4%2C+pages+3)
44. Manoel V. M. França, Gerson Zaverucha, and Artur S. d’Avila Garcez. Fast relational learning using bottom clause propositionalization with artificial neural networks. Machine Learning, 94(1):81–104, 2014. doi: 10.1007/s10994-013-5392-1. — [doi:10.1007/s10994-013-5392-1](https://doi.org/10.1007/s10994-013-5392-1) [find](https://scholar.google.com/scholar?q=Manoel+V%2E+M%2E+Fran%C3%A7a%2C+Gerson+Zaverucha%2C+and+Artur+S%2E+d%E2%80%99Avila+Garcez%2E+Fast+relational+learning+using+bottom+clause+propositionalization+with+artificial+neural+networks%2E+Machine+Learning%2C+94%281%29%3A81%E2%80%93104%2C+2)
45. Gustav Sourek, Vojtech Aschenbrenner, Filip Zelezný, and Ondrej Kuzelka. Lifted relational neural networks. In Proceedings of the NIPS Workshop on Cognitive Computation: Integrating Neural and Symbolic Approaches co-located with the 29th Annual Conference on Neural Information Processing Systems (NI… [find](https://scholar.google.com/scholar?q=Gustav+Sourek%2C+Vojtech+Aschenbrenner%2C+Filip+Zelezn%C3%BD%2C+and+Ondrej+Kuzelka%2E+Lifted+relational+neural+networks%2E+In+Proceedings+of+the+NIPS+Workshop+on+Cognitive+Computation%3A+Integrating+Neural+and+Symboli)
46. William W. Cohen. Tensorlog: A differentiable deductive database. CoRR, abs/1605.06523, 2016. — [arXiv:1605.06523](https://arxiv.org/abs/1605.06523) [find](https://scholar.google.com/scholar?q=William+W%2E+Cohen%2E+Tensorlog%3A+A+differentiable+deductive+database%2E+CoRR%2C+abs%2F1605%2E06523%2C+2016%2E+%E2%80%94+%5BarXiv%3A1605%2E06523%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1605%2E06523%29)
47. Luciano Serafini and Artur S. d’Avila Garcez. Logic tensor networks: Deep learning and logical reasoning from data and knowledge. In Proceedings of the 11th International Workshop on Neural-Symbolic Learning and Reasoning (NeSy’16) co-located with the Joint Multi-Conference on Human-Level Artificial… [find](https://scholar.google.com/scholar?q=Luciano+Serafini+and+Artur+S%2E+d%E2%80%99Avila+Garcez%2E+Logic+tensor+networks%3A+Deep+learning+and+logical+reasoning+from+data+and+knowledge%2E+In+Proceedings+of+the+11th+International+Workshop+on+Neural%2DSymbolic+L)
48. Ni Lao, Tom M. Mitchell, and William W. Cohen. Random walk inference and learning in A large scale knowledge base. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, EMNLP 2011, 27-31 July 2011, John McIntyre Conference Centre, Edinburgh, UK, A meeting of SIGD… [find](https://scholar.google.com/scholar?q=Ni+Lao%2C+Tom+M%2E+Mitchell%2C+and+William+W%2E+Cohen%2E+Random+walk+inference+and+learning+in+A+large+scale+knowledge+base%2E+In+Proceedings+of+the+2011+Conference+on+Empirical+Methods+in+Natural+Language+Proces)
49. Ni Lao, Amarnag Subramanya, Fernando C. N. Pereira, and William W. Cohen. Reading the web with learned syntactic-semantic inference rules. In Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, EMNLP-CoNLL 2012, Ju… [find](https://scholar.google.com/scholar?q=Ni+Lao%2C+Amarnag+Subramanya%2C+Fernando+C%2E+N%2E+Pereira%2C+and+William+W%2E+Cohen%2E+Reading+the+web+with+learned+syntactic%2Dsemantic+inference+rules%2E+In+Proceedings+of+the+2012+Joint+Conference+on+Empirical+Meth)
50. Tim Rocktäschel, Matko Bosnjak, Sameer Singh, and Sebastian Riedel. Low-Dimensional Embeddings of Logic. In ACL Workshop on Semantic Parsing (SP’14), 2014. [find](https://scholar.google.com/scholar?q=Tim+Rockt%C3%A4schel%2C+Matko+Bosnjak%2C+Sameer+Singh%2C+and+Sebastian+Riedel%2E+Low%2DDimensional+Embeddings+of+Logic%2E+In+ACL+Workshop+on+Semantic+Parsing+%28SP%E2%80%9914%29%2C+2014%2E)
51. Tim Rocktäschel, Sameer Singh, and Sebastian Riedel. Injecting logical background knowledge into embeddings for relation extraction. In NAACL HLT 2015, The 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Denver, Colorado, U… [find](https://scholar.google.com/scholar?q=Tim+Rockt%C3%A4schel%2C+Sameer+Singh%2C+and+Sebastian+Riedel%2E+Injecting+logical+background+knowledge+into+embeddings+for+relation+extraction%2E+In+NAACL+HLT+2015%2C+The+2015+Conference+of+the+North+American+Chapte)
52. Ivan Vendrov, Ryan Kiros, Sanja Fidler, and Raquel Urtasun. [Order-embeddings of images and language](https://md.research.elamur.ai/free/en/papers/vendrov-2015-order-embeddings-images-language/). In International Conference on Learning Representations (ICLR), 2016.
53. Zhiting Hu, Xuezhe Ma, Zhengzhong Liu, Eduard H. Hovy, and Eric P. Xing. [Harnessing deep neural networks with logic rules](https://md.research.elamur.ai/free/en/papers/hu-2016-harnessing-deep-neural-networks-logic/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, ACL 2016, August 7-12, 2016, Berlin, Germany, Volume 1: Long Papers, 2016.
54. Thomas Demeester, Tim Rocktäschel, and Sebastian Riedel. Lifted rule injection for relation embeddings. In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 1389–1399, 2016. [find](https://scholar.google.com/scholar?q=Thomas+Demeester%2C+Tim+Rockt%C3%A4schel%2C+and+Sebastian+Riedel%2E+Lifted+rule+injection+for+relation+embeddings%2E+In+Proceedings+of+the+2016+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+EMNLP)
55. Ekaterina Komendantskaya. Unification neural networks: unification by error-correction learning. Logic Journal of the IGPL, 19(6):821–847, 2011. doi: 10.1093/jigpal/jzq012. — [doi:10.1093/jigpal/jzq012](https://doi.org/10.1093/jigpal/jzq012) [find](https://scholar.google.com/scholar?q=Ekaterina+Komendantskaya%2E+Unification+neural+networks%3A+unification+by+error%2Dcorrection+learning%2E+Logic+Journal+of+the+IGPL%2C+19%286%29%3A821%E2%80%93847%2C+2011%2E+doi%3A+10%2E1093%2Fjigpal%2Fjzq012%2E+%E2%80%94+%5Bdoi%3A10%2E1093%2Fjigpal%2Fjzq01)
56. Steffen Hölldobler. A structured connectionist unification algorithm. In Proceedings of the 8th National Conference on Artificial Intelligence. Boston, Massachusetts, July 29 - August 3, 1990, 2 Volumes., pages 587–593, 1990. [find](https://scholar.google.com/scholar?q=Steffen+H%C3%B6lldobler%2E+A+structured+connectionist+unification+algorithm%2E+In+Proceedings+of+the+8th+National+Conference+on+Artificial+Intelligence%2E+Boston%2C+Massachusetts%2C+July+29+%2D+August+3%2C+1990%2C+2+Volum)
57. Stefan Schoenmackers, Jesse Davis, Oren Etzioni, and Daniel S. Weld. Learning first-order horn clauses from web text. In Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, EMNLP 2010, 9-11 October 2010, MIT Stata Center, Massachusetts, USA, A meeting of SIGDAT, a… [find](https://scholar.google.com/scholar?q=Stefan+Schoenmackers%2C+Jesse+Davis%2C+Oren+Etzioni%2C+and+Daniel+S%2E+Weld%2E+Learning+first%2Dorder+horn+clauses+from+web+text%2E+In+Proceedings+of+the+2010+Conference+on+Empirical+Methods+in+Natural+Language+Pro)
58. Stephen H Muggleton, Dianhuan Lin, and Alireza Tamaddoni-Nezhad. Meta-interpretive learning of higher-order dyadic datalog: Predicate invention revisited. Machine Learning, 100(1):49–73, 2015. [find](https://scholar.google.com/scholar?q=Stephen+H+Muggleton%2C+Dianhuan+Lin%2C+and+Alireza+Tamaddoni%2DNezhad%2E+Meta%2Dinterpretive+learning+of+higher%2Dorder+dyadic+datalog%3A+Predicate+invention+revisited%2E+Machine+Learning%2C+100%281%29%3A49%E2%80%9373%2C+2015%2E)
59. Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735–1780, 1997. doi: 10.1162/neco.1997.9.8.1735. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+J%C3%BCrgen+Schmidhuber%2E+Long+short%2Dterm+memory%2E+Neural+Computation%2C+9%288%29%3A1735%E2%80%931780%2C+1997%2E+doi%3A+10%2E1162%2Fneco%2E1997%2E9%2E8%2E1735%2E)
60. Rémi Coulom. Efficient selectivity and backup operators in monte-carlo tree search. In Computers and Games, 5th International Conference, CG 2006, Turin, Italy, May 29-31, 2006. Revised Papers, pages 72–83, 2006. doi: 10.1007/978-3-540-75538-8_7. — [doi:10.1007/978-3-540-75538-8_7](https://doi.org/10.1007/978-3-540-75538-8_7) [find](https://scholar.google.com/scholar?q=R%C3%A9mi+Coulom%2E+Efficient+selectivity+and+backup+operators+in+monte%2Dcarlo+tree+search%2E+In+Computers+and+Games%2C+5th+International+Conference%2C+CG+2006%2C+Turin%2C+Italy%2C+May+29%2D31%2C+2006%2E+Revised+Papers%2C+pages+)
61. Levente Kocsis and Csaba Szepesvári. Bandit based monte-carlo planning. In Machine Learning: ECML 2006, 17th European Conference on Machine Learning, Berlin, Germany, September 18-22, 2006, Proceedings, pages 282–293, 2006. doi: 10.1007/11871842_29. — [doi:10.1007/11871842_29](https://doi.org/10.1007/11871842_29) [find](https://scholar.google.com/scholar?q=Levente+Kocsis+and+Csaba+Szepesv%C3%A1ri%2E+Bandit+based+monte%2Dcarlo+planning%2E+In+Machine+Learning%3A+ECML+2006%2C+17th+European+Conference+on+Machine+Learning%2C+Berlin%2C+Germany%2C+September+18%2D22%2C+2006%2C+Proceeding)
62. David Silver, Aja Huang, Chris J. Maddison, Arthur Guez, Laurent Sifre, George van den Driessche, Julian Schrittwieser, Ioannis Antonoglou, Vedavyas Panneershelvam, Marc Lanctot, Sander Dieleman, Dominik Grewe, John Nham, Nal Kalchbrenner, Ilya Sutskever, Timothy P. Lillicrap, Madeleine Leach, Koray… [find](https://scholar.google.com/scholar?q=David+Silver%2C+Aja+Huang%2C+Chris+J%2E+Maddison%2C+Arthur+Guez%2C+Laurent+Sifre%2C+George+van+den+Driessche%2C+Julian+Schrittwieser%2C+Ioannis+Antonoglou%2C+Vedavyas+Panneershelvam%2C+Marc+Lanctot%2C+Sander+Dieleman%2C+Domi)
63. Marwin H. S. Segler, Mike Preuß, and Mark P. Waller. Towards "alphachem": Chemical synthesis planning with tree search and deep neural network policies. CoRR, abs/1702.00020, 2017. — [arXiv:1702.00020](https://arxiv.org/abs/1702.00020) [find](https://scholar.google.com/scholar?q=Marwin+H%2E+S%2E+Segler%2C+Mike+Preu%C3%9F%2C+and+Mark+P%2E+Waller%2E+Towards+%22alphachem%22%3A+Chemical+synthesis+planning+with+tree+search+and+deep+neural+network+policies%2E+CoRR%2C+abs%2F1702%2E00020%2C+2017%2E+%E2%80%94+%5BarXiv%3A1702%2E00020)
64. Mark E. Stickel. A prolog technology theorem prover. New Generation Comput., 2(4):371–383, 1984. doi: 10.1007/BF03037328. — [doi:10.1007/bf03037328](https://doi.org/10.1007/bf03037328) [find](https://scholar.google.com/scholar?q=Mark+E%2E+Stickel%2E+A+prolog+technology+theorem+prover%2E+New+Generation+Comput%2E%2C+2%284%29%3A371%E2%80%93383%2C+1984%2E+doi%3A+10%2E1007%2FBF03037328%2E+%E2%80%94+%5Bdoi%3A10%2E1007%2Fbf03037328%5D%28https%3A%2F%2Fdoi%2Eorg%2F10%2E1007%2Fbf03037328%29)
65. Cezary Kaliszyk, François Chollet, and Christian Szegedy. Holstep: A machine learning dataset for higher-order logic theorem proving. In International Conference on Learning Representations (ICLR), 2017. [find](https://scholar.google.com/scholar?q=Cezary+Kaliszyk%2C+Fran%C3%A7ois+Chollet%2C+and+Christian+Szegedy%2E+Holstep%3A+A+machine+learning+dataset+for+higher%2Dorder+logic+theorem+proving%2E+In+International+Conference+on+Learning+Representations+%28ICLR%29%2C+20)
66. Sarah M. Loos, Geoffrey Irving, Christian Szegedy, and Cezary Kaliszyk. In International Conferences on Logic for Programming, Artificial Intelligence and Reasoning (LPAR), 2017. [find](https://scholar.google.com/scholar?q=Sarah+M%2E+Loos%2C+Geoffrey+Irving%2C+Christian+Szegedy%2C+and+Cezary+Kaliszyk%2E+In+International+Conferences+on+Logic+for+Programming%2C+Artificial+Intelligence+and+Reasoning+%28LPAR%29%2C+2017%2E)
67. Diederik P. Kingma and Jimmy Ba. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). In International Conference on Learning Representations (ICLR), 2015.
68. Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, AISTATS 2010, Chia Laguna Resort, Sardinia, Italy, May 13-15, 2010, pages 249–256, 2010. [find](https://scholar.google.com/scholar?q=Xavier+Glorot+and+Yoshua+Bengio%2E+Understanding+the+difficulty+of+training+deep+feedforward+neural+networks%2E+In+Proceedings+of+the+Thirteenth+International+Conference+on+Artificial+Intelligence+and+Sta)
69. Martín Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Gregory S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian J. Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Józefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Le… — [TensorFlow: Large-Scale Machine Learning on Heterogeneous Distributed Systems](https://md.research.elamur.ai/free/en/papers/abadi-2016-tensorflow-large-scale-machine-learning/)

*Updated 2 November 2026*