---
title: "Graph Attention Networks — Veličkovic 2017"
description: "The authors propose Graph Attention Networks (GATs). 44 references, cited by 339 papers here."
authors: ["Petar Veli\\vckovic", "Guillem Cucurull", "Arantxa Casanova", "Adriana Romero", "Pietro Lio", "Yoshua Bengio"]
year: 2017
venue: "arXiv:1710.10903"
source: "https://arxiv.org/abs/1710.10903"
doi: "10.48550/arxiv.1710.10903"
arxiv: "1710.10903"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/veli-vckovic-2017-graph-attention-networks/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/veli-vckovic-2017-graph-attention-networks/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/) / Graph Attention Networks

# Graph Attention Networks

**Authors:** Petar Veli\\vckovic, [Guillem Cucurull](https://md.research.elamur.ai/free/en/authors/guillem-cucurull/), Arantxa Casanova, Adriana Romero, Pietro Lio and [Yoshua Bengio](https://md.research.elamur.ai/free/en/authors/yoshua-bengio/) · **Year:** 2017 · **Venue:** arXiv:1710.10903 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1710.10903) · [PDF](https://arxiv.org/pdf/1710.10903) · [DOI 10.48550/arxiv.1710.10903](https://doi.org/10.48550/arxiv.1710.10903) · [arXiv 1710.10903](https://arxiv.org/abs/1710.10903) · [OpenAlex](https://openalex.org/W2766453196)

**Citations:** 8,657 citations (OpenAlex, 2026-10-14) · [339](https://md.research.elamur.ai/free/en/papers/veli-vckovic-2017-graph-attention-networks/cited-by/) in our library, of the 140,884 papers whose bibliographies we read

## Summary
Earlier graph neural networks had two main weaknesses. Spectral filters are tied to one graph's Laplacian, so they cannot transfer to graphs with a different structure. Spatial operators have trouble with neighbourhoods of varying size. The authors propose Graph Attention Networks (GATs). In each GAT layer, a node scores how important each immediate neighbour is (including itself) and then aggregates the neighbours' linearly transformed features as a weighted sum. A small feedforward network with LeakyReLU produces the scores, and a softmax over the neighbourhood normalises them. Several independent attention heads run in parallel to make training more stable. The layer needs no eigendecomposition or other expensive matrix operations, can be parallelised across edges, and does not need the whole graph in advance. The authors tested GATs on three transductive citation benchmarks (Cora, Citeseer, Pubmed) and on an inductive protein-protein interaction (PPI) dataset whose test graphs are unseen during training. GATs matched or beat strong baselines such as GCN, MoNet and GraphSAGE. The paper matters because it made attention a standard building block for learning on graphs.

## Key points
- **Core mechanism:** attention is masked to first-order neighbours, and scores are computed from the concatenated, linearly projected features of node pairs. LeakyReLU uses a negative slope of 0.2.
- **Multi-head attention:** hidden layers concatenate the outputs of K heads, while the prediction layer averages them. This follows the approach of the Transformer (Vaswani et al., 2017).
- **Transductive results (classification accuracy):** about 83.0% on Cora, 72.5% on Citeseer and 79.0% on Pubmed. This equals or exceeds GCN and MoNet. These figures come from the paper's results tables, which were missing from the extracted text you provided, so it's worth checking them against the PDF.
- **Inductive result:** on PPI, which has 24 graphs, 121 labels and unseen test graphs, GAT reaches a micro-F1 of about 0.973. This is well above GraphSAGE. The figure is also from the missing results section and should be checked against the PDF.
- **Positioning:** GAT can be viewed as a particular instance of MoNet, and it relates to relational networks, VAIN and memory networks. A key advantage over spectral methods such as ChebNet and GCN is that it handles inductive tasks.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Other](https://md.research.elamur.ai/free/en/shelves/other/)

## Institutions

- [University of Cambridge](https://md.research.elamur.ai/free/en/universities/university-of-cambridge/) — University, Cambridge, United Kingdom · 119 works · 110,950 citations
- [Université de Montréal](https://md.research.elamur.ai/free/en/universities/universite-de-montreal/) — University, Montreal, Canada · 42 works · 71,559 citations
- [Universitat Politècnica de Catalunya](https://md.research.elamur.ai/free/en/universities/universitat-politecnica-de-catalunya/) — University, Barcelona, Spain · 22 works · 15,068 citations
- [Universitat Autònoma de Barcelona](https://md.research.elamur.ai/free/en/universities/universitat-autonoma-de-barcelona/) — University, Cerdanyola del Vallès, Spain · 2 works · 9,816 citations
- [HEC Montréal](https://md.research.elamur.ai/free/en/universities/hec-montreal/) — University, Montreal, Canada · 2 works · 9,514 citations

## Cite

Cite as: Petar Veli\\vckovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio and Yoshua Bengio. 2017. Graph Attention Networks. arXiv:1710.10903.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{velivckovic2017graph,
  title = {{Graph Attention Networks}},
  author = {Petar Velivckovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio and Yoshua Bengio},
  year = {2017},
  doi = {10.48550/arxiv.1710.10903},
  eprint = {1710.10903},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.1710.10903}
}
```

- [← previous MobileNets: Efficient Convolutional Neural Networks for Mobile Vision Applications](https://md.research.elamur.ai/free/en/papers/howard-2017-mobilenets-efficient-convolutional-neural-networks/)
- [next → A Unified Approach to Interpreting Model Predictions](https://md.research.elamur.ai/free/en/papers/lundberg-2017-unified-approach-interpreting-model-predictions/)
- [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/)

## References

44 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Martı́n Abadi, Ashish Agarwal, Paul Barham, Eugene Brevdo, Zhifeng Chen, Craig Citro, Greg S. Corrado, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Ian Goodfellow, Andrew Harp, Geoffrey Irving, Michael Isard, Yangqing Jia, Rafal Jozefowicz, Lukasz Kaiser, Manjunath Kudlur, Josh Levenbe… [find](https://scholar.google.com/scholar?q=Mart%C4%B1%CC%81n+Abadi%2C+Ashish+Agarwal%2C+Paul+Barham%2C+Eugene+Brevdo%2C+Zhifeng+Chen%2C+Craig+Citro%2C+Greg+S%2E+Corrado%2C+Andy+Davis%2C+Jeffrey+Dean%2C+Matthieu+Devin%2C+Sanjay+Ghemawat%2C+Ian+Goodfellow%2C+Andrew+Harp%2C+Geoffrey+)
2. James Atwood and Don Towsley. Diffusion-convolutional neural networks. In Advances in Neural Information Processing Systems, pp. 1993–2001, 2016. [find](https://scholar.google.com/scholar?q=James+Atwood+and+Don+Towsley%2E+Diffusion%2Dconvolutional+neural+networks%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pp%2E+1993%E2%80%932001%2C+2016%2E)
3. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. Neural machine translation by jointly learning to align and translate. International Conference on Learning Representations (ICLR), 2015. [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
4. Mikhail Belkin, Partha Niyogi, and Vikas Sindhwani. Manifold regularization: A geometric framework for learning from labeled and unlabeled examples. Journal of machine learning research, 7 (Nov):2399–2434, 2006. [find](https://scholar.google.com/scholar?q=Mikhail+Belkin%2C+Partha+Niyogi%2C+and+Vikas+Sindhwani%2E+Manifold+regularization%3A+A+geometric+framework+for+learning+from+labeled+and+unlabeled+examples%2E+Journal+of+machine+learning+research%2C+7+%28Nov%29%3A2399%E2%80%93)
5. Joan Bruna, Wojciech Zaremba, Arthur Szlam, and Yann LeCun. Spectral networks and locally connected networks on graphs. International Conference on Learning Representations (ICLR), 2014. [find](https://scholar.google.com/scholar?q=Joan+Bruna%2C+Wojciech+Zaremba%2C+Arthur+Szlam%2C+and+Yann+LeCun%2E+Spectral+networks+and+locally+connected+networks+on+graphs%2E+International+Conference+on+Learning+Representations+%28ICLR%29%2C+2014%2E)
6. Jianpeng Cheng, Li Dong, and Mirella Lapata. Long short-term memory-networks for machine reading. arXiv preprint arXiv:1601.06733, 2016. [our card](https://md.research.elamur.ai/free/en/papers/cheng-2016-long-short-term-memory-networks/)
7. Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014. [find](https://scholar.google.com/scholar?q=Kyunghyun+Cho%2C+Bart+Van+Merrie%CC%88nboer%2C+Caglar+Gulcehre%2C+Dzmitry+Bahdanau%2C+Fethi+Bougares%2C+Holger+Schwenk%2C+and+Yoshua+Bengio%2E+Learning+phrase+representations+using+rnn+encoder%2Ddecoder+for+statistical+ma)
8. Djork-Arné Clevert, Thomas Unterthiner, and Sepp Hochreiter. Fast and accurate deep network learning by exponential linear units (elus). International Conference on Learning Representations (ICLR), 2016. [our card](https://md.research.elamur.ai/free/en/papers/clevert-2015-fast-accurate-deep-network-learning/)
9. Michaël Defferrard, Xavier Bresson, and Pierre Vandergheynst. Convolutional neural networks on graphs with fast localized spectral filtering. In Advances in Neural Information Processing Systems, pp. 3844–3852, 2016. [our card](https://md.research.elamur.ai/free/en/papers/defferrard-2016-convolutional-neural-networks-graphs-fast/)
10. Misha Denil, Sergio Gómez Colmenarejo, Serkan Cabi, David Saxton, and Nando de Freitas. Programmable agents. arXiv preprint arXiv:1706.06383, 2017. — [arXiv:1706.06383](https://arxiv.org/abs/1706.06383) [find](https://scholar.google.com/scholar?q=Misha+Denil%2C+Sergio+Go%CC%81mez+Colmenarejo%2C+Serkan+Cabi%2C+David+Saxton%2C+and+Nando+de+Freitas%2E+Programmable+agents%2E+arXiv+preprint+arXiv%3A1706%2E06383%2C+2017%2E+%E2%80%94+%5BarXiv%3A1706%2E06383%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1706%2E063)
11. Yan Duan, Marcin Andrychowicz, Bradly Stadie, Jonathan Ho, Jonas Schneider, Ilya Sutskever, Pieter Abbeel, and Wojciech Zaremba. One-shot imitation learning. arXiv preprint arXiv:1703.07326, 2017. — [arXiv:1703.07326](https://arxiv.org/abs/1703.07326) [find](https://scholar.google.com/scholar?q=Yan+Duan%2C+Marcin+Andrychowicz%2C+Bradly+Stadie%2C+Jonathan+Ho%2C+Jonas+Schneider%2C+Ilya+Sutskever%2C+Pieter+Abbeel%2C+and+Wojciech+Zaremba%2E+One%2Dshot+imitation+learning%2E+arXiv+preprint+arXiv%3A1703%2E07326%2C+2017%2E+%E2%80%94+%5B)
12. David K Duvenaud, Dougal Maclaurin, Jorge Iparraguirre, Rafael Bombarell, Timothy Hirzel, Alán Aspuru-Guzik, and Ryan P Adams. Convolutional networks on graphs for learning molecular fingerprints. In Advances in neural information processing systems, pp. 2224–2232, 2015. [find](https://scholar.google.com/scholar?q=David+K+Duvenaud%2C+Dougal+Maclaurin%2C+Jorge+Iparraguirre%2C+Rafael+Bombarell%2C+Timothy+Hirzel%2C+Ala%CC%81n+Aspuru%2DGuzik%2C+and+Ryan+P+Adams%2E+Convolutional+networks+on+graphs+for+learning+molecular+fingerprints%2E+In)
13. Paolo Frasconi, Marco Gori, and Alessandro Sperduti. A general framework for adaptive processing of data structures. IEEE transactions on Neural Networks, 9(5):768–786, 1998. [find](https://scholar.google.com/scholar?q=Paolo+Frasconi%2C+Marco+Gori%2C+and+Alessandro+Sperduti%2E+A+general+framework+for+adaptive+processing+of+data+structures%2E+IEEE+transactions+on+Neural+Networks%2C+9%285%29%3A768%E2%80%93786%2C+1998%2E)
14. Jonas Gehring, Michael Auli, David Grangier, and Yann N. Dauphin. A convolutional encoder model for neural machine translation. CoRR, abs/1611.02344, 2016. URL http://arxiv.org/abs/1611.02344. [↗](http://arxiv.org/abs/1611.02344) [our card](https://md.research.elamur.ai/free/en/papers/gehring-2017-convolutional-encoder-model-neural-machine/)
15. Xavier Glorot and Yoshua Bengio. Understanding the difficulty of training deep feedforward neural networks. In Proceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics, pp. 249–256, 2010. [find](https://scholar.google.com/scholar?q=Xavier+Glorot+and+Yoshua+Bengio%2E+Understanding+the+difficulty+of+training+deep+feedforward+neural+networks%2E+In+Proceedings+of+the+Thirteenth+International+Conference+on+Artificial+Intelligence+and+Sta)
16. Marco Gori, Gabriele Monfardini, and Franco Scarselli. A new model for learning in graph domains. In IEEE International Joint Conference on Neural Networks, pp. 729734, 2005. [find](https://scholar.google.com/scholar?q=Marco+Gori%2C+Gabriele+Monfardini%2C+and+Franco+Scarselli%2E+A+new+model+for+learning+in+graph+domains%2E+In+IEEE+International+Joint+Conference+on+Neural+Networks%2C+pp%2E+729734%2C+2005%2E)
17. William L Hamilton, Rex Ying, and Jure Leskovec. Inductive representation learning on large graphs. Neural Information Processing Systems (NIPS), 2017. [our card](https://md.research.elamur.ai/free/en/papers/hamilton-2017-inductive-representation-learning-large-graphs/)
18. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 770–778, 2016. [find](https://scholar.google.com/scholar?q=Kaiming+He%2C+Xiangyu+Zhang%2C+Shaoqing+Ren%2C+and+Jian+Sun%2E+Deep+residual+learning+for+image+recognition%2E+In+Proceedings+of+the+IEEE+conference+on+computer+vision+and+pattern+recognition%2C+pp%2E+770%E2%80%93778%2C+2016)
19. Mikael Henaff, Joan Bruna, and Yann LeCun. Deep convolutional networks on graph-structured data. arXiv preprint arXiv:1506.05163, 2015. — [arXiv:1506.05163](https://arxiv.org/abs/1506.05163) [find](https://scholar.google.com/scholar?q=Mikael+Henaff%2C+Joan+Bruna%2C+and+Yann+LeCun%2E+Deep+convolutional+networks+on+graph%2Dstructured+data%2E+arXiv+preprint+arXiv%3A1506%2E05163%2C+2015%2E+%E2%80%94+%5BarXiv%3A1506%2E05163%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1506%2E05163%29)
20. Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8): 1735–1780, 1997. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+Long+short%2Dterm+memory%2E+Neural+computation%2C+9%288%29%3A+1735%E2%80%931780%2C+1997%2E)
21. Yedid Hoshen. Vain: Attentional multi-agent predictive modeling. In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett (eds.), Advances in Neural Information Processing Systems 30, pp. 2698– 2708. Curran Associates, Inc., 2017. URL [http://papers.nips.cc/paper/…](http://papers.nips.cc/paper/…) [↗](http://papers.nips.cc/paper/…)
22. Simon Jégou, Michal Drozdzal, David Vázquez, Adriana Romero, and Yoshua Bengio. The one hundred layers tiramisu: Fully convolutional densenets for semantic segmentation. In Workshop on Computer Vision in Vehicle Technology CVPRW, 2017. [find](https://scholar.google.com/scholar?q=Simon+Je%CC%81gou%2C+Michal+Drozdzal%2C+David+Va%CC%81zquez%2C+Adriana+Romero%2C+and+Yoshua+Bengio%2E+The+one+hundred+layers+tiramisu%3A+Fully+convolutional+densenets+for+semantic+segmentation%2E+In+Workshop+on+Computer+Visi)
23. Diederik Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. [find](https://scholar.google.com/scholar?q=Diederik+Kingma+and+Jimmy+Ba%2E+Adam%3A+A+method+for+stochastic+optimization%2E+arXiv+preprint+arXiv%3A1412%2E6980%2C+2014%2E)
24. Thomas N Kipf and Max Welling. Semi-supervised classification with graph convolutional networks. International Conference on Learning Representations (ICLR), 2017. [our card](https://md.research.elamur.ai/free/en/papers/kipf-2016-semi-supervised-classification-graph-convolutional/)
25. Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard Zemel. Gated graph sequence neural networks. International Conference on Learning Representations (ICLR), 2016. [find](https://scholar.google.com/scholar?q=Yujia+Li%2C+Daniel+Tarlow%2C+Marc+Brockschmidt%2C+and+Richard+Zemel%2E+Gated+graph+sequence+neural+networks%2E+International+Conference+on+Learning+Representations+%28ICLR%29%2C+2016%2E)
26. Zhouhan Lin, Minwei Feng, Cicero Nogueira dos Santos, Mo Yu, Bing Xiang, Bowen Zhou, and Yoshua Bengio. A structured self-attentive sentence embedding. arXiv preprint arXiv:1703.03130, 2017. [our card](https://md.research.elamur.ai/free/en/papers/lin-2017-structured-self-attentive-sentence-embedding/)
27. Qing Lu and Lise Getoor. Link-based classification. In Proceedings of the 20th International Conference on Machine Learning (ICML-03), pp. 496–503, 2003. [find](https://scholar.google.com/scholar?q=Qing+Lu+and+Lise+Getoor%2E+Link%2Dbased+classification%2E+In+Proceedings+of+the+20th+International+Conference+on+Machine+Learning+%28ICML%2D03%29%2C+pp%2E+496%E2%80%93503%2C+2003%2E)
28. Laurens van der Maaten and Geoffrey Hinton. Visualizing data using t-sne. Journal of Machine Learning Research, 9(Nov):2579–2605, 2008. [find](https://scholar.google.com/scholar?q=Laurens+van+der+Maaten+and+Geoffrey+Hinton%2E+Visualizing+data+using+t%2Dsne%2E+Journal+of+Machine+Learning+Research%2C+9%28Nov%29%3A2579%E2%80%932605%2C+2008%2E)
29. Federico Monti, Davide Boscaini, Jonathan Masci, Emanuele Rodolà, Jan Svoboda, and Michael M Bronstein. Geometric deep learning on graphs and manifolds using mixture model cnns. arXiv preprint arXiv:1611.08402, 2016. — [arXiv:1611.08402](https://arxiv.org/abs/1611.08402) [find](https://scholar.google.com/scholar?q=Federico+Monti%2C+Davide+Boscaini%2C+Jonathan+Masci%2C+Emanuele+Rodola%CC%80%2C+Jan+Svoboda%2C+and+Michael+M+Bronstein%2E+Geometric+deep+learning+on+graphs+and+manifolds+using+mixture+model+cnns%2E+arXiv+preprint+arXiv%3A)
30. Mathias Niepert, Mohamed Ahmed, and Konstantin Kutzkov. Learning convolutional neural networks for graphs. In Proceedings of The 33rd International Conference on Machine Learning, volume 48, pp. 2014–2023, 2016. [find](https://scholar.google.com/scholar?q=Mathias+Niepert%2C+Mohamed+Ahmed%2C+and+Konstantin+Kutzkov%2E+Learning+convolutional+neural+networks+for+graphs%2E+In+Proceedings+of+The+33rd+International+Conference+on+Machine+Learning%2C+volume+48%2C+pp%2E+2014%E2%80%93)
31. Bryan Perozzi, Rami Al-Rfou, and Steven Skiena. Deepwalk: Online learning of social representations. In Proceedings of the 20th ACM SIGKDD international conference on Knowledge discovery and data mining, pp. 701–710. ACM, 2014. [find](https://scholar.google.com/scholar?q=Bryan+Perozzi%2C+Rami+Al%2DRfou%2C+and+Steven+Skiena%2E+Deepwalk%3A+Online+learning+of+social+representations%2E+In+Proceedings+of+the+20th+ACM+SIGKDD+international+conference+on+Knowledge+discovery+and+data+mini)
32. Sam T. Roweis and Lawrence K. Saul. Nonlinear dimensionality reduction by locally linear embedding. Science, 290:2323–2326, 2000. [find](https://scholar.google.com/scholar?q=Sam+T%2E+Roweis+and+Lawrence+K%2E+Saul%2E+Nonlinear+dimensionality+reduction+by+locally+linear+embedding%2E+Science%2C+290%3A2323%E2%80%932326%2C+2000%2E)
33. Adam Santoro, David Raposo, David GT Barrett, Mateusz Malinowski, Razvan Pascanu, Peter Battaglia, and Timothy Lillicrap. A simple neural network module for relational reasoning. arXiv preprint arXiv:1706.01427, 2017. [our card](https://md.research.elamur.ai/free/en/papers/santoro-2017-simple-neural-network-module-relational/)
34. Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61–80, 2009. [find](https://scholar.google.com/scholar?q=Franco+Scarselli%2C+Marco+Gori%2C+Ah+Chung+Tsoi%2C+Markus+Hagenbuchner%2C+and+Gabriele+Monfardini%2E+The+graph+neural+network+model%2E+IEEE+Transactions+on+Neural+Networks%2C+20%281%29%3A61%E2%80%9380%2C+2009%2E)
35. Prithviraj Sen, Galileo Namata, Mustafa Bilgic, Lise Getoor, Brian Galligher, and Tina Eliassi-Rad. Collective classification in network data. AI magazine, 29(3):93, 2008. [find](https://scholar.google.com/scholar?q=Prithviraj+Sen%2C+Galileo+Namata%2C+Mustafa+Bilgic%2C+Lise+Getoor%2C+Brian+Galligher%2C+and+Tina+Eliassi%2DRad%2E+Collective+classification+in+network+data%2E+AI+magazine%2C+29%283%29%3A93%2C+2008%2E)
36. A. Sperduti and A. Starita. Supervised neural networks for the classification of structures. Trans. Neur. Netw., 8(3):714–735, May 1997. ISSN 1045-9227. doi: 10.1109/72.572108. — [doi:10.1109/72.572108](https://doi.org/10.1109/72.572108) [find](https://scholar.google.com/scholar?q=A%2E+Sperduti+and+A%2E+Starita%2E+Supervised+neural+networks+for+the+classification+of+structures%2E+Trans%2E+Neur%2E+Netw%2E%2C+8%283%29%3A714%E2%80%93735%2C+May+1997%2E+ISSN+1045%2D9227%2E+doi%3A+10%2E1109%2F72%2E572108%2E+%E2%80%94+%5Bdoi%3A10%2E1109%2F72%2E57210)
37. Nitish Srivastava, Geoffrey E Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. Dropout: a simple way to prevent neural networks from overfitting. Journal of machine learning research, 15(1):1929–1958, 2014. [find](https://scholar.google.com/scholar?q=Nitish+Srivastava%2C+Geoffrey+E+Hinton%2C+Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Ruslan+Salakhutdinov%2E+Dropout%3A+a+simple+way+to+prevent+neural+networks+from+overfitting%2E+Journal+of+machine+learning+research)
38. Aravind Subramanian, Pablo Tamayo, Vamsi K Mootha, Sayan Mukherjee, Benjamin L Ebert, Michael A Gillette, Amanda Paulovich, Scott L Pomeroy, Todd R Golub, Eric S Lander, et al. Gene set enrichment analysis: a knowledge-based approach for interpreting genome-wide expression profiles. Proceedings of t… [find](https://scholar.google.com/scholar?q=Aravind+Subramanian%2C+Pablo+Tamayo%2C+Vamsi+K+Mootha%2C+Sayan+Mukherjee%2C+Benjamin+L+Ebert%2C+Michael+A+Gillette%2C+Amanda+Paulovich%2C+Scott+L+Pomeroy%2C+Todd+R+Golub%2C+Eric+S+Lander%2C+et+al%2E+Gene+set+enrichment+ana)
39. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. Attention is all you need. arXiv preprint arXiv:1706.03762, 2017. [our card](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/)
40. Jason Weston, Frédéric Ratle, Hossein Mobahi, and Ronan Collobert. Deep learning via semi-supervised embedding. In Neural Networks: Tricks of the Trade, pp. 639–655. Springer, 2012. [find](https://scholar.google.com/scholar?q=Jason+Weston%2C+Fre%CC%81de%CC%81ric+Ratle%2C+Hossein+Mobahi%2C+and+Ronan+Collobert%2E+Deep+learning+via+semi%2Dsupervised+embedding%2E+In+Neural+Networks%3A+Tricks+of+the+Trade%2C+pp%2E+639%E2%80%93655%2E+Springer%2C+2012%2E)
41. Jason Weston, Sumit Chopra, and Antoine Bordes. Memory networks. CoRR, abs/1410.3916, 2014. URL http://arxiv.org/abs/1410.3916. [↗](http://arxiv.org/abs/1410.3916)
42. Zhilin Yang, William Cohen, and Ruslan Salakhudinov. Revisiting semi-supervised learning with graph embeddings. In International Conference on Machine Learning, pp. 40–48, 2016. [find](https://scholar.google.com/scholar?q=Zhilin+Yang%2C+William+Cohen%2C+and+Ruslan+Salakhudinov%2E+Revisiting+semi%2Dsupervised+learning+with+graph+embeddings%2E+In+International+Conference+on+Machine+Learning%2C+pp%2E+40%E2%80%9348%2C+2016%2E)
43. Xiaojin Zhu, Zoubin Ghahramani, and John D Lafferty. Semi-supervised learning using gaussian fields and harmonic functions. In Proceedings of the 20th International conference on Machine learning (ICML-03), pp. 912–919, 2003. [find](https://scholar.google.com/scholar?q=Xiaojin+Zhu%2C+Zoubin+Ghahramani%2C+and+John+D+Lafferty%2E+Semi%2Dsupervised+learning+using+gaussian+fields+and+harmonic+functions%2E+In+Proceedings+of+the+20th+International+conference+on+Machine+learning+%28ICM)
44. Marinka Zitnik and Jure Leskovec. Predicting multicellular function through multi-layer tissue networks. Bioinformatics, 33(14):i190–i198, 2017. [find](https://scholar.google.com/scholar?q=Marinka+Zitnik+and+Jure+Leskovec%2E+Predicting+multicellular+function+through+multi%2Dlayer+tissue+networks%2E+Bioinformatics%2C+33%2814%29%3Ai190%E2%80%93i198%2C+2017%2E)