---
title: "Densely Connected Graph Convolutional Networks for"
description: "Graph convolutional networks (GCNs) are a natural way to encode… 4.3 BLEU on LDC2017T10. 42 references, cited by 53 papers here."
authors: ["Zhijiang Guo", "Yan Zhang", "Zhiyang Teng", "Wei Lu"]
year: 2019
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/Q19-1019/"
doi: "10.1162/tacl_a_00269"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/guo-2019-densely-connected-graph-convolutional-networks/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/guo-2019-densely-connected-graph-convolutional-networks/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/) / Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learning

# Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learning

**Authors:** [Zhijiang Guo](https://md.research.elamur.ai/free/en/authors/zhijiang-guo/), [Yan Zhang](https://md.research.elamur.ai/free/en/authors/yan-zhang/), [Zhiyang Teng](https://md.research.elamur.ai/free/en/authors/zhiyang-teng/) and [Wei Lu](https://md.research.elamur.ai/free/en/authors/wei-lu/) · **Year:** 2019 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([Transactions of the Association for Computational Linguistics (TACL) 2019](https://md.research.elamur.ai/free/en/venues/tacl/2019/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/Q19-1019/) · [PDF](https://aclanthology.org/Q19-1019.pdf) · [DOI 10.1162/tacl\_a\_00269](https://doi.org/10.1162/tacl_a_00269) · [OpenAlex](https://openalex.org/W2951309718)

**Citations:** 140 citations (OpenAlex, 2026-10-14) · [53](https://md.research.elamur.ai/free/en/papers/guo-2019-densely-connected-graph-convolutional-networks/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Graph convolutional networks (GCNs) are a natural way to encode structured inputs such as AMR graphs before generating text. In practice, however, they stop improving after about two layers, so they mostly capture local neighborhoods. Guo et al. address this with the dense connectivity idea from DenseNets: each GCN layer receives the concatenated outputs of all earlier layers plus the original node features. To limit parameters, each layer is kept narrow, with its width equal to the input dimension divided by the number of layers. The resulting DCGCN encoder stacks blocks made of two densely connected sub-blocks. It adds GAT-style attention over neighbors and merges block outputs through a linear combination layer with a residual link. A 2-layer LSTM with attention then decodes the text. Evaluated on two tasks, producing text from AMR graphs and translating with syntactic structure guiding the neural model, the system outperforms earlier neural approaches, and its biggest improvements come on the AMR task. The work shows that convolutional graph encoders can be made deep, combining local and long-range structure while remaining more parallelizable than recurrent graph models.

## Key points
- On AMR-to-text generation, DCGCN beats the previous best neural models by 2 BLEU on LDC2015E86 and 4.3 BLEU on LDC2017T10.
- Earlier GCN work peaked at 2 layers, and layer-aggregation approaches (Xu et al., 2018) went to at most 6 layers. Dense connections let DCGCN train much deeper without degradation.
- Layer width follows d_hidden = d/L. For example, 3 layers with d = 300 gives 100 units per layer, and the concatenated output is again 300.
- The encoder combines DCGCN blocks (sub-blocks of n and m layers), graph attention in the style of Velickovic et al. (2018), and an ELMo-inspired linear combination layer. The decoder is a 2-layer LSTM.
- The implementation is built on MXNET and Sockeye. Code is available at github.com/Cartus/DCGCN.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Singapore University of Technology and Design](https://md.research.elamur.ai/free/en/universities/singapore-university-of-technology-and-design/) — University, Singapore, Singapore · 33 works · 10,760 citations
- [Westlake University](https://md.research.elamur.ai/free/en/universities/westlake-university/) — University, Hangzhou, China · 30 works · 7,732 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🧾 Semantic parsing & meaning representation](https://md.research.elamur.ai/free/en/topics/semantic-parsing/) · 135 works

## Cite

Cite as: Zhijiang Guo, Yan Zhang, Zhiyang Teng and Wei Lu. 2019. Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learning. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{guo2019densely,
  title = {{Densely Connected Graph Convolutional Networks for Graph-to-Sequence Learning}},
  author = {Zhijiang Guo, Yan Zhang, Zhiyang Teng and Wei Lu},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.1162/tacl\_a\_00269},
  url = {https://doi.org/10.1162/tacl_a_00269}
}
```

- [← previous The emergence of number and syntax units in LSTM language models](https://md.research.elamur.ai/free/en/papers/lakretz-2019-emergence-number-syntax-units-lstm/)
- [next → Unsupervised Question Answering by Cloze Translation](https://md.research.elamur.ai/free/en/papers/lewis-2019-unsupervised-question-answering-cloze-translation/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/)

## References

42 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Hieber Felix, Domhan Tobias, Denkowski Michael, Vilar David, Sokolov Artem, Clifton Ann, and Post Matt. 2017. Sockeye: A toolkit for neural machine translation. arXiv preprint arxiv:1712.05690. [find](https://scholar.google.com/scholar?q=Hieber+Felix%2C+Domhan+Tobias%2C+Denkowski+Michael%2C+Vilar+David%2C+Sokolov+Artem%2C+Clifton+Ann%2C+and+Post+Matt%2E+2017%2E+Sockeye%3A+A+toolkit+for+neural+machine+translation%2E+arXiv+preprint+arxiv%3A1712%2E05690%2E)
2. Chris Alberti, Daniel Andor, Ivan Bogatyy, Michael Collins, Daniel Gillick, Lingpeng Kong, Jeffrey Flanigan, Chris Dyer, Noah A. Smith, and Jaime G. Carbonell. 2016. [Generation from abstract meaning representation using tree transducers](https://md.research.elamur.ai/free/en/papers/flanigan-2016-generation-abstract-meaning-representation-tree/). In Proceedings of NAACL-HLT.
3. Philipp Koehn, Hieu Hoang, Alexandra Birch, Chris Callison-Burch, Marcello Federico, Nicola Bertoldi, Brooke Cowan, Wade Shen, Christine Moran, Richard Zens, Chris Dyer, Ondrej Bojar, Alexandra Constantin, and Evan Herbst. 2007. Moses: Open source toolkit for statistical machine translation. In Proc… [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Hieu+Hoang%2C+Alexandra+Birch%2C+Chris+Callison%2DBurch%2C+Marcello+Federico%2C+Nicola+Bertoldi%2C+Brooke+Cowan%2C+Wade+Shen%2C+Christine+Moran%2C+Richard+Zens%2C+Chris+Dyer%2C+Ondrej+Bojar%2C+Alexandra+Consta)
4. Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, and Yann Dauphin. 2017. [Convolutional sequence to sequence learning](https://md.research.elamur.ai/free/en/papers/gehring-2017-convolutional-sequence-sequence-learning/). In Proceedings of ICML.
5. Ross Girshick, Jeff Donahue, Trevor Darrell, and Jitendra Malik. 2014. [Rich feature hierarchies for accurate object detection and semantic segmentation](https://md.research.elamur.ai/free/en/papers/girshick-2013-rich-feature-hierarchies-accurate-object/). In Proceedings of CVPR.
6. Philipp Koehn, Franz Josef Och, and Daniel Marcu. 2003. Statistical phrase-based translation. In Proceedings of NAACL-HLT. [find](https://scholar.google.com/scholar?q=Philipp+Koehn%2C+Franz+Josef+Och%2C+and+Daniel+Marcu%2E+2003%2E+Statistical+phrase%2Dbased+translation%2E+In+Proceedings+of+NAACL%2DHLT%2E)
7. Michele Gori, Gabriele Monfardini, and Franco Scarselli. 2005. A new model for learning in graph domains. In Proceedings of IJCNN. [find](https://scholar.google.com/scholar?q=Michele+Gori%2C+Gabriele+Monfardini%2C+and+Franco+Scarselli%2E+2005%2E+A+new+model+for+learning+in+graph+domains%2E+In+Proceedings+of+IJCNN%2E)
8. Ioannis Konstas, Srinivasan Iyer, Mark Yatskar, Yejin Choi, and Luke S. Zettlemoyer. 2017. [Neural amr: Sequence-to-sequence models for parsing and generation](https://md.research.elamur.ai/free/en/papers/konstas-2017-neural-amr-sequence-sequence-models/). In Proceedings of ACL.
9. Jonathan L. Gross, Jay Yellen, and Ping Zhang. 2013. Handbook of Graph Theory, Second Edition. Chapman & Hall/CRC. [find](https://scholar.google.com/scholar?q=Jonathan+L%2E+Gross%2C+Jay+Yellen%2C+and+Ping+Zhang%2E+2013%2E+Handbook+of+Graph+Theory%2C+Second+Edition%2E+Chapman+%26+Hall%2FCRC%2E)
10. Ioannis Konstas and Mirella Lapata. 2012. Unsupervised concept-to-text generation with hypergraphs. In Proceedings of NAACL-HLT. [find](https://scholar.google.com/scholar?q=Ioannis+Konstas+and+Mirella+Lapata%2E+2012%2E+Unsupervised+concept%2Dto%2Dtext+generation+with+hypergraphs%2E+In+Proceedings+of+NAACL%2DHLT%2E)
11. Zhijiang Guo and Wei Lu. 2018. Better transition-based AMR parsing with a refined search space. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Zhijiang+Guo+and+Wei+Lu%2E+2018%2E+Better+transition%2Dbased+AMR+parsing+with+a+refined+search+space%2E+In+Proceedings+of+EMNLP%2E)
12. Ioannis Konstas and Mirella Lapata. 2013. Inducing document plans for concept-to-text generation. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Ioannis+Konstas+and+Mirella+Lapata%2E+2013%2E+Inducing+document+plans+for+concept%2Dto%2Dtext+generation%2E+In+Proceedings+of+EMNLP%2E)
13. William L. Hamilton, Rex Ying, and Jure Leskovec. 2017. [Inductive representation learning on large graphs](https://md.research.elamur.ai/free/en/papers/hamilton-2017-inductive-representation-learning-large-graphs/). In Proceedings of NIPS.
14. Qimai Li, Zhichao Han, and Xiao-Ming Wu. 2018. Deeper insights into graph convolutional networks for semi-supervised learning. In Proceedings of AAAI. [find](https://scholar.google.com/scholar?q=Qimai+Li%2C+Zhichao+Han%2C+and+Xiao%2DMing+Wu%2E+2018%2E+Deeper+insights+into+graph+convolutional+networks+for+semi%2Dsupervised+learning%2E+In+Proceedings+of+AAAI%2E)
15. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of CVPR. [find](https://scholar.google.com/scholar?q=Kaiming+He%2C+Xiangyu+Zhang%2C+Shaoqing+Ren%2C+and+Jian+Sun%2E+2016%2E+Deep+residual+learning+for+image+recognition%2E+In+Proceedings+of+CVPR%2E)
16. Yujia Li, Daniel Tarlow, Marc Brockschmidt, and Richard S. Zemel. 2016. Gated graph sequence neural networks. In Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=Yujia+Li%2C+Daniel+Tarlow%2C+Marc+Brockschmidt%2C+and+Richard+S%2E+Zemel%2E+2016%2E+Gated+graph+sequence+neural+networks%2E+In+Proceedings+of+ICLR%2E)
17. Mikael Henaff, Joan Bruna, and Yann LeCun. 2015. Deep convolutional networks on graph-structured data. arXiv preprint arxiv:1506.05163. — [arXiv:1506.05163](https://arxiv.org/abs/1506.05163) [find](https://scholar.google.com/scholar?q=Mikael+Henaff%2C+Joan+Bruna%2C+and+Yann+LeCun%2E+2015%2E+Deep+convolutional+networks+on+graph%2Dstructured+data%2E+arXiv+preprint+arxiv%3A1506%2E05163%2E+%E2%80%94+%5BarXiv%3A1506%2E05163%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1506%2E05163%29)
18. Wei Lu and Hwee Tou Ng. 2011. A probabilistic forest-to-string model for language generation from typed lambda calculus expressions. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Wei+Lu+and+Hwee+Tou+Ng%2E+2011%2E+A+probabilistic+forest%2Dto%2Dstring+model+for+language+generation+from+typed+lambda+calculus+expressions%2E+In+Proceedings+of+EMNLP%2E)
19. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735–1780. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+Computation%2C+9%288%29%3A1735%E2%80%931780%2E)
20. Wei Lu, Hwee Tou Ng, and Wee Sun Lee. 2009. Natural language generation with tree conditional random fields. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Wei+Lu%2C+Hwee+Tou+Ng%2C+and+Wee+Sun+Lee%2E+2009%2E+Natural+language+generation+with+tree+conditional+random+fields%2E+In+Proceedings+of+EMNLP%2E)
21. Gao Huang, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. 2017. Densely connected convolutional networks. In Proceedings of CVPR. [find](https://scholar.google.com/scholar?q=Gao+Huang%2C+Zhuang+Liu%2C+Laurens+van+der+Maaten%2C+and+Kilian+Q%2E+Weinberger%2E+2017%2E+Densely+connected+convolutional+networks%2E+In+Proceedings+of+CVPR%2E)
22. Diederik P. Kingma and Jimmy Ba. 2015. Adam: A method for stochastic optimization. In Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=Diederik+P%2E+Kingma+and+Jimmy+Ba%2E+2015%2E+Adam%3A+A+method+for+stochastic+optimization%2E+In+Proceedings+of+ICLR%2E)
23. Diego Marcheggiani and Ivan Titov. 2017. [Encoding sentences with graph convolutional networks for semantic role labeling](https://md.research.elamur.ai/free/en/papers/marcheggiani-2017-encoding-sentences-graph-convolutional-networks/). In Proceedings of EMNLP.
24. Thomas N. Kipf and Max Welling. 2017. [Semi-supervised classification with graph convolutional networks](https://md.research.elamur.ai/free/en/papers/kipf-2016-semi-supervised-classification-graph-convolutional/). In Proceedings of ICLR.
25. Vinod Nair and Geoffrey E. Hinton. 2010. Rectified linear units improve restricted boltzmann machines. In Proceedings of ICML. [find](https://scholar.google.com/scholar?q=Vinod+Nair+and+Geoffrey+E%2E+Hinton%2E+2010%2E+Rectified+linear+units+improve+restricted+boltzmann+machines%2E+In+Proceedings+of+ICML%2E)
26. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: a method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of ACL.
27. Linfeng Song, Yue Zhang, Zhiguo Wang, and Daniel Gildea. 2018. [A graph-to-sequence model for AMR-to-text generation](https://md.research.elamur.ai/free/en/papers/song-2018-graph-sequence-model-amr-text/). In Proceedings of ACL.
28. Matthew E. Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke S. Zettlemoyer. 2018. [Deep contextualized word representations](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/). In Proceedings of NAACL-HLT.
29. Rupesh Kumar Srivastava, Klaus Greff, and Jürgen Schmidhuber. 2015. Training very deep networks. In Proceedings of NIPS. [find](https://scholar.google.com/scholar?q=Rupesh+Kumar+Srivastava%2C+Klaus+Greff%2C+and+Ju%CC%88rgen+Schmidhuber%2E+2015%2E+Training+very+deep+networks%2E+In+Proceedings+of+NIPS%2E)
30. Ilya Sutskever, Oriol Vinyals, and Quoc V. Le. 2014. Sequence to sequence learning with neural networks. In Proceedings of NIPS. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V%2E+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Proceedings+of+NIPS%2E)
31. Maja Popovic. 2017. [chrf++: Words helping character n -grams](https://md.research.elamur.ai/free/en/papers/popovic-2017-chrf-words-helping-character-n/). In Proceedings of WMT@ACL.
32. Zhaopeng Tu, Zhengdong Lu, Yang Liu, Xiaohua Liu, and Hang Li. 2016. [Modeling coverage for neural machine translation](https://md.research.elamur.ai/free/en/papers/tu-2016-modeling-coverage-neural-machine-translation/). In Proceedings of ACL.
33. Nima Pourdamghani, Kevin Knight, and Ulf Hermjakob. 2016. [Generating english from abstract meaning representations](https://md.research.elamur.ai/free/en/papers/pourdamghani-2016-generating-english-abstract-meaning-representations/). In Proceedings of INLG.
34. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Proceedings of NIPS.
35. Franco Scarselli, Marco Gori, Ah Chung Tsoi, Markus Hagenbuchner, and Gabriele Monfardini. 2009. The graph neural network model. IEEE Transactions on Neural Networks, 20(1):61–80. [find](https://scholar.google.com/scholar?q=Franco+Scarselli%2C+Marco+Gori%2C+Ah+Chung+Tsoi%2C+Markus+Hagenbuchner%2C+and+Gabriele+Monfardini%2E+2009%2E+The+graph+neural+network+model%2E+IEEE+Transactions+on+Neural+Networks%2C+20%281%29%3A61%E2%80%9380%2E)
36. Petar Velickovic, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Liò, and Yoshua Bengio. 2018. Graph attention networks. In Proceedings of ICLR. [find](https://scholar.google.com/scholar?q=Petar+Velickovic%2C+Guillem+Cucurull%2C+Arantxa+Casanova%2C+Adriana+Romero%2C+Pietro+Lio%CC%80%2C+and+Yoshua+Bengio%2E+2018%2E+Graph+attention+networks%2E+In+Proceedings+of+ICLR%2E)
37. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In Proceedings of ACL.
38. Keyulu Xu, Chengtao Li, Yonglong Tian, Tomohiro Sonobe, Ken ichi Kawarabayashi, and Stefanie Jegelka. 2018. Representation learning on graphs with jumping knowledge networks. In Proceedings of ICML. [find](https://scholar.google.com/scholar?q=Keyulu+Xu%2C+Chengtao+Li%2C+Yonglong+Tian%2C+Tomohiro+Sonobe%2C+Ken+ichi+Kawarabayashi%2C+and+Stefanie+Jegelka%2E+2018%2E+Representation+learning+on+graphs+with+jumping+knowledge+networks%2E+In+Proceedings+of+ICML%2E)
39. Linfeng Song, Xiaochang Peng, Yue Zhang, Zhiguo Wang, and Daniel Gildea. 2017. [AMR-to-text generation with synchronous node replacement grammar](https://md.research.elamur.ai/free/en/papers/song-2017-amr-text-generation-synchronous-node/). In Proceedings of ACL.
40. Yue Zhang, Qi Liu, and Linfeng Song. 2018a. Sentence-state LSTM for text representation. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Yue+Zhang%2C+Qi+Liu%2C+and+Linfeng+Song%2E+2018a%2E+Sentence%2Dstate+LSTM+for+text+representation%2E+In+Proceedings+of+ACL%2E)
41. Linfeng Song, Yue Zhang, Xiaochang Peng, Zhiguo Wang, and Daniel Gildea. 2016. AMR-to-text generation as a traveling salesman problem. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Linfeng+Song%2C+Yue+Zhang%2C+Xiaochang+Peng%2C+Zhiguo+Wang%2C+and+Daniel+Gildea%2E+2016%2E+AMR%2Dto%2Dtext+generation+as+a+traveling+salesman+problem%2E+In+Proceedings+of+EMNLP%2E)
42. Yuhao Zhang, Peng Qi, and Christopher D. Manning. 2018b. [Graph convolution over pruned dependency trees improves relation extraction](https://md.research.elamur.ai/free/en/papers/zhang-2018-graph-convolution-over-pruned-dependency/). In Proceedings of EMNLP.

*Updated 24 October 2026*