---
title: "Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing — Bogin 2019"
description: "The SPIDER benchmark changes this: test questions target complex databases never seen during training. 6.5 points. 20 references, cited by 62 papers here."
authors: ["Ben Bogin", "Jonathan Berant", "Matt Gardner"]
year: 2019
venue: "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/P19-1448/"
doi: "10.18653/v1/p19-1448"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/bogin-2019-representing-schema-structure-graph-neural/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bogin-2019-representing-schema-structure-graph-neural/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/3/) / Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing

# Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing

**Authors:** [Ben Bogin](https://md.research.elamur.ai/free/en/authors/ben-bogin/), [Jonathan Berant](https://md.research.elamur.ai/free/en/authors/jonathan-berant/) and [Matt Gardner](https://md.research.elamur.ai/free/en/authors/matt-gardner/) · **Year:** 2019 · **Venue:** Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P19-1448/) · [PDF](https://aclanthology.org/P19-1448.pdf) · [DOI 10.18653/v1/p19-1448](https://doi.org/10.18653/v1/p19-1448) · [OpenAlex](https://openalex.org/W2952032096)

**Citations:** 210 citations (OpenAlex, 2026-10-14) · [62](https://md.research.elamur.ai/free/en/papers/bogin-2019-representing-schema-structure-graph-neural/cited-by/) in our library, of the 140,884 papers whose bibliographies we read

## Summary
Text-to-SQL parsers have usually been trained and tested on one fixed database or on single-table setups, so they rarely needed to understand how a schema is organized. The SPIDER benchmark changes this: test questions target complex databases never seen during training. Bogin, Berant and Gardner argue that, in this zero-shot setting, the links between tables and columns (especially foreign/primary keys) are what tell a parser whether a join is needed. They convert each schema into a graph whose nodes are tables and columns and whose typed edges mark table–column membership and key relations. A gated GNN then computes node representations, weighted by how relevant each item appears to be for the current question. These structure-aware vectors are fed into the grammar-based encoder-decoder of Krishnamurthy et al. (2017). On the decoder side, a self-attention component favors schema items related to those already generated. On the SPIDER test set, accuracy rises to 39.4%, roughly double the previous state of the art. The largest gains come on questions that involve several tables.

## Key points
- **Data and setup:** experiments use SPIDER (7,000 / 1,034 / 2,147 train/dev/test examples). The base parser uses an AllenNLP SQL grammar covering 98.3% of SPIDER queries, and the code is released at github.com/benbogin/spider-schema-gnn.
- **Main results:** test accuracy is 39.4%, compared with 33.8% for the same parser without the GNN and 19.7% for the prior best system (SyntaxSQLNet). On dev, multi-table accuracy climbs from 14.6% to 26.8%.
- **Ablations (dev set):**
  - Dropping the decoder self-attention costs about 2 points overall and 6.5 points on multi-table questions.
  - Using the GNN only through self-attention gives 35.9%.
  - Removing the relevance score gives 37.0%.
- **Relevance headroom:** an oracle relevance score raises dev accuracy to 54.3%, which suggests better schema-item relevance estimation is a promising direction.
- **Join quality:** malformed joins (a table joined with itself, or tables with no key connection) fall from 83.4% of join-containing predictions without the GNN to 15.6% with it.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [💻 Code & programming languages](https://md.research.elamur.ai/free/en/topics/code-and-programming/) · [🧾 Semantic parsing & meaning representation](https://md.research.elamur.ai/free/en/topics/semantic-parsing/)

## Institutions

- [Allen Institute for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/allen-institute-for-artificial-intelligence/) — Research institute, Seattle, United States · 169 works · 65,355 citations
- [Tel Aviv University](https://md.research.elamur.ai/free/en/universities/tel-aviv-university/) — University, Tel Aviv, Israel · 54 works · 7,966 citations

## Cite

Cite as: Ben Bogin, Jonathan Berant and Matt Gardner. 2019. Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{bogin2019representing,
  title = {{Representing Schema Structure with Graph Neural Networks for Text-to-SQL Parsing}},
  author = {Ben Bogin, Jonathan Berant and Matt Gardner},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1448},
  url = {https://doi.org/10.18653/v1/p19-1448}
}
```

- [← previous Automatic Language Identification in Texts: A Survey](https://md.research.elamur.ai/free/en/papers/jauhiainen-2019-automatic-language-identification-texts/)
- [next → Learning to Retrieve Reasoning Paths over Wikipedia Graph for Question Answering](https://md.research.elamur.ai/free/en/papers/asai-2019-learning-retrieve-reasoning-paths-over/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/3/)

## References

20 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. D. Bahdanau, K. Cho, and Y. Bengio. 2015. Neural machine translation by jointly learning to align and translate. In International Conference on Learning Representations (ICLR). [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
2. J. Cheng, S. Reddy, V. Saraswat, and M. Lapata. 2017. Learning structured natural language representations for semantic parsing. In Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=J%2E+Cheng%2C+S%2E+Reddy%2C+V%2E+Saraswat%2C+and+M%2E+Lapata%2E+2017%2E+Learning+structured+natural+language+representations+for+semantic+parsing%2E+In+Association+for+Computational+Linguistics+%28ACL%29%2E)
3. K. Cho, B. van Merrienboer, C. Gulcehre, D. Bahdanau, F. Bougares, H. Schwenk, and Y. Bengio. 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. In Empirical Methods in Natural Language Processing (EMNLP), pages 1724–1734. [find](https://scholar.google.com/scholar?q=K%2E+Cho%2C+B%2E+van+Merrienboer%2C+C%2E+Gulcehre%2C+D%2E+Bahdanau%2C+F%2E+Bougares%2C+H%2E+Schwenk%2C+and+Y%2E+Bengio%2E+2014%2E+Learning+phrase+representations+using+RNN+encoder%2Ddecoder+for+statistical+machine+translation%2E+In+Em)
4. N. De Cao, W. Aziz, and I. Titov. 2019. Question answering by reasoning across documents with graph convolutional networks. In North American Association for Computational Linguistics (NAACL). [find](https://scholar.google.com/scholar?q=N%2E+De+Cao%2C+W%2E+Aziz%2C+and+I%2E+Titov%2E+2019%2E+Question+answering+by+reasoning+across+documents+with+graph+convolutional+networks%2E+In+North+American+Association+for+Computational+Linguistics+%28NAACL%29%2E)
5. C. Finegan-Dollak, J. K. Kummerfeld, L. Zhang, K. Ramanathan, S. Sadasivam, R. Zhang, and D. Radev. 2018. Improving text-to-sql evaluation methodology. In Association for Computational Linguistics (ACL). [our card](https://md.research.elamur.ai/free/en/papers/dollak-2018-improving-text-sql-evaluation-methodology/)
6. M. Gardner, J. Grus, M. Neumann, O. Tafjord, P. Dasigi, N. Liu, M. Peters, M. Schmitz, and L. Zettlemoyer. 2018. AllenNLP: A deep semantic natural language processing platform. arXiv preprint arXiv:1803.07640. [our card](https://md.research.elamur.ai/free/en/papers/gardner-2018-allennlp-deep-semantic-natural-language/)
7. S. Hochreiter and J. Schmidhuber. 1997. Long short-term memory. Neural Computation, 9(8):1735– 1780. [find](https://scholar.google.com/scholar?q=S%2E+Hochreiter+and+J%2E+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+Computation%2C+9%288%29%3A1735%E2%80%93+1780%2E)
8. S. Iyer, I. Konstas, A. Cheung, J. Krishnamurthy, and L. Zettlemoyer. 2017. Learning a neural semantic parser from user feedback. In Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=S%2E+Iyer%2C+I%2E+Konstas%2C+A%2E+Cheung%2C+J%2E+Krishnamurthy%2C+and+L%2E+Zettlemoyer%2E+2017%2E+Learning+a+neural+semantic+parser+from+user+feedback%2E+In+Association+for+Computational+Linguistics+%28ACL%29%2E)
9. J. Krishnamurthy, P. Dasigi, and M. Gardner. 2017. Neural semantic parsing with type constraints for semi-structured tables. In Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=J%2E+Krishnamurthy%2C+P%2E+Dasigi%2C+and+M%2E+Gardner%2E+2017%2E+Neural+semantic+parsing+with+type+constraints+for+semi%2Dstructured+tables%2E+In+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2E)
10. Y. Li, D. Tarlow, M. Brockschmidt, and R. Zemel. 2016. Gated graph sequence neural networks. In International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Y%2E+Li%2C+D%2E+Tarlow%2C+M%2E+Brockschmidt%2C+and+R%2E+Zemel%2E+2016%2E+Gated+graph+sequence+neural+networks%2E+In+International+Conference+on+Learning+Representations+%28ICLR%29%2E)
11. K. Lin, B. Bogin, M. Neumann, J. Berant, and M. Gardner. 2019. Grammar-based neural text-to-sql generation. arXiv preprint arXiv:1905.13326. — [arXiv:1905.13326](https://arxiv.org/abs/1905.13326) [find](https://scholar.google.com/scholar?q=K%2E+Lin%2C+B%2E+Bogin%2C+M%2E+Neumann%2C+J%2E+Berant%2C+and+M%2E+Gardner%2E+2019%2E+Grammar%2Dbased+neural+text%2Dto%2Dsql+generation%2E+arXiv+preprint+arXiv%3A1905%2E13326%2E+%E2%80%94+%5BarXiv%3A1905%2E13326%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1905%2E13326%29)
12. M. Rabinovich, M. Stern, and D. Klein. 2017. Abstract syntax networks for code generation and semantic parsing. In Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=M%2E+Rabinovich%2C+M%2E+Stern%2C+and+D%2E+Klein%2E+2017%2E+Abstract+syntax+networks+for+code+generation+and+semantic+parsing%2E+In+Association+for+Computational+Linguistics+%28ACL%29%2E)
13. D. Sorokin and I. Gurevych. 2018. Modeling semantics with gated graph neural networks for knowledge base question answering. In International Conference on Computational Linguistics (COLING). [find](https://scholar.google.com/scholar?q=D%2E+Sorokin+and+I%2E+Gurevych%2E+2018%2E+Modeling+semantics+with+gated+graph+neural+networks+for+knowledge+base+question+answering%2E+In+International+Conference+on+Computational+Linguistics+%28COLING%29%2E)
14. C. Xiao, M. Dymetman, and C. Gardent. 2016. Sequence-based structured prediction for semantic parsing. In Association for Computational Linguistics (ACL). [find](https://scholar.google.com/scholar?q=C%2E+Xiao%2C+M%2E+Dymetman%2C+and+C%2E+Gardent%2E+2016%2E+Sequence%2Dbased+structured+prediction+for+semantic+parsing%2E+In+Association+for+Computational+Linguistics+%28ACL%29%2E)
15. P. Yin and G. Neubig. 2017. A syntactic neural model for general-purpose code generation. In Association for Computational Linguistics (ACL), pages 440– 450. [our card](https://md.research.elamur.ai/free/en/papers/yin-2017-syntactic-neural-model-general-purpose/)
16. T. Yu, M. Yasunaga, K. Yang, R. Zhang, D. Wang, Z. Li, and D. Radev. 2018a. SyntaxSQLNet: Syntax tree networks for complex and cross-domaintext-to-SQL task. In Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=T%2E+Yu%2C+M%2E+Yasunaga%2C+K%2E+Yang%2C+R%2E+Zhang%2C+D%2E+Wang%2C+Z%2E+Li%2C+and+D%2E+Radev%2E+2018a%2E+SyntaxSQLNet%3A+Syntax+tree+networks+for+complex+and+cross%2Ddomaintext%2Dto%2DSQL+task%2E+In+Empirical+Methods+in+Natural+Language+Pr)
17. T. Yu, R. Zhang, K. Yang, M. Yasunaga, D. Wang, Z. Li, J. Ma, I. Li, Q. Yao, S. Roman, Z. Zhang, and D. Radev. 2018b. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task. In Empirical Methods in Natural Language Processing (EMNLP). [our card](https://md.research.elamur.ai/free/en/papers/yu-2018-spider-large-scale-human-labeled/)
18. M. Zelle and R. J. Mooney. 1996. Learning to parse database queries using inductive logic programming. In Association for the Advancement of Artificial Intelligence (AAAI), pages 1050–1055. [our card](https://md.research.elamur.ai/free/en/papers/zelle-1996-learning-parse-database-queries-using/)
19. L. S. Zettlemoyer and M. Collins. 2005. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. In Uncertainty in Artificial Intelligence (UAI), pages 658– 666. [our card](https://md.research.elamur.ai/free/en/papers/zettlemoyer-2012-learning-map-sentences-logical-form/)
20. V. Zhong, C. Xiong, and R. Socher. 2017. Seq2sql: Generating structured queries from natural language using reinforcement learning. arXiv preprint arXiv:1709.00103. [find](https://scholar.google.com/scholar?q=V%2E+Zhong%2C+C%2E+Xiong%2C+and+R%2E+Socher%2E+2017%2E+Seq2sql%3A+Generating+structured+queries+from+natural+language+using+reinforcement+learning%2E+arXiv+preprint+arXiv%3A1709%2E00103%2E)