---
title: "CoSQL: A Conversational Text-to-SQL Challenge — Yu 2019"
description: "CoSQL targets dialogue-based natural language interfaces to databases (NLIDB), where users explore a relational database across several turns rather than…"
authors: ["Tao Yu", "Rui Zhang", "Heyang Er", "Suyi Li", "Eric Xue", "Bo Pang", "Xi Victoria Lin", "Yi Chern Tan", "Tianze Shi", "Zihan Li", "Youxuan Jiang", "Michihiro Yasunaga", "Sungrok Shim", "Tao Chen", "Alexander Fabbri", "Zifan Li", "Luyao Chen", "Yuwen Zhang", "Shreya Dixit", "Vincent Zhang", "Caiming Xiong", "Richard Socher", "Walter Lasecki", "Dragomir Radev"]
year: 2019
venue: "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)"
source: "https://aclanthology.org/D19-1204/"
doi: "10.18653/v1/d19-1204"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yu-2019-cosql-conversational-text-sql-challenge/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yu-2019-cosql-conversational-text-sql-challenge/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/2/) / CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases

# CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases

**Authors:** [Tao Yu](https://md.research.elamur.ai/free/en/authors/tao-yu/), [Rui Zhang](https://md.research.elamur.ai/free/en/authors/rui-zhang/), [Heyang Er](https://md.research.elamur.ai/free/en/authors/heyang-er/), [Suyi Li](https://md.research.elamur.ai/free/en/authors/suyi-li/), Eric Xue, [Bo Pang](https://md.research.elamur.ai/free/en/authors/bo-pang/), [Xi Victoria Lin](https://md.research.elamur.ai/free/en/authors/xi-victoria-lin/), [Yi Chern Tan](https://md.research.elamur.ai/free/en/authors/yi-chern-tan/), [Tianze Shi](https://md.research.elamur.ai/free/en/authors/tianze-shi/), [Zihan Li](https://md.research.elamur.ai/free/en/authors/zihan-li/), Youxuan Jiang, [Michihiro Yasunaga](https://md.research.elamur.ai/free/en/authors/michihiro-yasunaga/) et al. (24 authors) · **Year:** 2019 · **Venue:** Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D19-1204/) · [PDF](https://aclanthology.org/D19-1204.pdf) · [DOI 10.18653/v1/d19-1204](https://doi.org/10.18653/v1/d19-1204) · [OpenAlex](https://openalex.org/W2970442801)

**Citations:** 158 citations (openalex-snapshot · 2026-10-14) · [103](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
CoSQL targets dialogue-based natural language interfaces to databases (NLIDB), where users explore a relational database across several turns rather than asking one well-formed question. Earlier context-dependent resources such as ATIS and SParC assume every question maps to SQL and omit system replies. Task-oriented dialogue corpora, by contrast, depend on domain-specific slot-value schemes. The authors built a Wizard-of-Oz corpus in which Amazon Mechanical Turk workers played users and trained computer science students played SQL experts. The experts wrote queries, asked for clarification, flagged unanswerable requests and explained results in plain language. Dialogue state is expressed as executable SQL, and each database appears in only one of the train, dev or test splits, so models must cope with unseen domains. The paper proposes a trio of challenges: tracking the dialogue state with SQL as its representation, producing system replies based on queries and the results they return, and predicting user dialogue acts. Baselines score poorly, with under 16% question-level accuracy for state tracking and weak logical faithfulness in generated responses. The benchmark is therefore far from solved and is a useful step toward robust cross-domain conversational database querying.

## Key points
- **Corpus scale:** 3,007 dialogues with 30k+ turns annotated with dialogue acts and 10k+ expert-written SQL queries. The dialogues cover 200 complex databases spanning 138 domains, and 25 SQL-proficient students served as experts.
- **State tracking:** CD-Seq2Seq and SyntaxSQL-con reach only 13.9% and 14.1% test question match, and 2.6% and 2.2% interaction match. Both perform clearly worse than on SParC.
- **Response generation:** Three baselines were compared (Template, Seq2Seq, Pointer-generator).
  - Pointer-generator has the best test BLEU (15.1).
  - The template approach wins human evaluation, with a logic correctness rate (LCR) of 41% and a grammar score of 4.0/5.
- **User dialogue acts:** Always predicting INFORM_SQL yields 62.8% test accuracy, implying roughly 40% of user turns are not directly convertible to SQL. TBCNN-pair reaches 83.9% accuracy, but F-scores for INFER_SQL and AMBIGUOUS stay near 10%.
- **Availability:** The data, baselines and leaderboard are released at yale-lily.github.io/cosql.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

Topics: [Speech act theory](https://md.research.elamur.ai/free/en/topics/speech-act-theory/)

## Cited by

Cited by

103 papers of our library cite it (of the 140884 whose bibliographies we read); 6 of them have a page here, 97 not yet.

- [Ar-Spider: Text-to-SQL in Arabic](https://md.research.elamur.ai/free/en/papers/almohaimeed-2024-ar-spider-text-sql-arabic/)
- [Prompt Engineering Techniques for Context-dependent Text-to-SQL in Arabic](https://md.research.elamur.ai/free/en/papers/almohaimeed-2025-prompt-engineering-techniques-context-dependent/)
- [Bridging Textual and Tabular Data for Cross-Domain Text-to-SQL Semantic Parsing](https://md.research.elamur.ai/free/en/papers/lin-2020-bridging-textual-tabular-data-cross/)
- [Investigating Pretrained Language Models for Graph-to-Text Generation](https://md.research.elamur.ai/free/en/papers/ribeiro-2021-investigating-pretrained-language-models-graph/)
- [Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/)
- [UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models](https://md.research.elamur.ai/free/en/papers/xie-2022-unifiedskg-unifying-multi-tasking-structured/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{yu2019cosql,
  title = {{CoSQL: A Conversational Text-to-SQL Challenge Towards Cross-Domain Natural Language Interfaces to Databases}},
  author = {Yu, Tao and Zhang, Rui and Er, Heyang and Li, Suyi and Xue, Eric and Pang, Bo and Lin, Xi Victoria and Tan, Yi Chern and Shi, Tianze and Li, Zihan and Jiang, Youxuan and Yasunaga, Michihiro and Shim, Sungrok and Chen, Tao and Fabbri, Alexander and Li, Zifan and Chen, Luyao and Zhang, Yuwen and Dixit, Shreya and Zhang, Vincent and Xiong, Caiming and Socher, Richard and Lasecki, Walter and Radev, Dragomir},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year = {2019},
  doi = {10.18653/v1/d19-1204},
  url = {https://doi.org/10.18653/v1/d19-1204}
}
```

- [← previous Cosmos QA: Machine Reading Comprehension with Contextual Commonsense Reasoning](https://md.research.elamur.ai/free/en/papers/huang-2019-cosmos-qa-machine-reading-comprehension/)
- [next → Counterfactual Data Augmentation for Mitigating Gender Stereotypes in Languages with Rich Morphology](https://md.research.elamur.ai/free/en/papers/zmigrod-2019-counterfactual-data-augmentation-mitigating-gender/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/2/)

## References

60 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yoav Artzi and Luke S. Zettlemoyer. 2011. Bootstrapping semantic parsers from conversations. In EMNLP. [find](https://scholar.google.com/scholar?q=Yoav+Artzi+and+Luke+S%2E+Zettlemoyer%2E+2011%2E+Bootstrapping+semantic+parsers+from+conversations%2E+In+EMNLP%2E)
2. Layla El Asri, Hannes Schulz, Shikhar Sharma, Jeremie Zumer, Justin Harris, Emery Fine, Rahul Mehrotra, and Kaheer Suleman. 2017. Frames: A corpus for adding memory to goal-oriented dialogue systems. CoRR, abs/1704.00057. [find](https://scholar.google.com/scholar?q=Layla+El+Asri%2C+Hannes+Schulz%2C+Shikhar+Sharma%2C+Jeremie+Zumer%2C+Justin+Harris%2C+Emery+Fine%2C+Rahul+Mehrotra%2C+and+Kaheer+Suleman%2E+2017%2E+Frames%3A+A+corpus+for+adding+memory+to+goal%2Doriented+dialogue+systems%2E+) [our card](https://md.research.elamur.ai/free/en/papers/asri-2017-frames-corpus-adding-memory-goal/)
3. Dzmitry Bahdanau, Kyunghyun Cho, and Yoshua Bengio. 2015. Neural machine translation by jointly learning to align and translate. In ICLR. [find](https://scholar.google.com/scholar?q=Dzmitry+Bahdanau%2C+Kyunghyun+Cho%2C+and+Yoshua+Bengio%2E+2015%2E+Neural+machine+translation+by+jointly+learning+to+align+and+translate%2E+In+ICLR%2E) [our card](https://md.research.elamur.ai/free/en/papers/bahdanau-2014-neural-machine-translation-jointly-learning/)
4. Ben Bogin, Jonathan Berant, and Matt Gardner. 2019. Representing schema structure with graph neural networks for text-to-sql parsing. In ACL. [find](https://scholar.google.com/scholar?q=Ben+Bogin%2C+Jonathan+Berant%2C+and+Matt+Gardner%2E+2019%2E+Representing+schema+structure+with+graph+neural+networks+for+text%2Dto%2Dsql+parsing%2E+In+ACL%2E)
5. Antoine Bordes and Jason Weston. 2017. Learning end-to-end goal-oriented dialog. ICLR. [find](https://scholar.google.com/scholar?q=Antoine+Bordes+and+Jason+Weston%2E+2017%2E+Learning+end%2Dto%2Dend+goal%2Doriented+dialog%2E+ICLR%2E)
6. Pawel Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gasic. 2018. Multiwoz - a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. In EMNLP. [find](https://scholar.google.com/scholar?q=Pawel+Budzianowski%2C+Tsung%2DHsien+Wen%2C+Bo%2DHsiang+Tseng%2C+In%CC%83igo+Casanueva%2C+Stefan+Ultes%2C+Osman+Ramadan%2C+and+Milica+Gasic%2E+2018%2E+Multiwoz+%2D+a+large%2Dscale+multi%2Ddomain+wizard%2Dof%2Doz+dataset+for+task%2Doriente)
7. Yen-Chun Chen and Mohit Bansal. 2018. Fast abstractive summarization with reinforce-selected sentence rewriting. In Proceedings of ACL. [find](https://scholar.google.com/scholar?q=Yen%2DChun+Chen+and+Mohit+Bansal%2E+2018%2E+Fast+abstractive+summarization+with+reinforce%2Dselected+sentence+rewriting%2E+In+Proceedings+of+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/chen-2018-fast-abstractive-summarization-reinforce-selected/)
8. Deborah A. Dahl, Madeleine Bates, Michael Brown, William Fisher, Kate Hunicke-Smith, David Pallett, Christine Pao, Alexander Rudnicky, and Elizabeth Shriberg. 1994. Expanding the scope of the atis task: The atis-3 corpus. In Proceedings of the Workshop on Human Language Technology, HLT ’94, Stroudsb… [find](https://scholar.google.com/scholar?q=Deborah+A%2E+Dahl%2C+Madeleine+Bates%2C+Michael+Brown%2C+William+Fisher%2C+Kate+Hunicke%2DSmith%2C+David+Pallett%2C+Christine+Pao%2C+Alexander+Rudnicky%2C+and+Elizabeth+Shriberg%2E+1994%2E+Expanding+the+scope+of+the+atis+tas)
9. Bhuwan Dhingra, Lihong Li, Xiujun Li, Jianfeng Gao, Yun-Nung Chen, Faisal Ahmed, and Li Deng. 2016. End-to-end reinforcement learning of dialogue agents for information access. In ACL. [find](https://scholar.google.com/scholar?q=Bhuwan+Dhingra%2C+Lihong+Li%2C+Xiujun+Li%2C+Jianfeng+Gao%2C+Yun%2DNung+Chen%2C+Faisal+Ahmed%2C+and+Li+Deng%2E+2016%2E+End%2Dto%2Dend+reinforcement+learning+of+dialogue+agents+for+information+access%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/dhingra-2017-end-end-reinforcement-learning-dialogue/)
10. Li Dong and Mirella Lapata. 2018. Coarse-to-fine decoding for neural semantic parsing. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 731–742. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Li+Dong+and+Mirella+Lapata%2E+2018%2E+Coarse%2Dto%2Dfine+decoding+for+neural+semantic+parsing%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A+Long+Papers)
11. Catherine Finegan-Dollak, Jonathan K. Kummerfeld, Li Zhang, Karthik Ramanathan Dhanalakshmi Ramanathan, Sesh Sadasivam, Rui Zhang, and Dragomir Radev. 2018. Improving text-to-sql evaluation methodology. In ACL 2018. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Catherine+Finegan%2DDollak%2C+Jonathan+K%2E+Kummerfeld%2C+Li+Zhang%2C+Karthik+Ramanathan+Dhanalakshmi+Ramanathan%2C+Sesh+Sadasivam%2C+Rui+Zhang%2C+and+Dragomir+Radev%2E+2018%2E+Improving+text%2Dto%2Dsql+evaluation+methodolog) [our card](https://md.research.elamur.ai/free/en/papers/dollak-2018-improving-text-sql-evaluation-methodology/)
12. Stefan L. Frank. 2013. Uncertainty reduction as a measure of cognitive load in sentence comprehension. Topics in cognitive science, 5 3:475–94. [find](https://scholar.google.com/scholar?q=Stefan+L%2E+Frank%2E+2013%2E+Uncertainty+reduction+as+a+measure+of+cognitive+load+in+sentence+comprehension%2E+Topics+in+cognitive+science%2C+5+3%3A475%E2%80%9394%2E)
13. Jiaqi Guo, Zecheng Zhan, Yan Gao, Yan Xiao pei Liu Xiao, Jian-Guang Lou, Ting Liu, and Dongmei Zhang. 2019. Towards complex text-to-sql in cross-domain database with intermediate representation. In ACL. [find](https://scholar.google.com/scholar?q=Jiaqi+Guo%2C+Zecheng+Zhan%2C+Yan+Gao%2C+Yan+Xiao+pei+Liu+Xiao%2C+Jian%2DGuang+Lou%2C+Ting+Liu%2C+and+Dongmei+Zhang%2E+2019%2E+Towards+complex+text%2Dto%2Dsql+in+cross%2Ddomain+database+with+intermediate+representation%2E+In+AC)
14. John Hale. 2006. Uncertainty about the rest of the sentence. Cognitive science, 30 4:643–72. [find](https://scholar.google.com/scholar?q=John+Hale%2E+2006%2E+Uncertainty+about+the+rest+of+the+sentence%2E+Cognitive+science%2C+30+4%3A643%E2%80%9372%2E)
15. Charles T. Hemphill, John J. Godfrey, and George R. Doddington. 1990. The atis spoken language systems pilot corpus. In Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27,1990. [find](https://scholar.google.com/scholar?q=Charles+T%2E+Hemphill%2C+John+J%2E+Godfrey%2C+and+George+R%2E+Doddington%2E+1990%2E+The+atis+spoken+language+systems+pilot+corpus%2E+In+Speech+and+Natural+Language%3A+Proceedings+of+a+Workshop+Held+at+Hidden+Valley%2C+Pe)
16. Matthew Henderson, Blaise Thomson, and Jason D. Williams. 2014. The second dialog state tracking challenge. In SIGDIAL Conference. [find](https://scholar.google.com/scholar?q=Matthew+Henderson%2C+Blaise+Thomson%2C+and+Jason+D%2E+Williams%2E+2014%2E+The+second+dialog+state+tracking+challenge%2E+In+SIGDIAL+Conference%2E)
17. Matthew Henderson, Blaise Thomson, and Steve Young. 2013. Deep neural network approach for the dialog state tracking challenge. In Proceedings of the SIGDIAL 2013 Conference, pages 467–471, Metz, France. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Matthew+Henderson%2C+Blaise+Thomson%2C+and+Steve+Young%2E+2013%2E+Deep+neural+network+approach+for+the+dialog+state+tracking+challenge%2E+In+Proceedings+of+the+SIGDIAL+2013+Conference%2C+pages+467%E2%80%93471%2C+Metz%2C+Fran)
18. Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, Jayant Krishnamurthy, and Luke Zettlemoyer. 2017. Learning a neural semantic parser from user feedback. CoRR, abs/1704.08760. [find](https://scholar.google.com/scholar?q=Srinivasan+Iyer%2C+Ioannis+Konstas%2C+Alvin+Cheung%2C+Jayant+Krishnamurthy%2C+and+Luke+Zettlemoyer%2E+2017%2E+Learning+a+neural+semantic+parser+from+user+feedback%2E+CoRR%2C+abs%2F1704%2E08760%2E)
19. Srinivasan Iyer, Ioannis Konstas, Alvin Cheung, and Luke Zettlemoyer. 2016. Summarizing source code using a neural attention model. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2073–2083, Berlin, Germany. Association for Co… [find](https://scholar.google.com/scholar?q=Srinivasan+Iyer%2C+Ioannis+Konstas%2C+Alvin+Cheung%2C+and+Luke+Zettlemoyer%2E+2016%2E+Summarizing+source+code+using+a+neural+attention+model%2E+In+Proceedings+of+the+54th+Annual+Meeting+of+the+Association+for+Com)
20. Mohit Iyyer, Wen-tau Yih, and Ming-Wei Chang. 2017. Search-based neural structured learning for sequential question answering. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1821–1831. Association for Computational Linguistic… [find](https://scholar.google.com/scholar?q=Mohit+Iyyer%2C+Wen%2Dtau+Yih%2C+and+Ming%2DWei+Chang%2E+2017%2E+Search%2Dbased+neural+structured+learning+for+sequential+question+answering%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Computat) [our card](https://md.research.elamur.ai/free/en/papers/iyyer-2017-search-based-neural-structured-learning/)
21. Walter Lasecki, Ece Kamar, and Dan Bohus. 2013. Conversations in the crowd: Collecting data for task-oriented dialog learning. In In Proceedings of the Human Computation Workshop on Scaling Speech and Language Understanding and Dialog through Crowdsourcing at HCOMP 2013. [find](https://scholar.google.com/scholar?q=Walter+Lasecki%2C+Ece+Kamar%2C+and+Dan+Bohus%2E+2013%2E+Conversations+in+the+crowd%3A+Collecting+data+for+task%2Doriented+dialog+learning%2E+In+In+Proceedings+of+the+Human+Computation+Workshop+on+Scaling+Speech+and)
22. Walter S. Lasecki, Mitchell Gordon, Danai Koutra, Malte F. Jung, Steven P. Dow, and Jeffrey P. Bigham. 2014. Glance: Rapidly coding behavioral video with the crowd. In Proceedings of the 27th Annual ACM Symposium on User Interface Software and Technology, New York, NY, USA. ACM. [find](https://scholar.google.com/scholar?q=Walter+S%2E+Lasecki%2C+Mitchell+Gordon%2C+Danai+Koutra%2C+Malte+F%2E+Jung%2C+Steven+P%2E+Dow%2C+and+Jeffrey+P%2E+Bigham%2E+2014%2E+Glance%3A+Rapidly+coding+behavioral+video+with+the+crowd%2E+In+Proceedings+of+the+27th+Annual+A)
23. Ji Young Lee and Franck Dernoncourt. 2016. Sequential short-text classification with recurrent and convolutional neural networks. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 515–520, San Dieg… [find](https://scholar.google.com/scholar?q=Ji+Young+Lee+and+Franck+Dernoncourt%2E+2016%2E+Sequential+short%2Dtext+classification+with+recurrent+and+convolutional+neural+networks%2E+In+Proceedings+of+the+2016+Conference+of+the+North+American+Chapter+of) [our card](https://md.research.elamur.ai/free/en/papers/lee-2016-sequential-short-text-classification-recurrent/)
24. Roger P. Levy. 2008. Expectation-based syntactic comprehension. Cognition, 106:1126–1177. [find](https://scholar.google.com/scholar?q=Roger+P%2E+Levy%2E+2008%2E+Expectation%2Dbased+syntactic+comprehension%2E+Cognition%2C+106%3A1126%E2%80%931177%2E)
25. Fei Li and HV Jagadish. 2014. Constructing an interactive natural language interface for relational databases. VLDB. [find](https://scholar.google.com/scholar?q=Fei+Li+and+HV+Jagadish%2E+2014%2E+Constructing+an+interactive+natural+language+interface+for+relational+databases%2E+VLDB%2E)
26. Jiwei Li, Will Monroe, Alan Ritter, Daniel Jurafsky, Michel Galley, and Jianfeng Gao. 2016. Deep reinforcement learning for dialogue generation. In EMNLP. [find](https://scholar.google.com/scholar?q=Jiwei+Li%2C+Will+Monroe%2C+Alan+Ritter%2C+Daniel+Jurafsky%2C+Michel+Galley%2C+and+Jianfeng+Gao%2E+2016%2E+Deep+reinforcement+learning+for+dialogue+generation%2E+In+EMNLP%2E) [our card](https://md.research.elamur.ai/free/en/papers/li-2016-deep-reinforcement-learning-dialogue-generation/)
27. Yunyao Li, Huahai Yang, and HV Jagadish. 2006. Constructing a generic natural language interface for an xml database. In EDBT, volume 3896, pages 737–754. Springer. [find](https://scholar.google.com/scholar?q=Yunyao+Li%2C+Huahai+Yang%2C+and+HV+Jagadish%2E+2006%2E+Constructing+a+generic+natural+language+interface+for+an+xml+database%2E+In+EDBT%2C+volume+3896%2C+pages+737%E2%80%93754%2E+Springer%2E)
28. Kathleen R. McKeown. 1985. Text Generation: Using Discourse Strategies and Focus Constraints to Generate Natural Language Text. Cambridge University Press, New York, NY, USA. [find](https://scholar.google.com/scholar?q=Kathleen+R%2E+McKeown%2E+1985%2E+Text+Generation%3A+Using+Discourse+Strategies+and+Focus+Constraints+to+Generate+Natural+Language+Text%2E+Cambridge+University+Press%2C+New+York%2C+NY%2C+USA%2E)
29. Lili Mou, Rui Men, Ge Li, Yan Xu, Lu Zhang, Rui Yan, and Zhi Jin. 2016. Natural language inference by tree-based convolution and heuristic matching. In ACL. [find](https://scholar.google.com/scholar?q=Lili+Mou%2C+Rui+Men%2C+Ge+Li%2C+Yan+Xu%2C+Lu+Zhang%2C+Rui+Yan%2C+and+Zhi+Jin%2E+2016%2E+Natural+language+inference+by+tree%2Dbased+convolution+and+heuristic+matching%2E+In+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/mou-2016-natural-language-inference-tree-based/)
30. Nikola Mrkšić, Diarmuid Ó Séaghdha, Tsung-Hsien Wen, Blaise Thomson, and Steve Young. 2017. Neural belief tracker: Data-driven dialogue state tracking. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1777–1788. Association… [find](https://scholar.google.com/scholar?q=Nikola+Mrks%CC%8Cic%CC%81%2C+Diarmuid+O%CC%81+Se%CC%81aghdha%2C+Tsung%2DHsien+Wen%2C+Blaise+Thomson%2C+and+Steve+Young%2E+2017%2E+Neural+belief+tracker%3A+Data%2Ddriven+dialogue+state+tracking%2E+In+Proceedings+of+the+55th+Annual+Meeting+of)
31. Nikola Mrksic, Diarmuid Ó Séaghdha, Blaise Thomson, Milica Gasic, Pei hao Su, David Vandyke, Tsung-Hsien Wen, and Steve J. Young. 2015. Multi-domain dialog state tracking using recurrent neural networks. In ACL. [find](https://scholar.google.com/scholar?q=Nikola+Mrksic%2C+Diarmuid+O%CC%81+Se%CC%81aghdha%2C+Blaise+Thomson%2C+Milica+Gasic%2C+Pei+hao+Su%2C+David+Vandyke%2C+Tsung%2DHsien+Wen%2C+and+Steve+J%2E+Young%2E+2015%2E+Multi%2Ddomain+dialog+state+tracking+using+recurrent+neural+netw)
32. Nikola Mrksic, Diarmuid Ó Séaghdha, Tsung-Hsien Wen, Blaise Thomson, and Steve J. Young. 2017. Neural belief tracker: Data-driven dialogue state tracking. [find](https://scholar.google.com/scholar?q=Nikola+Mrksic%2C+Diarmuid+O%CC%81+Se%CC%81aghdha%2C+Tsung%2DHsien+Wen%2C+Blaise+Thomson%2C+and+Steve+J%2E+Young%2E+2017%2E+Neural+belief+tracker%3A+Data%2Ddriven+dialogue+state+tracking%2E)
33. P. J. Price. 1990. Evaluation of spoken language systems: the atis domain. In Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27,1990, pages 91– 95. [find](https://scholar.google.com/scholar?q=P%2E+J%2E+Price%2E+1990%2E+Evaluation+of+spoken+language+systems%3A+the+atis+domain%2E+In+Speech+and+Natural+Language%3A+Proceedings+of+a+Workshop+Held+at+Hidden+Valley%2C+Pennsylvania%2C+June+24%2D27%2C1990%2C+pages+91%E2%80%93+95%2E)
34. Antoine Raux, Brian Langner, Dan Bohus, Alan W. Black, and Maxine Eskénazi. 2005. Let’s go public! taking a spoken dialog system to the real world. In INTERSPEECH. [find](https://scholar.google.com/scholar?q=Antoine+Raux%2C+Brian+Langner%2C+Dan+Bohus%2C+Alan+W%2E+Black%2C+and+Maxine+Eske%CC%81nazi%2E+2005%2E+Let%E2%80%99s+go+public%21+taking+a+spoken+dialog+system+to+the+real+world%2E+In+INTERSPEECH%2E)
35. Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 1: Long Papers, pages 1073… [find](https://scholar.google.com/scholar?q=Abigail+See%2C+Peter+J%2E+Liu%2C+and+Christopher+D%2E+Manning%2E+2017%2E+Get+to+the+point%3A+Summarization+with+pointer%2Dgenerator+networks%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Computati)
36. Stephanie Seneff and Joseph Polifroni. 2000. Dialogue management in the mercury flight reservation system. In Proceedings of the 2000 ANLP/NAACL Workshop on Conversational Systems - Volume 3, ANLP/NAACL-ConvSyst ’00, pages 11–16. [find](https://scholar.google.com/scholar?q=Stephanie+Seneff+and+Joseph+Polifroni%2E+2000%2E+Dialogue+management+in+the+mercury+flight+reservation+system%2E+In+Proceedings+of+the+2000+ANLP%2FNAACL+Workshop+on+Conversational+Systems+%2D+Volume+3%2C+ANLP%2FNAA)
37. Tianze Shi, Kedar Tatwawadi, Kaushik Chakrabarti, Yi Mao, Oleksandr Polozov, and Weizhu Chen. 2018. Incsql: Training incremental text-to-sql parsers with non-deterministic oracles. arXiv preprint arXiv:1809.05054. — [arXiv:1809.05054](https://arxiv.org/abs/1809.05054) [find](https://scholar.google.com/scholar?q=Tianze+Shi%2C+Kedar+Tatwawadi%2C+Kaushik+Chakrabarti%2C+Yi+Mao%2C+Oleksandr+Polozov%2C+and+Weizhu+Chen%2E+2018%2E+Incsql%3A+Training+incremental+text%2Dto%2Dsql+parsers+with+non%2Ddeterministic+oracles%2E+arXiv+preprint+arXi)
38. Pei hao Su, Milica Gasic, Nikola Mrksic, Lina Maria Rojas-Barahona, Stefan Ultes, David Vandyke, Tsung-Hsien Wen, and Steve J. Young. 2016. Online active reward learning for policy optimisation in spoken dialogue systems. ACL. [find](https://scholar.google.com/scholar?q=Pei+hao+Su%2C+Milica+Gasic%2C+Nikola+Mrksic%2C+Lina+Maria+Rojas%2DBarahona%2C+Stefan+Ultes%2C+David+Vandyke%2C+Tsung%2DHsien+Wen%2C+and+Steve+J%2E+Young%2E+2016%2E+Online+active+reward+learning+for+policy+optimisation+in+spo)
39. Alane Suhr, Srinivasan Iyer, and Yoav Artzi. 2018. Learning to map context-dependent sentences to executable formal queries. In Proceedings of the Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2238–2249. Association for… [find](https://scholar.google.com/scholar?q=Alane+Suhr%2C+Srinivasan+Iyer%2C+and+Yoav+Artzi%2E+2018%2E+Learning+to+map+context%2Ddependent+sentences+to+executable+formal+queries%2E+In+Proceedings+of+the+Conference+of+the+North+American+Chapter+of+the+Assoc)
40. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Advances+in+neural+information+processing+systems%2C+pages+3104%E2%80%933112%2E)
41. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: A method for automatic evaluation of machine translation. In Proceedings of the 40th Annual Meeting on Association for Computational Linguistics, ACL ’02, pages 311–318. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+A+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+Annual+Meeting+on+Association+for+Computat) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
42. Marilyn A. Walker, Alexander I. Rudnicky, Rashmi Prasad, John S. Aberdeen, Elizabeth Owen Bratt, John S. Garofolo, Helen F. Hastie, Audrey N. Le, Bryan L. Pellom, Alexandros Potamianos, Rebecca J. Passonneau, Salim Roukos, Gregory A. Sanders, Stephanie Seneff, and David Stallard. 2002. Darpa communi… [find](https://scholar.google.com/scholar?q=Marilyn+A%2E+Walker%2C+Alexander+I%2E+Rudnicky%2C+Rashmi+Prasad%2C+John+S%2E+Aberdeen%2C+Elizabeth+Owen+Bratt%2C+John+S%2E+Garofolo%2C+Helen+F%2E+Hastie%2C+Audrey+N%2E+Le%2C+Bryan+L%2E+Pellom%2C+Alexandros+Potamianos%2C+Rebecca+J%2E+Pas)
43. Ana-Maria Popescu, Oren Etzioni, and Henry Kautz. 2003. Towards a theory of natural language interfaces to databases. In Proceedings of the 8th international conference on Intelligent user interfaces, pages 149–157. ACM. [find](https://scholar.google.com/scholar?q=Ana%2DMaria+Popescu%2C+Oren+Etzioni%2C+and+Henry+Kautz%2E+2003%2E+Towards+a+theory+of+natural+language+interfaces+to+databases%2E+In+Proceedings+of+the+8th+international+conference+on+Intelligent+user+interfaces%2C)
44. Chenglong Wang, Po-Sen Huang, Alex Polozov, Marc Brockschmidt, and Rishabh Singh. 2018. Execution-guided neural program decoding. In ICML workshop on Neural Abstract Machines and Program Induction v2 (NAMPI). [find](https://scholar.google.com/scholar?q=Chenglong+Wang%2C+Po%2DSen+Huang%2C+Alex+Polozov%2C+Marc+Brockschmidt%2C+and+Rishabh+Singh%2E+2018%2E+Execution%2Dguided+neural+program+decoding%2E+In+ICML+workshop+on+Neural+Abstract+Machines+and+Program+Induction+v2+)
45. Jonathan Kraft, Caiming Xiong, Richard Socher, and Dragomir Radev. 2019. Sparc: Cross-domain semantic parsing in context. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jonathan+Kraft%2C+Caiming+Xiong%2C+Richard+Socher%2C+and+Dragomir+Radev%2E+2019%2E+Sparc%3A+Cross%2Ddomain+semantic+parsing+in+context%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational) [our card](https://md.research.elamur.ai/free/en/papers/yu-2019-sparc-cross-domain-semantic-parsing/)
46. David HD Warren and Fernando CN Pereira. 1982. An efficient easily adaptable system for interpreting natural language queries. Computational Linguistics, 8(3-4):110–122. [find](https://scholar.google.com/scholar?q=David+HD+Warren+and+Fernando+CN+Pereira%2E+1982%2E+An+efficient+easily+adaptable+system+for+interpreting+natural+language+queries%2E+Computational+Linguistics%2C+8%283%2D4%29%3A110%E2%80%93122%2E)
47. Tsung-Hsien Wen, Milica Gasic, Nikola Mrksic, Lina Maria Rojas-Barahona, Pei-Hao Su, Stefan Ultes, David Vandyke, and Steve J. Young. 2016. A network-based end-to-end trainable task-oriented dialogue system. CoRR. [find](https://scholar.google.com/scholar?q=Tsung%2DHsien+Wen%2C+Milica+Gasic%2C+Nikola+Mrksic%2C+Lina+Maria+Rojas%2DBarahona%2C+Pei%2DHao+Su%2C+Stefan+Ultes%2C+David+Vandyke%2C+and+Steve+J%2E+Young%2E+2016%2E+A+network%2Dbased+end%2Dto%2Dend+trainable+task%2Doriented+dialogue+)
48. Sam Wiseman, Stuart Shieber, and Alexander Rush. 2017. Challenges in data-to-document generation. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 2253–2263. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sam+Wiseman%2C+Stuart+Shieber%2C+and+Alexander+Rush%2E+2017%2E+Challenges+in+data%2Dto%2Ddocument+generation%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+2253%E2%80%93) [our card](https://md.research.elamur.ai/free/en/papers/wiseman-2017-challenges-data-document-generation/)
49. Chien-Sheng Wu, Andrea Madotto, Ehsan Hosseini-Asl, Caiming Xiong, Richard Socher, and Pascale Fung. 2019. Transferable multi-domain state generator for task-oriented dialogue systems. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).… [find](https://scholar.google.com/scholar?q=Chien%2DSheng+Wu%2C+Andrea+Madotto%2C+Ehsan+Hosseini%2DAsl%2C+Caiming+Xiong%2C+Richard+Socher%2C+and+Pascale+Fung%2E+2019%2E+Transferable+multi%2Ddomain+state+generator+for+task%2Doriented+dialogue+systems%2E+In+Proceedings+)
50. Kun Xu, Lingfei Wu, Zhiguo Wang, Mo Yu, Liwei Chen, and Vadim Sheinin. 2018. Sql-to-text generation with graph-to-sequence model. EMNLP. [find](https://scholar.google.com/scholar?q=Kun+Xu%2C+Lingfei+Wu%2C+Zhiguo+Wang%2C+Mo+Yu%2C+Liwei+Chen%2C+and+Vadim+Sheinin%2E+2018%2E+Sql%2Dto%2Dtext+generation+with+graph%2Dto%2Dsequence+model%2E+EMNLP%2E)
51. Xiaojun Xu, Chang Liu, and Dawn Song. 2017. Sqlnet: Generating structured queries from natural language without reinforcement learning. arXiv preprint arXiv:1711.04436. [find](https://scholar.google.com/scholar?q=Xiaojun+Xu%2C+Chang+Liu%2C+and+Dawn+Song%2E+2017%2E+Sqlnet%3A+Generating+structured+queries+from+natural+language+without+reinforcement+learning%2E+arXiv+preprint+arXiv%3A1711%2E04436%2E)
52. Navid Yaghmazadeh, Yuepeng Wang, Isil Dillig, and Thomas Dillig. 2017. Sqlizer: query synthesis from natural language. Proceedings of the ACM on Programming Languages. [find](https://scholar.google.com/scholar?q=Navid+Yaghmazadeh%2C+Yuepeng+Wang%2C+Isil+Dillig%2C+and+Thomas+Dillig%2E+2017%2E+Sqlizer%3A+query+synthesis+from+natural+language%2E+Proceedings+of+the+ACM+on+Programming+Languages%2E)
53. Tao Yu, Zifan Li, Zilin Zhang, Rui Zhang, and Dragomir Radev. 2018a. Typesql: Knowledge-based type-aware neural text-to-sql generation. In Proceedings of NAACL. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Tao+Yu%2C+Zifan+Li%2C+Zilin+Zhang%2C+Rui+Zhang%2C+and+Dragomir+Radev%2E+2018a%2E+Typesql%3A+Knowledge%2Dbased+type%2Daware+neural+text%2Dto%2Dsql+generation%2E+In+Proceedings+of+NAACL%2E+Association+for+Computational+Linguisti)
54. Tao Yu, Michihiro Yasunaga, Kai Yang, Rui Zhang, Dongxu Wang, Zifan Li, and Dragomir Radev. 2018b. Syntaxsqlnet: Syntax tree networks for complex and cross-domain text-to-sql task. In Proceedings of EMNLP. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Tao+Yu%2C+Michihiro+Yasunaga%2C+Kai+Yang%2C+Rui+Zhang%2C+Dongxu+Wang%2C+Zifan+Li%2C+and+Dragomir+Radev%2E+2018b%2E+Syntaxsqlnet%3A+Syntax+tree+networks+for+complex+and+cross%2Ddomain+text%2Dto%2Dsql+task%2E+In+Proceedings+of+E)
55. Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, Zilin Zhang, and Dragomir Radev. 2018c. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-SQL task. In Proceedings of the 2018… [find](https://scholar.google.com/scholar?q=Tao+Yu%2C+Rui+Zhang%2C+Kai+Yang%2C+Michihiro+Yasunaga%2C+Dongxu+Wang%2C+Zifan+Li%2C+James+Ma%2C+Irene+Li%2C+Qingning+Yao%2C+Shanelle+Roman%2C+Zilin+Zhang%2C+and+Dragomir+Radev%2E+2018c%2E+Spider%3A+A+large%2Dscale+human%2Dlabeled+da) [our card](https://md.research.elamur.ai/free/en/papers/yu-2018-spider-large-scale-human-labeled/)
56. Tao Yu, Rui Zhang, Michihiro Yasunaga, Yi Chern Tan, Xi Victoria Lin, Suyi Li, Irene Li Heyang Er, Bo Pang, Tao Chen, Emily Ji, Shreya Dixit, David Proctor, Sungrok Shim, Vincent Zhang John M. Zelle and Raymond J. Mooney. 1996. Learning to parse database queries using inductive logic programming. In… [find](https://scholar.google.com/scholar?q=Tao+Yu%2C+Rui+Zhang%2C+Michihiro+Yasunaga%2C+Yi+Chern+Tan%2C+Xi+Victoria+Lin%2C+Suyi+Li%2C+Irene+Li+Heyang+Er%2C+Bo+Pang%2C+Tao+Chen%2C+Emily+Ji%2C+Shreya+Dixit%2C+David+Proctor%2C+Sungrok+Shim%2C+Vincent+Zhang+John+M%2E+Zelle+a)
57. Luke S. Zettlemoyer and Michael Collins. 2005. Learning to map sentences to logical form: Structured classification with probabilistic categorial grammars. UAI. [find](https://scholar.google.com/scholar?q=Luke+S%2E+Zettlemoyer+and+Michael+Collins%2E+2005%2E+Learning+to+map+sentences+to+logical+form%3A+Structured+classification+with+probabilistic+categorial+grammars%2E+UAI%2E)
58. Rui Zhang, Tao Yu, He Yang Er, Sungrok Shim, Eric Xue, Xi Victoria Lin, Tianze Shi, Caiming Xiong, Richard Socher, and Dragomir Radev. 2019. Editing-based sql query generation for cross-domain context-dependent questions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language… [find](https://scholar.google.com/scholar?q=Rui+Zhang%2C+Tao+Yu%2C+He+Yang+Er%2C+Sungrok+Shim%2C+Eric+Xue%2C+Xi+Victoria+Lin%2C+Tianze+Shi%2C+Caiming+Xiong%2C+Richard+Socher%2C+and+Dragomir+Radev%2E+2019%2E+Editing%2Dbased+sql+query+generation+for+cross%2Ddomain+context)
59. Victor Zhong, Caiming Xiong, and Richard Socher. 2017. Seq2sql: Generating structured queries from natural language using reinforcement learning. CoRR, abs/1709.00103. [find](https://scholar.google.com/scholar?q=Victor+Zhong%2C+Caiming+Xiong%2C+and+Richard+Socher%2E+2017%2E+Seq2sql%3A+Generating+structured+queries+from+natural+language+using+reinforcement+learning%2E+CoRR%2C+abs%2F1709%2E00103%2E)
60. Victor Zhong, Caiming Xiong, and Richard Socher. 2018. Global-locally self-attentive encoder for dialogue state tracking. In ACL. [find](https://scholar.google.com/scholar?q=Victor+Zhong%2C+Caiming+Xiong%2C+and+Richard+Socher%2E+2018%2E+Global%2Dlocally+self%2Dattentive+encoder+for+dialogue+state+tracking%2E+In+ACL%2E)