---
title: "A Stack-Propagation Framework with Token-Level — Qin 2019"
description: "Spoken language understanding (SLU) in task-oriented dialogue combines two linked subtasks: intent detection, which classifies an utterance, and slot…"
authors: ["Libo Qin", "Wanxiang Che", "Yangming Li", "Haoyang Wen", "Ting Liu"]
year: 2019
venue: "Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)"
source: "https://aclanthology.org/D19-1214/"
doi: "10.18653/v1/d19-1214"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/qin-2019-stack-propagation-framework-token-level/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/qin-2019-stack-propagation-framework-token-level/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/) / A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding

# A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding

**Authors:** [Libo Qin](https://md.research.elamur.ai/free/en/authors/libo-qin/), [Wanxiang Che](https://md.research.elamur.ai/free/en/authors/wanxiang-che/), Yangming Li, [Haoyang Wen](https://md.research.elamur.ai/free/en/authors/haoyang-wen/), [Ting Liu](https://md.research.elamur.ai/free/en/authors/ting-liu/) · **Year:** 2019 · **Venue:** Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/D19-1214/) · [PDF](https://aclanthology.org/D19-1214.pdf) · [DOI 10.18653/v1/d19-1214](https://doi.org/10.18653/v1/d19-1214) · [OpenAlex](https://openalex.org/W2971167298)

**Citations:** 298 citations (openalex-snapshot · 2026-10-14) · [86](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
Spoken language understanding (SLU) in task-oriented dialogue combines two linked subtasks: intent detection, which classifies an utterance, and slot filling, which labels its tokens. The slots tend to depend on the intent. Earlier joint models either shared parameters only or passed intent information through gate mechanisms. Gates are hard to interpret, and a single utterance-level intent can corrupt every slot decision if it is wrong. Qin et al. instead adapt Stack-Propagation, previously used for POS tagging and parsing. In their joint model, the predicted intent feeds directly into the slot-filling decoder, and the whole pipeline stays differentiable. A self-attentive encoder predicts an intent at every token, and the utterance intent is chosen by voting. If some tokens get the wrong intent, the correct ones still guide their own slots, which limits error propagation. On the SNIPS and ATIS benchmarks the model reached state-of-the-art results. Replacing the encoder with fine-tuned BERT raised scores further. The work shows that explicit, token-level intent signals give a more interpretable and more accurate way to couple the two SLU subtasks.

## Key points
- Without BERT, the model scores 94.2 slot F1, 98.0 intent accuracy and 86.9 overall accuracy on SNIPS. On ATIS it scores 95.9, 96.9 and 86.5 respectively.
- With BERT base as the encoder, results rise to 97.0 / 99.0 / 92.9 on SNIPS and 96.1 / 97.5 / 88.6 on ATIS. This slightly exceeds the BERT SLU baseline (Chen et al., 2019) in overall accuracy on both datasets.
- Separate BERT models for intent detection alone (using [CLS]) and for slot filling alone did worse than the joint BERT version. This supports modelling the two tasks together.
- Token-level intent voting works like an ensemble: it reduces prediction variance and keeps richer intent features for the slot decoder.
- The code is released at github.com/LeePleased/StackPropagation-SLU.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

86 papers of our library cite it (of the 140884 whose bibliographies we read); 1 of them have a page here, 85 not yet.

- [End-to-End Slot Alignment and Recognition for Cross-Lingual NLU](https://md.research.elamur.ai/free/en/papers/xu-2020-end-end-slot-alignment-recognition/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{qin2019stack,
  title = {{A Stack-Propagation Framework with Token-Level Intent Detection for Spoken Language Understanding}},
  author = {Qin, Libo and Che, Wanxiang and Li, Yangming and Wen, Haoyang and Liu, Ting},
  booktitle = {Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP)},
  year = {2019},
  doi = {10.18653/v1/d19-1214},
  url = {https://doi.org/10.18653/v1/d19-1214}
}
```

- [← previous A Simple Joint Model for Improved Contextual Neural Lemmatization](https://md.research.elamur.ai/free/en/papers/ud-2019-malaviya-simple-joint-model-improved-contextual/)
- [next → A Structural Probe for Finding Syntax in Word Representations](https://md.research.elamur.ai/free/en/papers/hewitt-2019-structural-probe-finding-syntax-word/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/)

## References

24 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Qian Chen, Zhu Zhuo, and Wen Wang. 2019. Bert for joint intent classification and slot filling. arXiv preprint arXiv:1902.10909. — [arXiv:1902.10909](https://arxiv.org/abs/1902.10909) [find](https://scholar.google.com/scholar?q=Qian+Chen%2C+Zhu+Zhuo%2C+and+Wen+Wang%2E+2019%2E+Bert+for+joint+intent+classification+and+slot+filling%2E+arXiv+preprint+arXiv%3A1902%2E10909%2E+%E2%80%94+%5BarXiv%3A1902%2E10909%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1902%2E10909%29)
2. Alice Coucke, Alaa Saade, Adrien Ball, Théodore Bluche, Alexandre Caulier, David Leroy, Clément Doumouro, Thibault Gisselbrecht, Francesco Caltagirone, Thibaut Lavril, et al. 2018. Snips voice platform: an embedded spoken language understanding system for private-by-design voice interfaces. arXiv… [find](https://scholar.google.com/scholar?q=Alice+Coucke%2C+Alaa+Saade%2C+Adrien+Ball%2C+The%CC%81odore+Bluche%2C+Alexandre+Caulier%2C+David+Leroy%2C+Cle%CC%81ment+Doumouro%2C+Thibault+Gisselbrecht%2C+Francesco+Caltagirone%2C+Thibaut+Lavril%2C+et+al%2E+2018%2E+Snips+voice+platf) [our card](https://md.research.elamur.ai/free/en/papers/coucke-2018-snips-voice-platform-embedded-spoken/)
3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2018%2E+Bert%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+arXiv+preprint+arXiv%3A1810%2E04805%2E)
4. Haihong E, Peiqing Niu, Zhongfu Chen, and Meina Song. 2019. A novel bi-directional interrelated model for joint intent detection and slot filling. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Haihong+E%2C+Peiqing+Niu%2C+Zhongfu+Chen%2C+and+Meina+Song%2E+2019%2E+A+novel+bi%2Ddirectional+interrelated+model+for+joint+intent+detection+and+slot+filling%2E+In+Proc%2E+of+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/e-2019-novel-bi-directional-interrelated-model/)
5. Chih-Wen Goo, Guang Gao, Yun-Kai Hsu, Chih-Li Huo, Tsung-Chieh Chen, Keng-Wei Hsu, and Yun-Nung Chen. 2018. Slot-gated modeling for joint slot filling and intent prediction. In Proc. of NAACL. [find](https://scholar.google.com/scholar?q=Chih%2DWen+Goo%2C+Guang+Gao%2C+Yun%2DKai+Hsu%2C+Chih%2DLi+Huo%2C+Tsung%2DChieh+Chen%2C+Keng%2DWei+Hsu%2C+and+Yun%2DNung+Chen%2E+2018%2E+Slot%2Dgated+modeling+for+joint+slot+filling+and+intent+prediction%2E+In+Proc%2E+of+NAACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/goo-2018-slot-gated-modeling-joint-slot/)
6. Patrick Haffner, Gokhan Tur, and Jerry H Wright. 2003. Optimizing svms for complex call classification. In In Proc. of ICASSP. [find](https://scholar.google.com/scholar?q=Patrick+Haffner%2C+Gokhan+Tur%2C+and+Jerry+H+Wright%2E+2003%2E+Optimizing+svms+for+complex+call+classification%2E+In+In+Proc%2E+of+ICASSP%2E)
7. Dilek Hakkani-Tür, Gökhan Tür, Asli Celikyilmaz, Yun-Nung Chen, Jianfeng Gao, Li Deng, and Ye-Yi Wang. 2016. Multi-domain joint semantic frame parsing using bi-directional rnn-lstm. In Interspeech. [find](https://scholar.google.com/scholar?q=Dilek+Hakkani%2DTu%CC%88r%2C+Go%CC%88khan+Tu%CC%88r%2C+Asli+Celikyilmaz%2C+Yun%2DNung+Chen%2C+Jianfeng+Gao%2C+Li+Deng%2C+and+Ye%2DYi+Wang%2E+2016%2E+Multi%2Ddomain+joint+semantic+frame+parsing+using+bi%2Ddirectional+rnn%2Dlstm%2E+In+Interspeech%2E)
8. Charles T Hemphill, John J Godfrey, and George R Doddington. 1990. The atis spoken language systems pilot corpus. In Speech and Natural Language: Proceedings of a Workshop Held at Hidden Valley, Pennsylvania, June 24-27, 1990. [find](https://scholar.google.com/scholar?q=Charles+T+Hemphill%2C+John+J+Godfrey%2C+and+George+R+Doddington%2E+1990%2E+The+atis+spoken+language+systems+pilot+corpus%2E+In+Speech+and+Natural+Language%3A+Proceedings+of+a+Workshop+Held+at+Hidden+Valley%2C+Penns)
9. Diederik P Kingma and Jimmy Ba. 2014. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980. [find](https://scholar.google.com/scholar?q=Diederik+P+Kingma+and+Jimmy+Ba%2E+2014%2E+Adam%3A+A+method+for+stochastic+optimization%2E+arXiv+preprint+arXiv%3A1412%2E6980%2E)
10. Stefan Lee, Senthil Purushwalkam Shiva Prakash, Michael Cogswell, Viresh Ranjan, David Crandall, and Dhruv Batra. 2016. Stochastic multiple choice learning for training diverse deep ensembles. In NIPS. [find](https://scholar.google.com/scholar?q=Stefan+Lee%2C+Senthil+Purushwalkam+Shiva+Prakash%2C+Michael+Cogswell%2C+Viresh+Ranjan%2C+David+Crandall%2C+and+Dhruv+Batra%2E+2016%2E+Stochastic+multiple+choice+learning+for+training+diverse+deep+ensembles%2E+In+NIPS)
11. Bing Liu and Ian Lane. 2016. Attention-based recurrent neural network models for joint intent detection and slot filling. arXiv preprint arXiv:1609.01454. — [arXiv:1609.01454](https://arxiv.org/abs/1609.01454) [find](https://scholar.google.com/scholar?q=Bing+Liu+and+Ian+Lane%2E+2016%2E+Attention%2Dbased+recurrent+neural+network+models+for+joint+intent+detection+and+slot+filling%2E+arXiv+preprint+arXiv%3A1609%2E01454%2E+%E2%80%94+%5BarXiv%3A1609%2E01454%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F16)
12. Christian Raymond and Giuseppe Riccardi. 2007. Generative and discriminative algorithms for spoken language understanding. In Eighth Annual Conference of the International Speech Communication Association. [find](https://scholar.google.com/scholar?q=Christian+Raymond+and+Giuseppe+Riccardi%2E+2007%2E+Generative+and+discriminative+algorithms+for+spoken+language+understanding%2E+In+Eighth+Annual+Conference+of+the+International+Speech+Communication+Associa)
13. Ruhi Sarikaya, Geoffrey E Hinton, and Bhuvana Ramabhadran. 2011. Deep belief nets for natural language call-routing. In ICASSP. [find](https://scholar.google.com/scholar?q=Ruhi+Sarikaya%2C+Geoffrey+E+Hinton%2C+and+Bhuvana+Ramabhadran%2E+2011%2E+Deep+belief+nets+for+natural+language+call%2Drouting%2E+In+ICASSP%2E)
14. Zhixing Tan, Mingxuan Wang, Jun Xie, Yidong Chen, and Xiaodong Shi. 2018. Deep semantic role labeling with self-attention. In Proc. of AAAI. [find](https://scholar.google.com/scholar?q=Zhixing+Tan%2C+Mingxuan+Wang%2C+Jun+Xie%2C+Yidong+Chen%2C+and+Xiaodong+Shi%2E+2018%2E+Deep+semantic+role+labeling+with+self%2Dattention%2E+In+Proc%2E+of+AAAI%2E) [our card](https://md.research.elamur.ai/free/en/papers/tan-2017-deep-semantic-role-labeling-self/)
15. Gokhan Tur and Renato De Mori. 2011. Spoken language understanding: Systems for extracting semantic information from speech. John Wiley & Sons. [find](https://scholar.google.com/scholar?q=Gokhan+Tur+and+Renato+De+Mori%2E+2011%2E+Spoken+language+understanding%3A+Systems+for+extracting+semantic+information+from+speech%2E+John+Wiley+%26+Sons%2E)
16. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, L ukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. In NIPS. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+L+ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+In+NIPS%2E)
17. Yu Wang, Yilin Shen, and Hongxia Jin. 2018. A bi-model based rnn semantic frame parsing model for intent detection and slot filling. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Yu+Wang%2C+Yilin+Shen%2C+and+Hongxia+Jin%2E+2018%2E+A+bi%2Dmodel+based+rnn+semantic+frame+parsing+model+for+intent+detection+and+slot+filling%2E+In+Proc%2E+of+ACL%2E)
18. Puyang Xu and Ruhi Sarikaya. 2013. Convolutional neural network based triangular crf for joint intent detection and slot filling. In 2013 IEEE Workshop on Automatic Speech Recognition and Understanding. [find](https://scholar.google.com/scholar?q=Puyang+Xu+and+Ruhi+Sarikaya%2E+2013%2E+Convolutional+neural+network+based+triangular+crf+for+joint+intent+detection+and+slot+filling%2E+In+2013+IEEE+Workshop+on+Automatic+Speech+Recognition+and+Understandin)
19. Kaisheng Yao, Baolin Peng, Yu Zhang, Dong Yu, Geoffrey Zweig, and Yangyang Shi. 2014. Spoken language understanding using long short-term memory neural networks. In SLT. [find](https://scholar.google.com/scholar?q=Kaisheng+Yao%2C+Baolin+Peng%2C+Yu+Zhang%2C+Dong+Yu%2C+Geoffrey+Zweig%2C+and+Yangyang+Shi%2E+2014%2E+Spoken+language+understanding+using+long+short%2Dterm+memory+neural+networks%2E+In+SLT%2E)
20. Qingyu Yin, Yu Zhang, Wei-Nan Zhang, Ting Liu, and William Yang Wang. 2018. Deep reinforcement learning for chinese zero pronoun resolution. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Qingyu+Yin%2C+Yu+Zhang%2C+Wei%2DNan+Zhang%2C+Ting+Liu%2C+and+William+Yang+Wang%2E+2018%2E+Deep+reinforcement+learning+for+chinese+zero+pronoun+resolution%2E+In+Proc%2E+of+ACL%2E) [our card](https://md.research.elamur.ai/free/en/papers/yin-2018-deep-reinforcement-learning-chinese-zero/)
21. Chenwei Zhang, Yaliang Li, Nan Du, Wei Fan, and Philip Yu. 2019. Joint slot filling and intent detection via capsule neural networks. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Chenwei+Zhang%2C+Yaliang+Li%2C+Nan+Du%2C+Wei+Fan%2C+and+Philip+Yu%2E+2019%2E+Joint+slot+filling+and+intent+detection+via+capsule+neural+networks%2E+In+Proc%2E+of+ACL%2E)
22. Xiaodong Zhang and Houfeng Wang. 2016. A joint model of intent determination and slot filling for spoken language understanding. In Proc. of IJCAI. [find](https://scholar.google.com/scholar?q=Xiaodong+Zhang+and+Houfeng+Wang%2E+2016%2E+A+joint+model+of+intent+determination+and+slot+filling+for+spoken+language+understanding%2E+In+Proc%2E+of+IJCAI%2E)
23. Yuan Zhang and David Weiss. 2016. Stack-propagation: Improved representation learning for syntax. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Yuan+Zhang+and+David+Weiss%2E+2016%2E+Stack%2Dpropagation%3A+Improved+representation+learning+for+syntax%2E+In+Proc%2E+of+ACL%2E)
24. Victor Zhong, Caiming Xiong, and Richard Socher. 2018. Global-locally self-attentive encoder for dialogue state tracking. In Proc. of ACL. [find](https://scholar.google.com/scholar?q=Victor+Zhong%2C+Caiming+Xiong%2C+and+Richard+Socher%2E+2018%2E+Global%2Dlocally+self%2Dattentive+encoder+for+dialogue+state+tracking%2E+In+Proc%2E+of+ACL%2E)