---
title: "Joint Slot Filling and Intent Detection via Capsule Neural Networks"
description: "The authors propose CAPSULE-NLU, a capsule network with three layers: WordCaps, SlotCaps and… 22 references, cited by 62 papers here."
authors: ["Chenwei Zhang", "Yaliang Li", "Nan Du", "Wei Fan", "Philip Yu"]
year: 2019
venue: "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/P19-1519/"
doi: "10.18653/v1/p19-1519"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2019-joint-slot-filling-intent-detection/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2019-joint-slot-filling-intent-detection/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/3/) / Joint Slot Filling and Intent Detection via Capsule Neural Networks

# Joint Slot Filling and Intent Detection via Capsule Neural Networks

**Authors:** [Chenwei Zhang](https://md.research.elamur.ai/free/en/authors/chenwei-zhang/), [Yaliang Li](https://md.research.elamur.ai/free/en/authors/yaliang-li/), [Nan Du](https://md.research.elamur.ai/free/en/authors/nan-du/), [Wei Fan](https://md.research.elamur.ai/free/en/authors/wei-fan/) and [Philip Yu](https://md.research.elamur.ai/free/en/authors/philip-s-yu/) · **Year:** 2019 · **Venue:** [Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2019](https://md.research.elamur.ai/free/en/venues/acl/2019/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P19-1519/) · [PDF](https://aclanthology.org/P19-1519.pdf) · [DOI 10.18653/v1/p19-1519](https://doi.org/10.18653/v1/p19-1519) · [OpenAlex](https://openalex.org/W2963033987)

**Citations:** 253 citations (OpenAlex, 2026-10-14) · [62](https://md.research.elamur.ai/free/en/papers/zhang-2019-joint-slot-filling-intent-detection/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Voice assistants and similar NLU systems must make two linked decisions. They tag each word with a slot label, such as marking "Sungmin" as an artist. They also assign an intent to the whole utterance, such as AddToPlaylist. Earlier systems either ran these steps as a pipeline or used joint sequence-labeling models that compress the utterance into one vector, which ignores the natural word → slot → intent hierarchy.

The authors propose CAPSULE-NLU, a capsule network with three layers: WordCaps, SlotCaps and IntentCaps. WordCaps produces context-aware word vectors with a BiLSTM. Instead of CRF tagging, words are assigned to slot capsules through dynamic routing-by-agreement. The resulting slot representations are then routed to intent capsules. A new re-routing step feeds the predicted intent back into the word-to-slot routing to refine slot labels.

On two real-world datasets, the model beat alternative architectures and existing commercial NLU services. The work shows that modeling the semantic hierarchy explicitly lets slot filling and intent detection reinforce each other.

## Key points
- **Architecture:** CAPSULE-NLU has three capsule types.
  - WordCaps is a BiLSTM with word embeddings trained from scratch; the authors note that ELMo or BERT could be plugged in.
  - SlotCaps holds one capsule per slot type.
  - IntentCaps holds one capsule per intent label.
- **Routing method:** Slot filling is not done by sequence labeling. Each word's prediction vector is routed to slots using the dynamic routing-by-agreement algorithm of Sabour et al. (2017), with a squash nonlinearity and iterative agreement updates.
- **Re-routing:** The inferred intent representation re-enters the WordCaps–SlotCaps routing. This helps recover slots that the first routing pass missed.
- **Interpretability:** Visualizations of routing weights show how extra iterations help. For example, the model groups "operetta for the theatre organ" as a single object_name span, and these slots then push the utterance toward the correct SearchCreativeWork intent.
- **Evaluation:** The model was compared with joint baselines (e.g., slot-gated and attention-based RNN models) and with commercial NLU services on two real-world datasets. The supplied excerpt omits the experimental section, so dataset names and exact scores are not included here.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 137 works · 83,262 citations
- [University of Illinois Chicago](https://md.research.elamur.ai/free/en/universities/university-of-illinois-chicago/) — University, Chicago, United States · 38 works · 19,844 citations

Also on the byline: [Alibaba Group (United States)](https://ror.org/00rn0m335).

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: Chenwei Zhang, Yaliang Li, Nan Du, Wei Fan and Philip Yu. 2019. Joint Slot Filling and Intent Detection via Capsule Neural Networks. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2019joint,
  title = {{Joint Slot Filling and Intent Detection via Capsule Neural Networks}},
  author = {Chenwei Zhang, Yaliang Li, Nan Du, Wei Fan and Philip Yu},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1519},
  url = {https://doi.org/10.18653/v1/p19-1519}
}
```

- [← previous Document Expansion by Query Prediction](https://md.research.elamur.ai/free/en/papers/nogueira-2019-document-expansion-query-prediction/)
- [next → Choosing Transfer Languages for Cross-Lingual Learning](https://md.research.elamur.ai/free/en/papers/lin-2019-choosing-transfer-languages-cross-lingual/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/3/)

## References

22 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Yun-Nung Chen, Dilek Hakkani-Tür, Gökhan Tür, Jianfeng Gao, and Li Deng. 2016. End-to-end memory networks with knowledge carryover for multi-turn spoken language understanding. In INTER-SPEECH, pages 3245–3249. [find](https://scholar.google.com/scholar?q=Yun%2DNung+Chen%2C+Dilek+Hakkani%2DTu%CC%88r%2C+Go%CC%88khan+Tu%CC%88r%2C+Jianfeng+Gao%2C+and+Li+Deng%2E+2016%2E+End%2Dto%2Dend+memory+networks+with+knowledge+carryover+for+multi%2Dturn+spoken+language+understanding%2E+In+INTER%2DSPEECH%2C+pag)
2. Jianpeng Cheng, Li Dong, and Mirella Lapata. 2016. [Long short-term memory-networks for machine reading](https://md.research.elamur.ai/free/en/papers/cheng-2016-long-short-term-memory-networks/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 551–561.
3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805.
4. Jingjing Gong, Xipeng Qiu, Shaojing Wang, and Xuanjing Huang. 2018. Information aggregation via dynamic routing for sequence encoding. In Proceedings of the 27th International Conference on Computational Linguistics, pages 2742–2752. [find](https://scholar.google.com/scholar?q=Jingjing+Gong%2C+Xipeng+Qiu%2C+Shaojing+Wang%2C+and+Xuanjing+Huang%2E+2018%2E+Information+aggregation+via+dynamic+routing+for+sequence+encoding%2E+In+Proceedings+of+the+27th+International+Conference+on+Computatio)
5. Chih-Wen Goo, Guang Gao, Yun-Kai Hsu, Chih-Li Huo, Tsung-Chieh Chen, Keng-Wei Hsu, and Yun-Nung Chen. 2018. [Slot-gated modeling for joint slot filling and intent prediction](https://md.research.elamur.ai/free/en/papers/goo-2018-slot-gated-modeling-joint-slot/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human La…
6. Dilek Hakkani-Tür, Gökhan Tür, Asli Celikyilmaz, Yun-Nung Chen, Jianfeng Gao, Li Deng, and Ye-Yi Wang. 2016. Multi-domain joint semantic frame parsing using bi-directional rnn-lstm. In Interspeech, pages 715–719. [find](https://scholar.google.com/scholar?q=Dilek+Hakkani%2DTu%CC%88r%2C+Go%CC%88khan+Tu%CC%88r%2C+Asli+Celikyilmaz%2C+Yun%2DNung+Chen%2C+Jianfeng+Gao%2C+Li+Deng%2C+and+Ye%2DYi+Wang%2E+2016%2E+Multi%2Ddomain+joint+semantic+frame+parsing+using+bi%2Ddirectional+rnn%2Dlstm%2E+In+Interspeech%2C)
7. Geoffrey E Hinton, Alex Krizhevsky, and Sida D Wang. 2011. Transforming auto-encoders. In International Conference on Artificial Neural Networks, pages 44–51. Springer. [find](https://scholar.google.com/scholar?q=Geoffrey+E+Hinton%2C+Alex+Krizhevsky%2C+and+Sida+D+Wang%2E+2011%2E+Transforming+auto%2Dencoders%2E+In+International+Conference+on+Artificial+Neural+Networks%2C+pages+44%E2%80%9351%2E+Springer%2E)
8. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural computation, 9(8):1735–1780. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+Ju%CC%88rgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+computation%2C+9%288%29%3A1735%E2%80%931780%2E)
9. Jian Hu, Gang Wang, Fred Lochovsky, Jian-tao Sun, and Zheng Chen. 2009. Understanding user’s query intent with wikipedia. In WWW, pages 471–480. ACM. [find](https://scholar.google.com/scholar?q=Jian+Hu%2C+Gang+Wang%2C+Fred+Lochovsky%2C+Jian%2Dtao+Sun%2C+and+Zheng+Chen%2E+2009%2E+Understanding+user%E2%80%99s+query+intent+with+wikipedia%2E+In+WWW%2C+pages+471%E2%80%93480%2E+ACM%2E)
10. John D. Lafferty, Andrew McCallum, and Fernando C. N. Pereira. 2001. Conditional random fields: Probabilistic models for segmenting and labeling sequence data. In ICML, ICML 2001, pages 282–289. [find](https://scholar.google.com/scholar?q=John+D%2E+Lafferty%2C+Andrew+McCallum%2C+and+Fernando+C%2E+N%2E+Pereira%2E+2001%2E+Conditional+random+fields%3A+Probabilistic+models+for+segmenting+and+labeling+sequence+data%2E+In+ICML%2C+ICML+2001%2C+pages+282%E2%80%93289%2E)
11. Bing Liu and Ian Lane. 2016. Attention-based recurrent neural network models for joint intent detection and slot filling. Interspeech, pages 685–689. [find](https://scholar.google.com/scholar?q=Bing+Liu+and+Ian+Lane%2E+2016%2E+Attention%2Dbased+recurrent+neural+network+models+for+joint+intent+detection+and+slot+filling%2E+Interspeech%2C+pages+685%E2%80%93689%2E)
12. Matthew E Peters, Mark Neumann, Mohit Iyyer, Matt Gardner, Christopher Clark, Kenton Lee, and Luke Zettlemoyer. 2018. [Deep contextualized word representations](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/). arXiv preprint arXiv:1802.05365.
13. Sara Sabour, Nicholas Frosst, and Geoffrey E Hinton. 2017. Dynamic routing between capsules. In NIPS, pages 3859–3869. [find](https://scholar.google.com/scholar?q=Sara+Sabour%2C+Nicholas+Frosst%2C+and+Geoffrey+E+Hinton%2E+2017%2E+Dynamic+routing+between+capsules%2E+In+NIPS%2C+pages+3859%E2%80%933869%2E)
14. Tao Shen, Tianyi Zhou, Guodong Long, Jing Jiang, Shirui Pan, and Chengqi Zhang. 2017. Disan: Directional self-attention network for rnn/cnn-free language understanding. arXiv preprint arXiv:1709.04696. [find](https://scholar.google.com/scholar?q=Tao+Shen%2C+Tianyi+Zhou%2C+Guodong+Long%2C+Jing+Jiang%2C+Shirui+Pan%2C+and+Chengqi+Zhang%2E+2017%2E+Disan%3A+Directional+self%2Dattention+network+for+rnn%2Fcnn%2Dfree+language+understanding%2E+arXiv+preprint+arXiv%3A1709%2E04696)
15. Zhixing Tan, Mingxuan Wang, Jun Xie, Yidong Chen, and Xiaodong Shi. 2017. [Deep semantic role labeling with self-attention](https://md.research.elamur.ai/free/en/papers/tan-2017-deep-semantic-role-labeling-self/). arXiv preprint arXiv:1712.01586.
16. Tijmen Tieleman and Geoffrey Hinton. 2012. Lecture 6.5-rmsprop: Divide the gradient by a running average of its recent magnitude. COURSERA: Neural networks for machine learning, 4(2):26–31. [find](https://scholar.google.com/scholar?q=Tijmen+Tieleman+and+Geoffrey+Hinton%2E+2012%2E+Lecture+6%2E5%2Drmsprop%3A+Divide+the+gradient+by+a+running+average+of+its+recent+magnitude%2E+COURSERA%3A+Neural+networks+for+machine+learning%2C+4%282%29%3A26%E2%80%9331%2E)
17. Gokhan Tur, Dilek Hakkani-Tür, and Larry Heck. 2010. What is left to be understood in atis? In Spoken Language Technology Workshop (SLT), 2010 IEEE, pages 19–24. IEEE. [find](https://scholar.google.com/scholar?q=Gokhan+Tur%2C+Dilek+Hakkani%2DTu%CC%88r%2C+and+Larry+Heck%2E+2010%2E+What+is+left+to+be+understood+in+atis%3F+In+Spoken+Language+Technology+Workshop+%28SLT%29%2C+2010+IEEE%2C+pages+19%E2%80%9324%2E+IEEE%2E)
18. Congying Xia, Chenwei Zhang, Xiaohui Yan, Yi Chang, and Philip S Yu. 2018. [Zero-shot user intent detection via capsule neural networks](https://md.research.elamur.ai/free/en/papers/xia-2018-zero-shot-user-intent-detection/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 3090–3099.
19. Puyang Xu and Ruhi Sarikaya. 2013. Convolutional neural network based triangular crf for joint intent detection and slot filling. In ASRU, pages 78–83. IEEE. [find](https://scholar.google.com/scholar?q=Puyang+Xu+and+Ruhi+Sarikaya%2E+2013%2E+Convolutional+neural+network+based+triangular+crf+for+joint+intent+detection+and+slot+filling%2E+In+ASRU%2C+pages+78%E2%80%9383%2E+IEEE%2E)
20. Chenwei Zhang, Nan Du, Wei Fan, Yaliang Li, Chun-Ta Lu, and Philip S Yu. 2017. Bringing semantic structures to user intent detection in online medical queries. In IEEE Big Data, pages 1019–1026. [find](https://scholar.google.com/scholar?q=Chenwei+Zhang%2C+Nan+Du%2C+Wei+Fan%2C+Yaliang+Li%2C+Chun%2DTa+Lu%2C+and+Philip+S+Yu%2E+2017%2E+Bringing+semantic+structures+to+user+intent+detection+in+online+medical+queries%2E+In+IEEE+Big+Data%2C+pages+1019%E2%80%931026%2E)
21. Chenwei Zhang, Wei Fan, Nan Du, and Philip S Yu. 2016. Mining user intentions from medical queries: A neural network based heterogeneous jointly modeling approach. In WWW, pages 1373–1384. [find](https://scholar.google.com/scholar?q=Chenwei+Zhang%2C+Wei+Fan%2C+Nan+Du%2C+and+Philip+S+Yu%2E+2016%2E+Mining+user+intentions+from+medical+queries%3A+A+neural+network+based+heterogeneous+jointly+modeling+approach%2E+In+WWW%2C+pages+1373%E2%80%931384%2E)
22. Wei Zhao, Jianbo Ye, Min Yang, Zeyang Lei, Suofei Zhang, and Zhou Zhao. 2018. Investigating capsule networks with dynamic routing for text classification. arXiv preprint arXiv:1804.00538. [find](https://scholar.google.com/scholar?q=Wei+Zhao%2C+Jianbo+Ye%2C+Min+Yang%2C+Zeyang+Lei%2C+Suofei+Zhang%2C+and+Zhou+Zhao%2E+2018%2E+Investigating+capsule+networks+with+dynamic+routing+for+text+classification%2E+arXiv+preprint+arXiv%3A1804%2E00538%2E)

*Updated 24 October 2026*