---
title: "Deep Unknown Intent Detection with Margin Loss — Lin 2019"
description: "It replaces the usual softmax loss with large margin cosine loss (LMCL), which… 14.9 points. 21 references, cited by 63 papers here."
authors: ["Ting-En Lin", "Hua Xu"]
year: 2019
venue: "Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics"
source: "https://aclanthology.org/P19-1548/"
doi: "10.18653/v1/p19-1548"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/lin-2019-deep-unknown-intent-detection-margin/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lin-2019-deep-unknown-intent-detection-margin/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/) / Deep Unknown Intent Detection with Margin Loss

# Deep Unknown Intent Detection with Margin Loss

**Authors:** Ting-En Lin and [Hua Xu](https://md.research.elamur.ai/free/en/authors/hua-xu/) · **Year:** 2019 · **Venue:** [Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2019](https://md.research.elamur.ai/free/en/venues/acl/2019/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/P19-1548/) · [PDF](https://aclanthology.org/P19-1548.pdf) · [DOI 10.18653/v1/p19-1548](https://doi.org/10.18653/v1/p19-1548) · [OpenAlex](https://openalex.org/W2952409498)

**Citations:** 142 citations (OpenAlex, 2026-10-14) · 63 in our library, of the 141,840 papers whose bibliographies we read

## Summary
Dialogue systems need to recognize user requests whose intent class never appeared in training. Such requests can point to new business needs and guide developers, but there are no examples of them and their number is unknown. Lin and Xu frame this as open-world classification, where all unseen intents are merged into one extra "unknown" class. Their two-stage approach first trains a BiLSTM intent classifier on known intents only. It replaces the usual softmax loss with large margin cosine loss (LMCL), which normalizes features and class weights and enforces an angular margin. As a result, vectors of the same class cluster tightly and different classes move apart. In the second stage, these sentence vectors go to local outlier factor (LOF), a density-based novelty detector that flags utterances lying in sparse regions. On SNIPS and ATIS, with 25–75% of classes kept as known, the method beat MSP, DOC and a softmax-trained LOF ablation in every setting. The work shows that shaping the feature space, not just the decision rule, matters for spotting novel intents.

## Key points
- Datasets: SNIPS has 7 balanced intents across several domains and 13,084 training utterances. ATIS has 18 imbalanced intents, all in the flight domain, and 4,978 training utterances.
- Setup:
  - Known classes are drawn by weighted random sampling (25%, 50% or 75% of classes), and the rest are held out as unknown.
  - Embeddings are initialized with GloVe, and the BiLSTM output dimension is 128.
  - LMCL uses s = 30 and m = 0.35.
  - Results are macro F1 averaged over 10 runs.
- On SNIPS, LOF (LMCL) reached 79.2 / 84.1 / 78.8 macro F1, which is 6.7, 16.2 and 14.9 points above DOC.
- On ATIS, gains were smaller (69.6 / 63.4 / 39.6). All methods degrade as more classes become known, because the intents there are semantically close to each other.
- t-SNE plots show that LMCL features are more compact within classes and better separated between classes than softmax features, which helps density-based detection.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 137 works · 83,262 citations
- [State Key Laboratory of Intelligent Technology and Systems](https://md.research.elamur.ai/free/en/labs/state-key-laboratory-of-intelligent-technology-and-systems/) — Research institute, Beijing, China · 21 works · 9,641 citations
- [Beijing National Research Center for Information Science and Technology](https://md.research.elamur.ai/free/en/labs/beijing-national-research-center-for-information-science-and-technology/) — Research institute, Beijing, China · 21 works · 5,843 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: Ting-En Lin and Hua Xu. 2019. Deep Unknown Intent Detection with Margin Loss. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{lin2019deep,
  title = {{Deep Unknown Intent Detection with Margin Loss}},
  author = {Ting-En Lin and Hua Xu},
  booktitle = {Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics},
  year = {2019},
  doi = {10.18653/v1/p19-1548},
  url = {https://doi.org/10.18653/v1/p19-1548}
}
```

- [← previous Proactive Human-Machine Conversation with Explicit Conversation Goal](https://md.research.elamur.ai/free/en/papers/wu-2019-proactive-human-machine-conversation-explicit/)
- [next → Neural Arabic Question Answering](https://md.research.elamur.ai/free/en/papers/mozannar-2019-neural-arabic-question-answering/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/)

## References

21 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abhijit Bendale and Terrance E. Boult. 2016. Towards open set deep networks. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 1563–1572. [find](https://scholar.google.com/scholar?q=Abhijit+Bendale+and+Terrance+E%2E+Boult%2E+2016%2E+Towards+open+set+deep+networks%2E+2016+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition+%28CVPR%29%2C+pages+1563%E2%80%931572%2E)
2. Markus M Breunig, Hans-Peter Kriegel, Raymond T Ng, and Jörg Sander. 2000. Lof: identifying density-based local outliers. In ACM sigmod record, volume 29, pages 93–104. [find](https://scholar.google.com/scholar?q=Markus+M+Breunig%2C+Hans%2DPeter+Kriegel%2C+Raymond+T+Ng%2C+and+Jo%CC%88rg+Sander%2E+2000%2E+Lof%3A+identifying+density%2Dbased+local+outliers%2E+In+ACM+sigmod+record%2C+volume+29%2C+pages+93%E2%80%93104%2E)
3. Tomas Brychcin and Pavel Král. [Unsupervised dialogue act induction using gaussian mixtures](https://md.research.elamur.ai/free/en/papers/brychcin-2017-unsupervised-dialogue-act-induction-gaussian/). In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2017, pages 485–490.
4. Geli Fei and Bing Liu. 2016. Breaking the closed world assumption in text classification. In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 506–514. [find](https://scholar.google.com/scholar?q=Geli+Fei+and+Bing+Liu%2E+2016%2E+Breaking+the+closed+world+assumption+in+text+classification%2E+In+Proceedings+of+the+2016+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Lingu)
5. Chih-Wen Goo, Guang Gao, Yun-Kai Hsu, Chih-Li Huo, Tsung-Chieh Chen, Keng-Wei Hsu, and Yun-Nung Chen. 2018. [Slot-gated modeling for joint slot filling and intent prediction](https://md.research.elamur.ai/free/en/papers/goo-2018-slot-gated-modeling-joint-slot/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human La…
6. Dan Hendrycks and Kevin Gimpel. 2016. A baseline for detecting misclassified and out-of-distribution examples in neural networks. arXiv preprint arXiv:1610.02136. [find](https://scholar.google.com/scholar?q=Dan+Hendrycks+and+Kevin+Gimpel%2E+2016%2E+A+baseline+for+detecting+misclassified+and+out%2Dof%2Ddistribution+examples+in+neural+networks%2E+arXiv+preprint+arXiv%3A1610%2E02136%2E)
7. Joo-Kyung Kim and Young-Bum Kim. 2018. Joint learning of domain classification and out-of-domain detection with dynamic class weighting for satisficing false acceptance rates. In Interspeech 2018, 19th Annual Conference of the International Speech Communication Association, Hyderabad, India, 2-6 Sep… [find](https://scholar.google.com/scholar?q=Joo%2DKyung+Kim+and+Young%2DBum+Kim%2E+2018%2E+Joint+learning+of+domain+classification+and+out%2Dof%2Ddomain+detection+with+dynamic+class+weighting+for+satisficing+false+acceptance+rates%2E+In+Interspeech+2018%2C+19t)
8. Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. 2017. Sphereface: Deep hypersphere embedding for face recognition. In 2017 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, Honolulu, HI, USA, July 21-26, 2017, pages 6738–6746. [find](https://scholar.google.com/scholar?q=Weiyang+Liu%2C+Yandong+Wen%2C+Zhiding+Yu%2C+Ming+Li%2C+Bhiksha+Raj%2C+and+Le+Song%2E+2017%2E+Sphereface%3A+Deep+hypersphere+embedding+for+face+recognition%2E+In+2017+IEEE+Conference+on+Computer+Vision+and+Pattern+Recog)
9. Weiyang Liu, Yandong Wen, Zhiding Yu, and Meng Yang. 2016. Large-margin softmax loss for convolutional neural networks. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, pages 507–516. [find](https://scholar.google.com/scholar?q=Weiyang+Liu%2C+Yandong+Wen%2C+Zhiding+Yu%2C+and+Meng+Yang%2E+2016%2E+Large%2Dmargin+softmax+loss+for+convolutional+neural+networks%2E+In+Proceedings+of+the+33nd+International+Conference+on+Machine+Learning%2C+ICML+20)
10. Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-sne. Journal of machine learning research, 9(Nov):2579–2605. [find](https://scholar.google.com/scholar?q=Laurens+van+der+Maaten+and+Geoffrey+Hinton%2E+2008%2E+Visualizing+data+using+t%2Dsne%2E+Journal+of+machine+learning+research%2C+9%28Nov%29%3A2579%E2%80%932605%2E)
11. Grégoire Mesnil, Yann Dauphin, Kaisheng Yao, Yoshua Bengio, Li Deng, Dilek Hakkani-Tur, Xiaodong He, Larry Heck, Gokhan Tur, Dong Yu, et al. 2015. Using recurrent neural networks for slot filling in spoken language understanding. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 23(3… [find](https://scholar.google.com/scholar?q=Gre%CC%81goire+Mesnil%2C+Yann+Dauphin%2C+Kaisheng+Yao%2C+Yoshua+Bengio%2C+Li+Deng%2C+Dilek+Hakkani%2DTur%2C+Xiaodong+He%2C+Larry+Heck%2C+Gokhan+Tur%2C+Dong+Yu%2C+et+al%2E+2015%2E+Using+recurrent+neural+networks+for+slot+filling+in+)
12. Eric Nalisnick, Akihiro Matsukawa, Yee Whye Teh, Dilan Gorur, and Balaji Lakshminarayanan. 2019. Do deep generative models know what they don’t know? In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Eric+Nalisnick%2C+Akihiro+Matsukawa%2C+Yee+Whye+Teh%2C+Dilan+Gorur%2C+and+Balaji+Lakshminarayanan%2E+2019%2E+Do+deep+generative+models+know+what+they+don%E2%80%99t+know%3F+In+International+Conference+on+Learning+Representa)
13. Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In EMNLP, volume 14, pages 1532– 1543.
14. Rajeev Ranjan, Carlos D. Castillo, and Rama Chellappa. 2017. L2-constrained softmax loss for discriminative face verification. CoRR, abs/1703.09507. — [arXiv:1703.09507](https://arxiv.org/abs/1703.09507) [find](https://scholar.google.com/scholar?q=Rajeev+Ranjan%2C+Carlos+D%2E+Castillo%2C+and+Rama+Chellappa%2E+2017%2E+L2%2Dconstrained+softmax+loss+for+discriminative+face+verification%2E+CoRR%2C+abs%2F1703%2E09507%2E+%E2%80%94+%5BarXiv%3A1703%2E09507%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1703%2E095)
15. Walter J. Scheirer, Anderson Rocha, Archana Sapkota, and Terrance E. Boult. 2013. Toward open set recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence, 35:1757–1772. [find](https://scholar.google.com/scholar?q=Walter+J%2E+Scheirer%2C+Anderson+Rocha%2C+Archana+Sapkota%2C+and+Terrance+E%2E+Boult%2E+2013%2E+Toward+open+set+recognition%2E+IEEE+Transactions+on+Pattern+Analysis+and+Machine+Intelligence%2C+35%3A1757%E2%80%931772%2E)
16. Lei Shu, Hu Xu, and Bing Liu. 2017. Doc: Deep open classification of text documents. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, EMNLP 2017, Copenhagen, Denmark, September 9-11, 2017, pages 2911–2916. [find](https://scholar.google.com/scholar?q=Lei+Shu%2C+Hu+Xu%2C+and+Bing+Liu%2E+2017%2E+Doc%3A+Deep+open+classification+of+text+documents%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+EMNLP+2017%2C+Copenhagen%2C+)
17. Gökhan Tür, Dilek Z. Hakkani-Tür, and Larry P. Heck. 2010. What is left to be understood in atis? 2010 IEEE Spoken Language Technology Workshop, pages 19–24. [find](https://scholar.google.com/scholar?q=Go%CC%88khan+Tu%CC%88r%2C+Dilek+Z%2E+Hakkani%2DTu%CC%88r%2C+and+Larry+P%2E+Heck%2E+2010%2E+What+is+left+to+be+understood+in+atis%3F+2010+IEEE+Spoken+Language+Technology+Workshop%2C+pages+19%E2%80%9324%2E)
18. Feng Wang, Jian Cheng, Weiyang Liu, and Haijun Liu. 2018a. Additive margin softmax for face verification. IEEE Signal Processing Letters, 25(7):926– 930. [find](https://scholar.google.com/scholar?q=Feng+Wang%2C+Jian+Cheng%2C+Weiyang+Liu%2C+and+Haijun+Liu%2E+2018a%2E+Additive+margin+softmax+for+face+verification%2E+IEEE+Signal+Processing+Letters%2C+25%287%29%3A926%E2%80%93+930%2E)
19. Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. 2018b. Cosface: Large margin cosine loss for deep face recognition. In 2018 IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2018, Salt Lake City, UT, USA, June 18-22, 2018, pages 5265–52… [find](https://scholar.google.com/scholar?q=Hao+Wang%2C+Yitong+Wang%2C+Zheng+Zhou%2C+Xing+Ji%2C+Dihong+Gong%2C+Jingchao+Zhou%2C+Zhifeng+Li%2C+and+Wei+Liu%2E+2018b%2E+Cosface%3A+Large+margin+cosine+loss+for+deep+face+recognition%2E+In+2018+IEEE+Conference+on+Computer)
20. Yu Wang, Yilin Shen, and Hongxia Jin. 2018c. A bi-model based rnn semantic frame parsing model for intent detection and slot filling. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2018, pag… [find](https://scholar.google.com/scholar?q=Yu+Wang%2C+Yilin+Shen%2C+and+Hongxia+Jin%2E+2018c%2E+A+bi%2Dmodel+based+rnn+semantic+frame+parsing+model+for+intent+detection+and+slot+filling%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Chapte)
21. Yang Yu, Wei-Yang Qu, Nan Li, and Zimin Guo. 2017. Open-category classification by adversarial sample generation. In Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI 2017, pages 3357–3363. [find](https://scholar.google.com/scholar?q=Yang+Yu%2C+Wei%2DYang+Qu%2C+Nan+Li%2C+and+Zimin+Guo%2E+2017%2E+Open%2Dcategory+classification+by+adversarial+sample+generation%2E+In+Proceedings+of+the+Twenty%2DSixth+International+Joint+Conference+on+Artificial+Intell)

*Updated 24 October 2026*