---
title: "From Latents to Labels — Vuth 2026"
description: "Zero-shot NER is usually done by prompting LLMs, which is costly, sensitive to wording and hard to make consistent, or by probing dense hidden states…"
authors: ["Nakanyseth Vuth", "Gilles Sérasset", "Didier Schwab"]
year: 2026
venue: "Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026)"
source: "https://aclanthology.org/2026.starsem-conference.11/"
doi: "10.18653/v1/2026.starsem-conference.11"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/vuth-2026-latents-labels-zero-shot-named/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/vuth-2026-latents-labels-zero-shot-named/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/) / From Latents to Labels: Zero-Shot Named Entity Recognition using Sparse Autoencoder Features

# From Latents to Labels: Zero-Shot Named Entity Recognition using Sparse Autoencoder Features

**Authors:** Nakanyseth Vuth, [Gilles Sérasset](https://md.research.elamur.ai/free/en/authors/gilles-serasset/), [Didier Schwab](https://md.research.elamur.ai/free/en/authors/didier-schwab/) · **Year:** 2026 · **Venue:** Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (\*SEM 2026) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.starsem-conference.11/) · [PDF](https://aclanthology.org/2026.starsem-conference.11.pdf) · [DOI 10.18653/v1/2026.starsem-conference.11](https://doi.org/10.18653/v1/2026.starsem-conference.11) · [OpenAlex](https://openalex.org/W7166885680)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14)

## Summary
Zero-shot NER is usually done by prompting LLMs, which is costly, sensitive to wording and hard to make consistent, or by probing dense hidden states, where polysemantic neurons make supervision necessary. The authors propose SAE-NER, which treats entity typing as detecting interpretable features in a Sparse Autoencoder. Candidate spans come from noun phrases found by a constituency parser plus a class-agnostic mention detector. Token-level SAE activations are pooled per span using norm-based weights and a log transform. Using cached activations from a gold training set, each feature receives a precision estimate per entity type, which yields a sparse feature-to-label matrix. At test time, spans are labelled by weighted evidence, and a confidence threshold rejects weak predictions. No classifier is trained and no prompt is used. Across general and biomedical corpora, the method beats trained probes, with the largest gains in the biomedical domain. Its output can also serve as silver data in cold-start settings. Controlled noise experiments indicate that missed entities hurt more than boundary errors or false positives.

## Key points
- The SAEs come from Lieberum et al. (2024) and were trained on Gemma-2-2B, Gemma-2-9B and Gemma-2-9B-IT:
  - **9B models:** layers 9/20/31, widths 16k and 131k.
  - **2B model:** layers 5/12/19, widths 16k, 65k and 1M.
- Candidate spans are generated by two tools:
  - the Berkeley Neural Parser running inside spaCy;
  - the Maverick (SapienzaNLP) mention detector.
- Features that fire too often (above 2% on NeelNanda/pile-10k) are discarded before feature-to-label mapping.
- Evaluation uses a coarse-grained general-domain corpus and AnEM, a biomedical corpus with 11 fine-grained anatomical types. SAE-NER outperforms probing classifiers by up to +20 F1 in the biomedical domain.
- The authors list several limitations:
  - only English and only the Gemma-2 SAEs are tested;
  - feature mapping still requires gold annotations;
  - heuristic span generation limits recall;
  - exact-match F1 may understate the method's quality.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{vuth2026latents,
  title = {{From Latents to Labels: Zero-Shot Named Entity Recognition using Sparse Autoencoder Features}},
  author = {Vuth, Nakanyseth and Sérasset, Gilles and Schwab, Didier},
  booktitle = {Proceedings of the 15th Joint Conference on Lexical and Computational Semantics (*SEM 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.starsem-conference.11},
  url = {https://doi.org/10.18653/v1/2026.starsem-conference.11}
}
```

- [← previous From Insights to Impact: Actionable Interpretability for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/sarti-2026-insights-impact-actionable-interpretability-neural/)
- [next → From Lemmatization to Legal Terminology: Assessing an Hybrid Pipeline on Justinian's Digest](https://md.research.elamur.ai/free/en/papers/marongiu-2026-lemmatization-legal-terminology-assessing-hybrid/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/)

## References

19 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, Robert Lasenby, Yifan Wu, Shauna Kravec, Nicholas Schiefer, Tim Maxwell, Nicholas Joseph, Zac Hatfield-Dodds, Alex Tamkin, Karina Nguyen, Brayden McLean, Josiah… [find](https://scholar.google.com/scholar?q=Trenton+Bricken%2C+Adly+Templeton%2C+Joshua+Batson%2C+Brian+Chen%2C+Adam+Jermyn%2C+Tom+Conerly%2C+Nick+Turner%2C+Cem+Anil%2C+Carson+Denison%2C+Amanda+Askell%2C+Robert+Lasenby%2C+Yifan+Wu%2C+Shauna+Kravec%2C+Nicholas+Schiefer%2C+)
2. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, T. J. Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeff Wu, Clemens Winter, C… [find](https://scholar.google.com/scholar?q=Tom+B%2E+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+Sandhini+Agarwal%2C+Ariel+Herbert%2DVoss%2C+Gretche)
3. Tomoko Ohta, Sampo Pyysalo, Jun’ichi Tsujii, and Sophia Ananiadou. 2012. Open-domain anatomical entity mention detection. In Proceedings of the Workshop on Detecting Structure in Scholarly Discourse, pages 27–36, Jeju Island, Korea. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Tomoko+Ohta%2C+Sampo+Pyysalo%2C+Jun%E2%80%99ichi+Tsujii%2C+and+Sophia+Ananiadou%2E+2012%2E+Open%2Ddomain+anatomical+entity+mention+detection%2E+In+Proceedings+of+the+Workshop+on+Detecting+Structure+in+Scholarly+Discourse%2C+)
4. Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. 2023. Sparse autoencoders find highly interpretable features in language models. ArXiv, abs/2309.08600. [find](https://scholar.google.com/scholar?q=Hoagy+Cunningham%2C+Aidan+Ewart%2C+Logan+Riggs%2C+Robert+Huben%2C+and+Lee+Sharkey%2E+2023%2E+Sparse+autoencoders+find+highly+interpretable+features+in+language+models%2E+ArXiv%2C+abs%2F2309%2E08600%2E)
5. Senthooran Rajamanoharan, Tom Lieberum, Nicolas Sonnerat, Arthur Conmy, Vikrant Varma, János Kramár, and Neel Nanda. 2024. Jumping ahead: Improving reconstruction fidelity with jumprelu sparse autoencoders. ArXiv, abs/2407.14435. — [arXiv:2407.14435](https://arxiv.org/abs/2407.14435) [find](https://scholar.google.com/scholar?q=Senthooran+Rajamanoharan%2C+Tom+Lieberum%2C+Nicolas+Sonnerat%2C+Arthur+Conmy%2C+Vikrant+Varma%2C+J%C3%A1nos+Kram%C3%A1r%2C+and+Neel+Nanda%2E+2024%2E+Jumping+ahead%3A+Improving+reconstruction+fidelity+with+jumprelu+sparse+autoenc)
6. Ning Ding, Guangwei Xu, Yulin Chen, Xiaobin Wang, Xu Han, Pengjun Xie, Haitao Zheng, and Zhiyuan Liu. 2021. Few-NERD: A few-shot named entity recognition dataset. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on… [find](https://scholar.google.com/scholar?q=Ning+Ding%2C+Guangwei+Xu%2C+Yulin+Chen%2C+Xiaobin+Wang%2C+Xu+Han%2C+Pengjun+Xie%2C+Haitao+Zheng%2C+and+Zhiyuan+Liu%2E+2021%2E+Few%2DNERD%3A+A+few%2Dshot+named+entity+recognition+dataset%2E+In+Proceedings+of+the+59th+Annual+Mee) [our card](https://md.research.elamur.ai/free/en/papers/ding-2021-few-nerd-few-shot-named/)
7. Yongliang Shen, Zeqi Tan, Shuhui Wu, Wenqi Zhang, Rongsheng Zhang, Yadong Xi, Weiming Lu, and Yueting Zhuang. 2023. PromptNER: Prompt locating and typing for named entity recognition. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),… [find](https://scholar.google.com/scholar?q=Yongliang+Shen%2C+Zeqi+Tan%2C+Shuhui+Wu%2C+Wenqi+Zhang%2C+Rongsheng+Zhang%2C+Yadong+Xi%2C+Weiming+Lu%2C+and+Yueting+Zhuang%2E+2023%2E+PromptNER%3A+Prompt+locating+and+typing+for+named+entity+recognition%2E+In+Proceedings+o)
8. Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, Roger Grosse, Sam McCandlish, Jared Kaplan, Dario Amodei, Martin Wattenberg, and Christopher Olah. 2022. Toy models of superposition. Transformer… [find](https://scholar.google.com/scholar?q=Nelson+Elhage%2C+Tristan+Hume%2C+Catherine+Olsson%2C+Nicholas+Schiefer%2C+Tom+Henighan%2C+Shauna+Kravec%2C+Zac+Hatfield%2DDodds%2C+Robert+Lasenby%2C+Dawn+Drain%2C+Carol+Chen%2C+Roger+Grosse%2C+Sam+McCandlish%2C+Jared+Kaplan%2C+D)
9. Leo Gao, Tom Dupr’e la Tour, Henk Tillman, Gabriel Goh, Rajan Troll, Alec Radford, Ilya Sutskever, Jan Leike, and Jeffrey Wu. 2024. Scaling and evaluating sparse autoencoders. ArXiv, abs/2406.04093. [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Tom+Dupr%E2%80%99e+la+Tour%2C+Henk+Tillman%2C+Gabriel+Goh%2C+Rajan+Troll%2C+Alec+Radford%2C+Ilya+Sutskever%2C+Jan+Leike%2C+and+Jeffrey+Wu%2E+2024%2E+Scaling+and+evaluating+sparse+autoencoders%2E+ArXiv%2C+abs%2F2406%2E04093%2E)
10. John Hewitt and Christopher D. Manning. 2019. A structural probe for finding syntax in word representations. In North American Chapter of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=John+Hewitt+and+Christopher+D%2E+Manning%2E+2019%2E+A+structural+probe+for+finding+syntax+in+word+representations%2E+In+North+American+Chapter+of+the+Association+for+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/en/papers/hewitt-2019-structural-probe-finding-syntax-word/)
11. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Yejin Bang, Delong Chen, Wenliang Dai, Andrea Madotto, and Pascale Fung. 2022. Survey of hallucination in natural language generation. ACM Computing Surveys, 55:1 – 38. [find](https://scholar.google.com/scholar?q=Ziwei+Ji%2C+Nayeon+Lee%2C+Rita+Frieske%2C+Tiezheng+Yu%2C+Dan+Su%2C+Yan+Xu%2C+Etsuko+Ishii%2C+Yejin+Bang%2C+Delong+Chen%2C+Wenliang+Dai%2C+Andrea+Madotto%2C+and+Pascale+Fung%2E+2022%2E+Survey+of+hallucination+in+natural+languag) [our card](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/)
12. Yangming Li, Lemao Liu, and Shuming Shi. 2020. Empirical analysis of unlabeled entity problem in named entity recognition. ArXiv, abs/2012.05426. — [arXiv:2012.05426](https://arxiv.org/abs/2012.05426) [find](https://scholar.google.com/scholar?q=Yangming+Li%2C+Lemao+Liu%2C+and+Shuming+Shi%2E+2020%2E+Empirical+analysis+of+unlabeled+entity+problem+in+named+entity+recognition%2E+ArXiv%2C+abs%2F2012%2E05426%2E+%E2%80%94+%5BarXiv%3A2012%2E05426%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2012%2E05426%29)
13. Tom Lieberum, Senthooran Rajamanoharan, Arthur Conmy, Lewis Smith, Nicolas Sonnerat, Vikrant Varma, J’anos Kram’ar, Anca Dragan, Rohin Shah, and Neel Nanda. 2024. Gemma scope: Open sparse autoencoders everywhere all at once on gemma 2. ArXiv, abs/2408.05147. [find](https://scholar.google.com/scholar?q=Tom+Lieberum%2C+Senthooran+Rajamanoharan%2C+Arthur+Conmy%2C+Lewis+Smith%2C+Nicolas+Sonnerat%2C+Vikrant+Varma%2C+J%E2%80%99anos+Kram%E2%80%99ar%2C+Anca+Dragan%2C+Rohin+Shah%2C+and+Neel+Nanda%2E+2024%2E+Gemma+scope%3A+Open+sparse+autoencoders)
14. Stephen Mayhew, Snigdha Chaturvedi, Chen-Tse Tsai, and Dan Roth. 2019. Named entity recognition with Jake Snell, Kevin Swersky, and Richard S. Zemel. 2017. Prototypical networks for few-shot learning. In Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Stephen+Mayhew%2C+Snigdha+Chaturvedi%2C+Chen%2DTse+Tsai%2C+and+Dan+Roth%2E+2019%2E+Named+entity+recognition+with+Jake+Snell%2C+Kevin+Swersky%2C+and+Richard+S%2E+Zemel%2E+2017%2E+Prototypical+networks+for+few%2Dshot+learning%2E)
15. Qingyu Tan, Lu Xu, Lidong Bing, Hwee Tou Ng, and Sharifah Mahani Aljunied. 2022. Revisiting docred – addressing the false negative problem in relation extraction. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Qingyu+Tan%2C+Lu+Xu%2C+Lidong+Bing%2C+Hwee+Tou+Ng%2C+and+Sharifah+Mahani+Aljunied%2E+2022%2E+Revisiting+docred+%E2%80%93+addressing+the+false+negative+problem+in+relation+extraction%2E+In+Proceedings+of+EMNLP%2E)
16. Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. Bert rediscovers the classical nlp pipeline. In Annual Meeting of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Ian+Tenney%2C+Dipanjan+Das%2C+and+Ellie+Pavlick%2E+2019%2E+Bert+rediscovers+the+classical+nlp+pipeline%2E+In+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2E) [our card](https://md.research.elamur.ai/free/en/papers/tenney-2019-bert-rediscovers-classical-nlp-pipeline/)
17. Hugo Touvron, Louis Martin, Kevin R. Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Niko lay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Daniel M. Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Bri… [find](https://scholar.google.com/scholar?q=Hugo+Touvron%2C+Louis+Martin%2C+Kevin+R%2E+Stone%2C+Peter+Albert%2C+Amjad+Almahairi%2C+Yasmine+Babaei%2C+Niko+lay+Bashlykov%2C+Soumya+Batra%2C+Prajjwal+Bhargava%2C+Shruti+Bhosale%2C+Daniel+M%2E+Bikel%2C+Lukas+Blecher%2C+Cristian)
18. Shuhe Wang, Xiaofei Sun, Xiaoya Li, Rongbin Ouyang, Fei Wu, Tianwei Zhang, Jiwei Li, Guoyin Wang, and Chen Guo. 2025. GPT-NER: Named entity recognition via large language models. In Findings of the Association for Computational Linguistics: NAACL 2025, pages 4257–4275, Albuquerque, New Mexico. Assoc… [find](https://scholar.google.com/scholar?q=Shuhe+Wang%2C+Xiaofei+Sun%2C+Xiaoya+Li%2C+Rongbin+Ouyang%2C+Fei+Wu%2C+Tianwei+Zhang%2C+Jiwei+Li%2C+Guoyin+Wang%2C+and+Chen+Guo%2E+2025%2E+GPT%2DNER%3A+Named+entity+recognition+via+large+language+models%2E+In+Findings+of+the+As) [our card](https://md.research.elamur.ai/free/en/papers/wang-2025-gpt-ner-named-entity-recognition/)
19. Tony Zhao, Eric Wallace, Shi Feng, Dan Klein, and Sameer Singh. 2021. Calibrate before use: Improving few-shot performance of language models. In International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Tony+Zhao%2C+Eric+Wallace%2C+Shi+Feng%2C+Dan+Klein%2C+and+Sameer+Singh%2E+2021%2E+Calibrate+before+use%3A+Improving+few%2Dshot+performance+of+language+models%2E+In+International+Conference+on+Machine+Learning%2E) [our card](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/)