---
title: "Lexical Familiarity Predicts Processing Depth for Nonliteral"
description: "The paper asks whether large language models handle different kinds of nonliteral language at different depths, and what… 28 references."
authors: ["Lang-Ching Yeh", "Yu-Chieh Wang", "Shu-Kai Hsieh"]
year: 2026
venue: "Proceedings of the 6th Workshop on Trustworthy NLP (TrustNLP 2026)"
source: "https://aclanthology.org/2026.trustnlp-main.32/"
doi: "10.18653/v1/2026.trustnlp-main.32"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/yeh-2026-lexical-familiarity-predicts-processing-depth/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yeh-2026-lexical-familiarity-predicts-processing-depth/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/39/) / Lexical Familiarity Predicts Processing Depth for Nonliteral Language in Large Language Models

# Lexical Familiarity Predicts Processing Depth for Nonliteral Language in Large Language Models

**Authors:** [Lang-Ching Yeh](https://md.research.elamur.ai/free/en/authors/lang-ching-yeh/), [Yu-Chieh Wang](https://md.research.elamur.ai/free/en/authors/yu-chieh-wang/) and [Shu-Kai Hsieh](https://md.research.elamur.ai/free/en/authors/shu-kai-hsieh/) · **Year:** 2026 · **Venue:** [Proceedings of the 6th Workshop on Trustworthy NLP (TrustNLP 2026)](https://md.research.elamur.ai/free/en/venues/trustnlp/) ([Workshop on Trustworthy NLP 2026](https://md.research.elamur.ai/free/en/venues/trustnlp/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.trustnlp-main.32/) · [PDF](https://aclanthology.org/2026.trustnlp-main.32.pdf) · [DOI 10.18653/v1/2026.trustnlp-main.32](https://doi.org/10.18653/v1/2026.trustnlp-main.32) · [OpenAlex](https://openalex.org/W7166851144)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper asks whether large language models handle different kinds of nonliteral language at different depths, and what principle governs this. The authors built minimal pairs, each matching a figurative sentence with a literal equivalent, across five categories: semantic-shift slang, neologisms, constructional slang, metaphors (from FLUTE) and idioms (from the PIE corpus). They added two PAWS-based controls: identical pairs and literal paraphrases. For every pair, they projected token representations from all 48 layers of Gemma-3-12B-IT into Gemma Scope 2 SAE feature space and tracked cosine divergence layer by layer.

The layer of peak divergence is determined by the extent of the model's existing word-level familiarity with an expression, rather than by its figurative category. Idioms diverge at once, expressions built from familiar words peak in early layers, and novel coined words peak much deeper. The pattern also appears in base models, so it is not a product of alignment tuning. The work matters for robustness and fairness, since unfamiliar slang may be processed less reliably. It also flags a confound in SAE-based interpretability.

## Key points
- **Data:**
  - Semantic-shift slang: 1,002 Gen-Z pairs from Kaggle, checked against Green's Dictionary of Slang and Urban Dictionary.
  - Neologisms: 1,000 items drawn from Urban Dictionary and HuggingFace data, with LLM-generated literal counterparts that were manually reviewed.
  - Constructional slang: only 37 items, treated as exploratory.
  - Metaphors: 625 pairs from FLUTE.
  - Idioms: 823 pairs from PIE.
  - Controls: 1,000 identical pairs and 1,000 PAWS paraphrase pairs.
- **Peak layers:**
  - Idioms: L1 (0.830).
  - Paraphrase noise floor: L3 (0.428).
  - Constructional slang: L7.
  - Metaphors: L8.
  - Semantic-shift slang: L9.
  - Neologisms: L41 (0.857), activating roughly three times as many unique features as other categories.
- **Residual analysis:** After subtracting the paraphrase baseline, only the two extremes stay clearly above noise (idioms 0.409, neologisms 0.505). This supports three tiers: entrenched retrieval, known-word reanalysis and novel-word construction.
- **SAE confound:** Raw SAE feature counts peak at L24–25 for every condition, including the controls. This reflects how dense activations are at those layers, not linguistic processing.
- **Replication:** The peak structure holds for four of five categories in Gemma-3-12B-PT. The endpoint ordering is preserved in a Qwen base model.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [National Taiwan University](https://md.research.elamur.ai/free/en/universities/national-taiwan-university/) — University, Taipei, Taiwan · 32 works · 2,116 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🧷 Idioms & multiword expressions](https://md.research.elamur.ai/free/en/topics/multiword-expressions/) · 222 works

## Cite

Cite as: Lang-Ching Yeh, Yu-Chieh Wang and Shu-Kai Hsieh. 2026. Lexical Familiarity Predicts Processing Depth for Nonliteral Language in Large Language Models. Proceedings of the 6th Workshop on Trustworthy NLP (TrustNLP 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{yeh2026lexical,
  title = {{Lexical Familiarity Predicts Processing Depth for Nonliteral Language in Large Language Models}},
  author = {Lang-Ching Yeh, Yu-Chieh Wang and Shu-Kai Hsieh},
  booktitle = {Proceedings of the 6th Workshop on Trustworthy NLP (TrustNLP 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.trustnlp-main.32},
  url = {https://doi.org/10.18653/v1/2026.trustnlp-main.32}
}
```

- [← previous Lexical Conditioning of Model's Distribution through Uncertainty-gated Soft-Mixing of Probabilities](https://md.research.elamur.ai/free/en/papers/papucci-2026-lexical-conditioning-model-s-distribution/)
- [next → Lexical Innovation in Business Colour Idioms: Evidence from Large Language Models in Five Languages](https://md.research.elamur.ai/free/en/papers/oleskeviciene-2026-lexical-innovation-business-colour-idioms/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/39/)

## References

28 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Trenton Bricken, Adly Templeton, Joshua Batson, Brian Chen, Adam Jermyn, Tom Conerly, Nick Turner, Cem Anil, Carson Denison, Amanda Askell, and 1 others. 2023. Towards monosemanticity: Decomposing language models with dictionary learning. Technical report, Anthropic. [find](https://scholar.google.com/scholar?q=Trenton+Bricken%2C+Adly+Templeton%2C+Joshua+Batson%2C+Brian+Chen%2C+Adam+Jermyn%2C+Tom+Conerly%2C+Nick+Turner%2C+Cem+Anil%2C+Carson+Denison%2C+Amanda+Askell%2C+and+1+others%2E+2023%2E+Towards+monosemanticity%3A+Decomposing+lan)
2. Tuhin Chakrabarty, Arkadiy Saakyan, Debanjan Ghosh, and Smaranda Muresan. 2022. [FLUTE: Figurative language understanding through textual explanations](https://md.research.elamur.ai/free/en/papers/chakrabarty-2022-flute-figurative-language-understanding-through/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 7139–7159. Association for Computational Linguistics…
3. Hoagy Cunningham, Aidan Ewart, Logan Riggs, Robert Huben, and Lee Sharkey. 2024. Sparse autoencoders find highly interpretable features in language models. In Proceedings of the Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Hoagy+Cunningham%2C+Aidan+Ewart%2C+Logan+Riggs%2C+Robert+Huben%2C+and+Lee+Sharkey%2E+2024%2E+Sparse+autoencoders+find+highly+interpretable+features+in+language+models%2E+In+Proceedings+of+the+Twelfth+International+)
4. Verna Dankers, Christopher Lucas, and Ivan Titov. 2022. [Can transformer be too compositional? analysing idiom processing in neural machine translation](https://md.research.elamur.ai/free/en/papers/dankers-2022-can-transformer-be-too-compositional/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3608–3626, Dublin, Ireland…
5. Nelson Elhage, Tristan Hume, Catherine Olsson, Nicholas Schiefer, Tom Henighan, Shauna Kravec, Zac Hatfield-Dodds, Robert Lasenby, Dawn Drain, Carol Chen, and 1 others. 2022. Toy models of superposition. Technical report, Anthropic. [find](https://scholar.google.com/scholar?q=Nelson+Elhage%2C+Tristan+Hume%2C+Catherine+Olsson%2C+Nicholas+Schiefer%2C+Tom+Henighan%2C+Shauna+Kravec%2C+Zac+Hatfield%2DDodds%2C+Robert+Lasenby%2C+Dawn+Drain%2C+Carol+Chen%2C+and+1+others%2E+2022%2E+Toy+models+of+superpositi)
6. Ranuga Disansa Gamage. 2025. genz-slang-pairs-1k. Kaggle Dataset, Revision 9728f69. [find](https://scholar.google.com/scholar?q=Ranuga+Disansa+Gamage%2E+2025%2E+genz%2Dslang%2Dpairs%2D1k%2E+Kaggle+Dataset%2C+Revision+9728f69%2E)
7. Gemma Team. 2025. Gemma 3 technical report. Technical report, Google DeepMind. [find](https://scholar.google.com/scholar?q=Gemma+Team%2E+2025%2E+Gemma+3+technical+report%2E+Technical+report%2C+Google+DeepMind%2E)
8. Jonathon Green. 2025. Green’s dictionary of slang. Accessed: 2025. [find](https://scholar.google.com/scholar?q=Jonathon+Green%2E+2025%2E+Green%E2%80%99s+dictionary+of+slang%2E+Accessed%3A+2025%2E)
9. John Hewitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 2733–2743. Association for Computational… [find](https://scholar.google.com/scholar?q=John+Hewitt+and+Percy+Liang%2E+2019%2E+Designing+and+interpreting+probes+with+control+tasks%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the+9th+Internati)
10. Ganesh Jawahar, Benoît Sagot, and Djamé Seddah. 2019. [What does BERT learn about the structure of language?](https://md.research.elamur.ai/free/en/papers/jawahar-2019-what-does-bert-learn-about/) In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 3651–3657. Association for Computational Linguistics.
11. Yi Jing, Zijun Yao, Hongzhu Guo, Lingxu Ran, Xiaozhi Wang, Lei Hou, and Juanzi Li. 2025. LinguaLens: Towards interpreting linguistic mechanisms of large language models via sparse auto-encoder. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 28232– 28… [find](https://scholar.google.com/scholar?q=Yi+Jing%2C+Zijun+Yao%2C+Hongzhu+Guo%2C+Lingxu+Ran%2C+Xiaozhi+Wang%2C+Lei+Hou%2C+and+Juanzi+Li%2E+2025%2E+LinguaLens%3A+Towards+interpreting+linguistic+mechanisms+of+large+language+models+via+sparse+auto%2Dencoder%2E+In+Pro)
12. Jisu Kim, Youngwoo Shin, Uiji Hwang, Jihun Choi, Richeng Xuan, and Taeuk Kim. 2025. [Memorization or reasoning? exploring the idiom understanding of LLMs](https://md.research.elamur.ai/free/en/papers/kim-2025-memorization-or-reasoning-exploring-idiom/). In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 21678–21699, Suzhou, China. Association for Compu…
13. Rohit Kulkarni. 2025. Urban dictionary words dataset. HuggingFace Dataset. [find](https://scholar.google.com/scholar?q=Rohit+Kulkarni%2E+2025%2E+Urban+dictionary+words+dataset%2E+HuggingFace+Dataset%2E)
14. Wen Lai, Viktor Hangya, and Alexander Fraser. 2024. Style-specific neurons for steering LLMs in text style transfer. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, pages 13427–13443. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Wen+Lai%2C+Viktor+Hangya%2C+and+Alexander+Fraser%2E+2024%2E+Style%2Dspecific+neurons+for+steering+LLMs+in+text+style+transfer%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Association+for+Computational+Ling)
15. LM-Lexicon. 2025. Lm-lexicon: Slang term list. HuggingFace Dataset. [find](https://scholar.google.com/scholar?q=LM%2DLexicon%2E+2025%2E+Lm%2Dlexicon%3A+Slang+term+list%2E+HuggingFace+Dataset%2E)
16. Callum McDougall, Arthur Conmy, János Kramár, Tom Lieberum, Senthooran Rajamanoharan, and Neel Nanda. 2025. Gemma scope 2: A technical report. Technical report, Google DeepMind. models. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics, pages 5701–5715. Assoc… [find](https://scholar.google.com/scholar?q=Callum+McDougall%2C+Arthur+Conmy%2C+J%C3%A1nos+Kram%C3%A1r%2C+Tom+Lieberum%2C+Senthooran+Rajamanoharan%2C+and+Neel+Nanda%2E+2025%2E+Gemma+scope+2%3A+A+technical+report%2E+Technical+report%2C+Google+DeepMind%2E+models%2E+In+Proceedings)
17. Soyoung Oh, Xinting Huang, Mathis Pink, Michael Hahn, and Vera Demberg. 2026. [Tug-of-war between idioms’ figurative and literal interpretations in llms](https://md.research.elamur.ai/free/en/papers/oh-2026-tug-war-between-idioms-figurative/). Preprint, arXiv:2506.01723.
18. Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. [BERT rediscovers the classical NLP pipeline](https://md.research.elamur.ai/free/en/papers/tenney-2019-bert-rediscovers-classical-nlp-pipeline/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4593– 4601. Association for Computational Linguistics.
19. Zhengqi Pei, Zhewei Sun, and Yang Xu. 2019. Slang detection and identification. In Proceedings of the 23rd Conference on Computational Natural Language Learning, pages 881–889. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zhengqi+Pei%2C+Zhewei+Sun%2C+and+Yang+Xu%2E+2019%2E+Slang+detection+and+identification%2E+In+Proceedings+of+the+23rd+Conference+on+Computational+Natural+Language+Learning%2C+pages+881%E2%80%93889%2E+Association+for+Computa)
20. Qwen Team. 2026a. [Qwen-Scope: Turning sparse features into development tools for large language models](https://md.research.elamur.ai/free/en/papers/deng-2026-qwen-scope-turning-sparse/).
21. Qwen Team. 2026b. Qwen3.5: Accelerating productivity with native multimodal agents. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+2026b%2E+Qwen3%2E5%3A+Accelerating+productivity+with+native+multimodal+agents%2E)
22. Anna Rogers, Olga Kovaleva, and Anna Rumshisky. 2020. [A primer in BERTology: What we know about how BERT works](https://md.research.elamur.ai/free/en/papers/rogers-2020-primer-bertology-what-we-know/). Transactions of the Association for Computational Linguistics, 8:842–866.
23. Zhewei Sun, Qian Hu, Rahul Gupta, Richard Zemel, and Yang Xu. 2024. Toward informal language processing: Knowledge of slang in large language models. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Vol… [find](https://scholar.google.com/scholar?q=Zhewei+Sun%2C+Qian+Hu%2C+Rahul+Gupta%2C+Richard+Zemel%2C+and+Yang+Xu%2E+2024%2E+Toward+informal+language+processing%3A+Knowledge+of+slang+in+large+language+models%2E+In+Proceedings+of+the+2024+Conference+of+the+North)
24. Zhewei Sun and Yang Xu. 2022. Tracing semantic variation in slang. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1299–1313. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zhewei+Sun+and+Yang+Xu%2E+2022%2E+Tracing+semantic+variation+in+slang%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+1299%E2%80%931313%2E+Association+for+Computati)
25. Debra A. Titone and Cynthia M. Connine. 1999. On the compositional and noncompositional nature of idiomatic expressions. Journal of Pragmatics, 31(12):1655–1674. Literal and Figurative Language. [find](https://scholar.google.com/scholar?q=Debra+A%2E+Titone+and+Cynthia+M%2E+Connine%2E+1999%2E+On+the+compositional+and+noncompositional+nature+of+idiomatic+expressions%2E+Journal+of+Pragmatics%2C+31%2812%29%3A1655%E2%80%931674%2E+Literal+and+Figurative+Language%2E)
26. Urban Dictionary. 2025. Urban dictionary. Accessed: 2025. [find](https://scholar.google.com/scholar?q=Urban+Dictionary%2E+2025%2E+Urban+dictionary%2E+Accessed%3A+2025%2E)
27. Yuan Zhang, Jason Baldridge, and Luheng He. 2019. [PAWS: Paraphrase adversaries from word scrambling](https://md.research.elamur.ai/free/en/papers/zhang-2019-paws-paraphrase-adversaries-word-scrambling/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1298–1308. Association for Computational Lingu…
28. Jianing Zhou, Hongyu Zheng, and Suma Bhat. 2021. PIE: A parallel idiomatic expression corpus for idiomatic sentence generation and paraphrasing. In Proceedings of the 17th Workshop on Multiword Expressions, pages 33–48. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jianing+Zhou%2C+Hongyu+Zheng%2C+and+Suma+Bhat%2E+2021%2E+PIE%3A+A+parallel+idiomatic+expression+corpus+for+idiomatic+sentence+generation+and+paraphrasing%2E+In+Proceedings+of+the+17th+Workshop+on+Multiword+Expres)

*Updated 2 November 2026*