--- title: "Using BERT to Explore Lexical Semantic — Radchankava 2026" description: "Prepositions are very polysemous and sit between grammatical and lexical words, yet NLP has given them little attention." authors: ["Liudmila Radchankava", "Vasily Konovalov"] year: 2026 venue: "The Proceedings for the 6th International Workshop on Computational Approaches to Language Change (LChange'26)" source: "https://aclanthology.org/2026.lchange-1.10/" doi: "10.18653/v1/2026.lchange-1.10" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/papers/radchankava-2026-bert-explore-lexical-semantic-change/" --- [Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/radchankava-2026-bert-explore-lexical-semantic-change/) [home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2026](https://md.research.elamur.ai/free/papers/2026/page/5/) / Using BERT to Explore Lexical Semantic Change of Prepositions # Using BERT to Explore Lexical Semantic Change of Prepositions **Authors:** Liudmila Radchankava, Vasily Konovalov · **Year:** 2026 · **Venue:** The Proceedings for the 6th International Workshop on Computational Approaches to Language Change (LChange'26) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.lchange-1.10/) · [PDF](https://aclanthology.org/2026.lchange-1.10.pdf) · [DOI 10.18653/v1/2026.lchange-1.10](https://doi.org/10.18653/v1/2026.lchange-1.10) · [OpenAlex](https://openalex.org/W7140121667) **Citations:** 0 citations (openalex-snapshot · 2026-10-14) ## Summary Prepositions are very polysemous and sit between grammatical and lexical words, yet NLP has given them little attention. This paper asks whether contextual BERT embeddings can tell preposition senses apart and whether they can reveal how preposition usage shifts over time. The authors take embeddings of each preposition occurrence by averaging BERT's top four layers. They then label senses with a kNN classifier (Euclidean distance) trained on The Preposition Project (TPP) data from SemEval 2007 Task 6. The approach uses no external knowledge, yet bert-base-uncased reaches accuracy close to systems that do. Next, the trained classifier is applied to 2,000 sampled sentences per preposition from COHA. The distance to the nearest labeled TPP example serves as a proxy for contextual divergence, and Spearman's ρ measures how this distance relates to the year of use. Five compound prepositions (into, onto, beside, above, about) show clear diachronic trends, while five simple ones (in, on, at, with, of) show little. The method could be carried over to other languages through multilingual BERT, given a suitable sense-labeled dataset. ## Key points - **Data:** TPP spans 34 English prepositions and supplies 16,557 sentences for training and 8,096 for testing, all taken from FrameNet/BNC. Senses follow the Oxford Dictionary of English, and the original split is used. - **Disambiguation accuracy:** kNN on bert-base-uncased reaches 79.8% test accuracy. This beats every SemEval 2007 participant but trails the multilingual LSTM approach of Gonen and Goldberg (2016) at 81.7%. When tuning k, base BERT scores about 80, bert-large-uncased about 77, and bert-base-multilingual-uncased about 75–76. - **Diachronic corpus:** COHA contains over 475M words from the 1820s–2010s and is genre-balanced by decade. Sentences were sampled uniformly, without temporal stratification. - **Correlations for compound prepositions:** above (−0.13), onto (−0.10), beside (−0.22) and about (−0.13) correlate negatively with year, all with very small p-values. Older uses therefore lie farther from the benchmark examples. Into is the exception, with a weak positive ρ of 0.075. - **Correlations for simple prepositions:** values are near zero (|ρ| ≤ 0.06), and in and of are not significant. The authors explain this by gradual, radially organized change that distance-based methods detect poorly. Manual inspection shows that some outliers come from dialectal text or pronoun contexts rather than genuinely new senses. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{radchankava2026using, title = {{Using BERT to Explore Lexical Semantic Change of Prepositions}}, author = {Radchankava, Liudmila and Konovalov, Vasily}, booktitle = {The Proceedings for the 6th International Workshop on Computational Approaches to Language Change (LChange'26)}, year = {2026}, doi = {10.18653/v1/2026.lchange-1.10}, url = {https://doi.org/10.18653/v1/2026.lchange-1.10} } ``` - [← previous UrBLiMP: A Benchmark for Evaluating the Linguistic Competence of Large Language Models in Urdu](https://md.research.elamur.ai/free/papers/adeeba-2026-urblimp-benchmark-evaluating-linguistic-competence/) - [next → Variation Is the Norm: Embracing Sociolinguistics in NLP](https://md.research.elamur.ai/free/papers/lutgen-2026-variation-is-norm-embracing-sociolinguistics/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/papers/2026/page/5/) ## References 1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Timothy Baldwin, Valia Kordoni, and Aline Villavicencio. 2009. Prepositions in applications: A survey and introduction to the special issue. Comput. Linguistics, 35(2):119–149. [↗](https://doi.org/10.1162/coli.2009.35.2.119)