---
title: "CASE – Condition-Aware Sentence Embeddings for Conditional"
description: "The authors propose CASE, a two-stage bi-encoder approach built on LLM-based embedding models… 27 references, cited by 1 paper here."
authors: ["Gaifan Zhang", "Yi Zhou", "Danushka Bollegala"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.eacl-long.231/"
doi: "10.18653/v1/2026.eacl-long.231"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2026-case-condition-aware-sentence-embeddings/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2026-case-condition-aware-sentence-embeddings/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/10/) / CASE – Condition-Aware Sentence Embeddings for Conditional Semantic Textual Similarity Measurement

# CASE – Condition-Aware Sentence Embeddings for Conditional Semantic Textual Similarity Measurement

**Authors:** Gaifan Zhang, [Yi Zhou](https://md.research.elamur.ai/free/en/authors/yi-zhou/) and [Danushka Bollegala](https://md.research.elamur.ai/free/en/authors/danushka-bollegala/) · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-long.231/) · [PDF](https://aclanthology.org/2026.eacl-long.231.pdf) · [DOI 10.18653/v1/2026.eacl-long.231](https://doi.org/10.18653/v1/2026.eacl-long.231) · [OpenAlex](https://openalex.org/W7140098840)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary

The paper addresses Conditional Semantic Textual Similarity (C-STS), where the similarity of two sentences depends on a stated condition (aspect). The authors propose CASE, a two-stage bi-encoder approach built on LLM-based embedding models without retraining them. First, the LLM encodes the condition, while each sentence is placed in the instruction prompt. The sentence thus only shapes the attention over condition tokens during pooling. A condition-only embedding is then subtracted, which makes the embedding space more isotropic and consistently raises C-STS scores. Second, a small supervised feed-forward network maps the embeddings into a much lower-dimensional space, and cosine similarity gives the rating. Training uses the refined C-STS data of Zhang et al. (2025). Prompting the model to encode the condition given the sentence turned out better than the reverse order. LLM embeddings beat MLM embeddings on this task. CASE achieves much of the gain of full fine-tuning at a fraction of the compute. This makes it practical for retrieval, QA answer comparison and KGC-style similarity.

## Key points

- **Data:** The original C-STS dataset by Deshpande et al. (2023) contains 18,908 instances. Training uses the LLM-refined version by Zhang et al. (2025), which has 14,176 instances.
- **Model size and dimensionality:** The projection FFN has fewer than 5M parameters. It compresses embeddings 8×, for example from 4096 (NV-embed-v2, Qwen3-Embedding-8B) down to 512 dimensions.
- **Results with frozen Qwen3-8B (Spearman):** The raw embeddings score 24.59, while CASE reaches 71.64.
- **Comparison with LoRA fine-tuning:** A LoRA fine-tuned Qwen3-8B reaches 79.91, so CASE gets about 90% of that performance. Adding the FFN on top of the fine-tuned model gives 79.11 at 512 dimensions, which keeps about 99% of the performance.
- **Compute cost:** The FFN trains in under a minute on one RTX A6000 GPU using less than 1 GB of memory. LoRA fine-tuning takes more than 4 hours on two A100 GPUs.
- **Limitations and code:** The evaluation covers English only. Code is available at github.com/LivNLP/CASE.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

## Cite

Cite as: Gaifan Zhang, Yi Zhou and Danushka Bollegala. 2026. CASE – Condition-Aware Sentence Embeddings for Conditional Semantic Textual Similarity Measurement. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2026case,
  title = {{CASE – Condition-Aware Sentence Embeddings for Conditional Semantic Textual Similarity Measurement}},
  author = {Gaifan Zhang, Yi Zhou and Danushka Bollegala},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-long.231},
  url = {https://doi.org/10.18653/v1/2026.eacl-long.231}
}
```

- [← previous Can you map it to English? The Role of Cross-Lingual Alignment in the Multilingual Performance of LLMs](https://md.research.elamur.ai/free/en/papers/ravisankar-2026-can-you-map-it-english/)
- [next → CaseFacts: A Benchmark for Legal Fact-Checking and Precedent Retrieval](https://md.research.elamur.ai/free/en/papers/putta-2026-casefacts-benchmark-legal-fact-checking/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/10/)

## References

27 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Joshua Ainslie, Tao Lei, Michiel de Jong, Santiago Ontanon, Siddhartha Brahma, Yury Zemlyanskiy, David Uthus, Mandy Guo, James Lee-Thorp, Yi Tay, Yun-Hsuan Sung, and Sumit Sanghai. 2023. [CoLT5: Faster long-range transformers with conditional computation](https://md.research.elamur.ai/free/en/papers/ainslie-2023-colt5-faster-long-range-transformers/). In Proceedings of the 2023 Conference on Empi…
2. Ameet Deshpande, Carlos Jimenez, Howard Chen, Vishvak Murahari, Victoria Graf, Tanmay Rajpurohit, Ashwin Kalyan, Danqi Chen, and Karthik Narasimhan. 2023. C-STS: Conditional semantic [https://huggingface.co/models](https://huggingface.co/models) textual similarity. In Proceedings of the 2023 Conference on Empirical Methods in Natur… [↗](https://huggingface.co/models)
3. Samantha Durdy, Michael W. Gaultois, Vladimir Gusev, Danushka Bollegala, and Matthew J. Rosseinsky. 2023. Metrics for quantifying isotropy in high dimensional unsupervised clustering tasks in a materials context. Preprint, arXiv:2305.16372. — [arXiv:2305.16372](https://arxiv.org/abs/2305.16372) [find](https://scholar.google.com/scholar?q=Samantha+Durdy%2C+Michael+W%2E+Gaultois%2C+Vladimir+Gusev%2C+Danushka+Bollegala%2C+and+Matthew+J%2E+Rosseinsky%2E+2023%2E+Metrics+for+quantifying+isotropy+in+high+dimensional+unsupervised+clustering+tasks+in+a+materi)
4. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. [SimCSE: Simple contrastive learning of sentence embeddings](https://md.research.elamur.ai/free/en/papers/gao-2021-simcse-simple-contrastive-learning-sentence/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 6894–6910, Stroudsburg, PA, USA. Association for Computational Linguistics.
5. David Ha, Andrew M Dai, and Quoc V Le. 2017. Hyper-Networks. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=David+Ha%2C+Andrew+M+Dai%2C+and+Quoc+V+Le%2E+2017%2E+Hyper%2DNetworks%2E+In+International+Conference+on+Learning+Representations%2E)
6. Geoffrey E. Hinton, Nitish Srivastava, Alex Krizhevsky, Ilya Sutskever, and Ruslan R. Salakhutdinov. 2012. Improving neural networks by preventing co-adaptation of feature detectors. Preprint, arXiv:1207.0580. [find](https://scholar.google.com/scholar?q=Geoffrey+E%2E+Hinton%2C+Nitish+Srivastava%2C+Alex+Krizhevsky%2C+Ilya+Sutskever%2C+and+Ruslan+R%2E+Salakhutdinov%2E+2012%2E+Improving+neural+networks+by+preventing+co%2Dadaptation+of+feature+detectors%2E+Preprint%2C+arXiv%3A1)
7. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. [LoRA: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In International Conference on Learning Representations.
8. Masahiro Kaneko and D Bollegala. 2021. Unmasking the mask - evaluating social biases in masked language models. National Conference on Artificial Intelligence, pages 11954–11962. [find](https://scholar.google.com/scholar?q=Masahiro+Kaneko+and+D+Bollegala%2E+2021%2E+Unmasking+the+mask+%2D+evaluating+social+biases+in+masked+language+models%2E+National+Conference+on+Artificial+Intelligence%2C+pages+11954%E2%80%9311962%2E)
9. Diederik P Kingma and Jimmy Ba. 2014. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). Int Conf Learn Represent, abs/1412.6980.
10. Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, et al. 2022. Matryoshka representation learning. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Aditya+Kusupati%2C+Gantavya+Bhatt%2C+Aniket+Rege%2C+Matthew+Wallingford%2C+Aditya+Sinha%2C+Vivek+Ramanujan%2C+William+Howard%2DSnyder%2C+Kaifeng+Chen%2C+Sham+Kakade%2C+Prateek+Jain%2C+et+al%2E+2022%2E+Matryoshka+representation)
11. Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2024. [Nv-embed: Improved techniques for training llms as generalist embedding models](https://md.research.elamur.ai/free/en/papers/lee-2024-nv-embed-improved-techniques-training/). arXiv preprint arXiv:2405.17428.
12. Baixuan Li, Yunlong Fan, and Zhiqiang Gao. 2024. SEAVER: Attention reallocation for mitigating distractions in language models for conditional semantic textual similarity measurement. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 78–95, Stroudsburg, PA, USA. Associa… [find](https://scholar.google.com/scholar?q=Baixuan+Li%2C+Yunlong+Fan%2C+and+Zhiqiang+Gao%2E+2024%2E+SEAVER%3A+Attention+reallocation+for+mitigating+distractions+in+language+models+for+conditional+semantic+textual+similarity+measurement%2E+In+Findings+of+t)
13. Xianming Li and Jing Li. 2024. BeLLM: Backward dependency enhanced large language model for sentence embeddings. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 792–804, M… [find](https://scholar.google.com/scholar?q=Xianming+Li+and+Jing+Li%2E+2024%2E+BeLLM%3A+Backward+dependency+enhanced+large+language+model+for+sentence+embeddings%2E+In+Proceedings+of+the+2024+Conference+of+the+North+American+Chapter+of+the+Association+)
14. Ziyong Lin, Quansen Wang, Zixia Jia, and Zilong Zheng. 2024. Varying sentence representations via condition-specified routers. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 17390–17401, Stroudsburg, PA, USA. Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Ziyong+Lin%2C+Quansen+Wang%2C+Zixia+Jia%2C+and+Zilong+Zheng%2E+2024%2E+Varying+sentence+representations+via+condition%2Dspecified+routers%2E+In+Proceedings+of+the+2024+Conference+on+Empirical+Methods+in+Natural+Lan)
15. Xinyue Liu, Zeyang Qin, Zeyu Wang, Wenxin Liang, Linlin Zong, and Bo Xu. 2025. Conditional semantic textual similarity via conditional contrastive learning. In Proceedings of the 31st International Conference on Computational Linguistics, pages 4548–4560. [find](https://scholar.google.com/scholar?q=Xinyue+Liu%2C+Zeyang+Qin%2C+Zeyu+Wang%2C+Wenxin+Liang%2C+Linlin+Zong%2C+and+Bo+Xu%2E+2025%2E+Conditional+semantic+textual+similarity+via+conditional+contrastive+learning%2E+In+Proceedings+of+the+31st+International+Co)
16. Goutam Majumder, Partha Pakray, Alexander Gelbukh, and David Pinto. 2016. Semantic textual similarity methods, tools, and applications: A survey. Computación y Sistemas, 20(4):647–665. [find](https://scholar.google.com/scholar?q=Goutam+Majumder%2C+Partha+Pakray%2C+Alexander+Gelbukh%2C+and+David+Pinto%2E+2016%2E+Semantic+textual+similarity+methods%2C+tools%2C+and+applications%3A+A+survey%2E+Computaci%C3%B3n+y+Sistemas%2C+20%284%29%3A647%E2%80%93665%2E)
17. Christopher D Manning, Prabhakar Raghavan, and Hinrich Schutze. 2008. Introduction to Information Retrieval. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Christopher+D+Manning%2C+Prabhakar+Raghavan%2C+and+Hinrich+Schutze%2E+2008%2E+Introduction+to+Information+Retrieval%2E+Cambridge+University+Press%2E)
18. Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. [MTEB: Massive text embedding benchmark](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-mteb-massive-text-embedding-benchmark/). In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014–2037, Dubrovnik, Croatia. Association for Computational Linguistic…
19. Sara Rajaee and Mohammad Taher Pilehvar. 2022. An isotropy analysis in the multilingual BERT embedding space. In Findings of the Association for Computational Linguistics: ACL 2022, pages 1309–1316, Dublin, Ireland. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Sara+Rajaee+and+Mohammad+Taher+Pilehvar%2E+2022%2E+An+isotropy+analysis+in+the+multilingual+BERT+embedding+space%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2022%2C+pages+1309%E2%80%931316%2C+D)
20. Nils Reimers and Iryna Gurevych. 2019. [Sentence-BERT: Sentence embeddings using Siamese BERT-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3980–3990,…
21. Julian Risch, Timo Möller, Julian Gutsch, and Malte Pietsch. 2021. Semantic answer similarity for evaluating question answering models. In Proceedings of the 3rd Workshop on Machine Reading for Question Answering, pages 149–157, Stroudsburg, PA, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Julian+Risch%2C+Timo+M%C3%B6ller%2C+Julian+Gutsch%2C+and+Malte+Pietsch%2E+2021%2E+Semantic+answer+similarity+for+evaluating+question+answering+models%2E+In+Proceedings+of+the+3rd+Workshop+on+Machine+Reading+for+Questi)
22. Jianlin Su, Jiarun Cao, Weijie Liu, and Yangyiwen Ou. 2021. Whitening sentence representations for better semantics and faster retrieval. Preprint, arXiv:2103.15316. [find](https://scholar.google.com/scholar?q=Jianlin+Su%2C+Jiarun+Cao%2C+Weijie+Liu%2C+and+Yangyiwen+Ou%2E+2021%2E+Whitening+sentence+representations+for+better+semantics+and+faster+retrieval%2E+Preprint%2C+arXiv%3A2103%2E15316%2E)
23. Chongyang Tao, Tao Shen, Shen Gao, Junshuo Zhang, Zhen Li, Kai Hua, Wenpeng Hu, Zhengwei Tao, and Shuai Ma. 2025. Llms are also effective embedding models: An in-depth overview. Preprint, arXiv:2412.12591. — [arXiv:2412.12591](https://arxiv.org/abs/2412.12591) [find](https://scholar.google.com/scholar?q=Chongyang+Tao%2C+Tao+Shen%2C+Shen+Gao%2C+Junshuo+Zhang%2C+Zhen+Li%2C+Kai+Hua%2C+Wenpeng+Hu%2C+Zhengwei+Tao%2C+and+Shuai+Ma%2E+2025%2E+Llms+are+also+effective+embedding+models%3A+An+in%2Ddepth+overview%2E+Preprint%2C+arXiv%3A2412%2E1)
24. Jingxuan Tu, Keer Xu, Liulu Yue, Bingyang Ye, Kyeongmin Rim, and James Pustejovsky. 2024. Linguistically conditioned semantic textual similarity. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1161–1172, Stroudsburg, PA, USA.… [find](https://scholar.google.com/scholar?q=Jingxuan+Tu%2C+Keer+Xu%2C+Liulu+Yue%2C+Bingyang+Ye%2C+Kyeongmin+Rim%2C+and+James+Pustejovsky%2E+2024%2E+Linguistically+conditioned+semantic+textual+similarity%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Assoc)
25. Kosuke Yamada and Peinan Zhang. 2025. Out-of-the-box conditional text embeddings from large language models. Preprint, arXiv:2504.16411. — [arXiv:2504.16411](https://arxiv.org/abs/2504.16411) [find](https://scholar.google.com/scholar?q=Kosuke+Yamada+and+Peinan+Zhang%2E+2025%2E+Out%2Dof%2Dthe%2Dbox+conditional+text+embeddings+from+large+language+models%2E+Preprint%2C+arXiv%3A2504%2E16411%2E+%E2%80%94+%5BarXiv%3A2504%2E16411%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2504%2E16411%29)
26. Young Yoo, Jii Cha, Changhyeon Kim, and Taeuk Kim. 2024. Hyper-CL: Conditioning sentence representations with hypernetworks. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 700–711, Bangkok, Thailand. Association for Computati… [find](https://scholar.google.com/scholar?q=Young+Yoo%2C+Jii+Cha%2C+Changhyeon+Kim%2C+and+Taeuk+Kim%2E+2024%2E+Hyper%2DCL%3A+Conditioning+sentence+representations+with+hypernetworks%2E+In+Proceedings+of+the+62nd+Annual+Meeting+of+the+Association+for+Computatio)
27. Gaifan Zhang, Yi Zhou, and Danushka Bollegala. 2025. Annotating training data for conditional semantic textual similarity measurement using large language models. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 27015–27027, Suzhou, China. Association… [find](https://scholar.google.com/scholar?q=Gaifan+Zhang%2C+Yi+Zhou%2C+and+Danushka+Bollegala%2E+2025%2E+Annotating+training+data+for+conditional+semantic+textual+similarity+measurement+using+large+language+models%2E+In+Proceedings+of+the+2025+Conference)

*Updated 31 October 2026*