---
title: "CAMMAR — paper by Awinat and Puente (2026)"
description: "The design follows Al-Jurjānī's theory of naẓm, in which figurative meaning is built on top of literal relations. 29 references."
authors: ["Suzan Awinat", "Alfonso Ortega del Puente"]
year: 2026
venue: "arXiv:2607.15847"
source: "https://arxiv.org/abs/2607.15847"
doi: "10.48550/arxiv.2607.15847"
arxiv: "2607.15847"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/awinat-2026-cammar-culture-aware-matryoshka-metaphorical/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/awinat-2026-cammar-culture-aware-matryoshka-metaphorical/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/24/) / CAMMAR: Culture-Aware Matryoshka for Metaphorical Arabic Representations

# CAMMAR: Culture-Aware Matryoshka for Metaphorical Arabic Representations

**Authors:** Suzan Awinat and Alfonso Ortega del Puente · **Year:** 2026 · **Venue:** arXiv:2607.15847 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2607.15847) · [PDF](https://arxiv.org/pdf/2607.15847) · [DOI 10.48550/arxiv.2607.15847](https://doi.org/10.48550/arxiv.2607.15847) · [arXiv 2607.15847](https://arxiv.org/abs/2607.15847) · [OpenAlex](https://openalex.org/W7169755730)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Arabic metaphor carries culturally specific knowledge, but standard Arabic encoders mix surface word meaning, cultural symbolism and figurative sense into one vector. The authors call this "semantic smearing." CAMMAR addresses it by attaching three nested token-level projection heads to a single NeoAraBERT-MSA encoder: a lexical layer, a cultural layer and a metaphorical layer, each smaller than the one before. The heads are trained one after another, and earlier parts are frozen as each new phase begins. The design follows Al-Jurjānī's theory of naẓm, in which figurative meaning is built on top of literal relations. Metaphoricity is read without a classifier: the metaphorical embedding is mapped back into the lexical frame, and its distance from the lexical embedding gives a metaphoricity score W_met. On a new span-annotated set of 184 word-matched figurative/literal pairs, the score separates metaphor well when paired supervision shapes the geometry. It stays at chance when only an unsupervised corpus contrast is used. The result matters because it offers an interpretable, per-word figurativeness measure for a language that LLM benchmarks show is poorly handled.

## Key points
- The backbone is NeoAraBERT-MSA, a 28-layer encoder with d=768 and a 65k SentencePiece vocabulary. The nested heads have 768, 384 and 192 dimensions. Each head is a linear map followed by GELU and layer normalization.
- Training has three phases:
  - Phase 1 is lexical grounding on 100k Arabic Wikipedia passages, sampled from about 5.2M, using 15% MLM among its objectives. The encoder is tuned only in this phase.
  - Phase 2 trains the cultural head, with earlier parts frozen.
  - Phase 3 trains the metaphorical head and adopts CAMEL's self-paced contrastive temperature schedule.
- Under paired supervision, the readout reaches AUC up to 0.84. The figurative member of a pair scores above its literal twin in 82.6% of pairs.
- The unsupervised arm contrasts poetry with Wikipedia and lands at chance. The authors attribute this to corpus identity mixing up register with figurativity.
- Anchoring the lexical layer to morphological roots yields a modest yet consistent improvement in the readout. Direct probing does not show this gain.
- Known limitations:
  - The gold data was generated with LLMs and curated inline by one native speaker, with no inter-annotator agreement reported.
  - There is no vanilla-MRL or supervised baseline.
  - The cultural layer generalizes only weakly to held-out anchors.

## Interesting ideas
- Each successive nested prefix of one embedding gets its own more abstract objective, and earlier layers are frozen as later ones train, so the subspaces stay specialized instead of blending together.
- Figurativeness is scored as the distance between a word's literal-anchor representation and its abstract representation, after a small projector puts both in one coordinate frame.
- Evaluation uses minimal pairs that hold the target word fixed and switch only between figurative and literal use, which removes word-identity confounds.
- The unsupervised corpus-contrast arm is reported openly as a chance-level result, which shows that a register proxy does not deliver figurativity.
- Root grounding helped only when judged by its effect on a downstream readout, not by direct probing. In other words, a layer's value can lie in being a stable measurement anchor.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Staged training with freezing could let a gparse-style encoder add new heads, for example for semantic roles, without degrading the parsing representation already learned.
- **I2** (frames): The prag irony or implicature module could flag a word whose contextual reading drifts far from its lexical/VerbNet default, then explain the flag by naming the literal anchor it departs from.
- **I3** (learn): Same-word literal/figurative or same-verb different-frame pairs would be cheap, sharp tests that frame assignment truly depends on context.
- **I4** (learn): This is a model negative control. Every proxy-supervised signal should be checked against a condition where it is expected to fail, so that style artifacts are not mistaken for meaning.
- **I5** (tokens): Improvements to lemmas or morphology should be judged by their downstream effect on the frames and world steps, not only by intrinsic tagging accuracy.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Universidad Autónoma de Madrid](https://md.research.elamur.ai/free/en/universities/universidad-autonoma-de-madrid/) — University, Madrid, Spain · 7 works · 21 citations

Also on the byline: [IE University](https://ror.org/02jjdwm75).

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 870 works

🧭 Topics (2): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works · [🧷 Idioms & multiword expressions](https://md.research.elamur.ai/free/en/topics/multiword-expressions/) · 217 works

## Cite

Cite as: Suzan Awinat and Alfonso Ortega del Puente. 2026. CAMMAR: Culture-Aware Matryoshka for Metaphorical Arabic Representations. arXiv:2607.15847.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{awinat2026cammar,
  title = {{CAMMAR: Culture-Aware Matryoshka for Metaphorical Arabic Representations}},
  author = {Suzan Awinat and Alfonso Ortega {del Puente}},
  year = {2026},
  doi = {10.48550/arxiv.2607.15847},
  eprint = {2607.15847},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2607.15847}
}
```

- [← previous Call Support Copilot: A Reproducible Multimodal System for Speech Emotion Recognition, Intent Understanding, and Agent Assistance](https://md.research.elamur.ai/free/en/papers/rakhmatillokhon-2026-call-support-copilot-reproducible-multimodal/)
- [next → Can a Remedy Find a Researcher? Exploring the Development of Semantic Knowledge in Italian BabyLMs](https://md.research.elamur.ai/free/en/papers/suozzi-2026-can-remedy-find-researcher-exploring/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/24/)

## References

29 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Chadi Abou Chakra, Hadi Hamoud, Osama Rakan Al Mraikhat, Qusai Abu Obaida, Mohamad Ballout, and Fadi A. Zaraket. 2026. [NeoAraBERT: A modern foundation model for arabic embeddings with diacritics-aware tokenization and pos-targeted masking](https://md.research.elamur.ai/free/en/papers/chakra-2026-neoarabert-modern-foundation-model-arabic/). In Findings of the Association for Computational Linguistics: ACL 2026, San Diego, California, United States. Accepted paper.
2. Wafa Al Ghallabi, Ritesh Thawkar, Sara Ghaboura, Ketan Pravin More, Omkar Thawakar, Hisham Cholakkal, Salman Khan, and Rao Muhammad Anwer. 2025. [Fann or flop: A multigenre, multiera benchmark for Arabic poetry understanding in LLMs](https://md.research.elamur.ai/free/en/papers/ghallabi-2025-fann-or-flop-multigenre-multiera/). In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 20224–20244, Suzhou, China. Association for Computational Linguistics. Abd al-Qahir Al-Jurjani. 1078. Dalail al-Ijaz . Dar al-Khanji, Cairo, Egypt.
3. Abd al-Qahir Al-Jurjani. 1954. Asrar al-Balgha. Mat.ba’at Wizārat al-Ma’ārif, Istanbul. Originally composed ca. 1078 CE. The classic critical edition by Hellmut Ritter, based on three Istanbul manuscript witnesses (Husrev Paşa 654, Feyzullah 1771, Hamidiye 1238) and the Rashid Rida printed edition. Contains a 27-page English-language editorial introduction by Ritter (pp. 1– 27). Cite this work for the metaphor/isti’āra discussions and the h.aqı̄qa/majāz distinction; cite jurjani:dalail for the foundational naz.m doctrine. [find](https://scholar.google.com/scholar?q=Abd+al%2DQahir+Al%2DJurjani%2E+1954%2E+Asrar+al%2DBalgha%2E+Mat%2Eba%E2%80%99at+Wiza%CC%84rat+al%2DMa%E2%80%99a%CC%84rif%2C+Istanbul%2E+Originally+composed+ca%2E+1078+CE%2E+The+classic+critical+edition+by+Hellmut+Ritter%2C+based+on+three+Istanbul+manus)
4. Youssef Bin Abi Bakr Al-Sakkaki. 1983. Miftah Al-Ulum. Dar al-Kutub al-Ilmiyya, Beirut. [find](https://scholar.google.com/scholar?q=Youssef+Bin+Abi+Bakr+Al%2DSakkaki%2E+1983%2E+Miftah+Al%2DUlum%2E+Dar+al%2DKutub+al%2DIlmiyya%2C+Beirut%2E)
5. Jar Allah Al-Zamakhshari. 1882. Asas al-Balagha. Dar al-Kutub al-Ilmiyya, Beirut. [find](https://scholar.google.com/scholar?q=Jar+Allah+Al%2DZamakhshari%2E+1882%2E+Asas+al%2DBalagha%2E+Dar+al%2DKutub+al%2DIlmiyya%2C+Beirut%2E)
6. Mena Attia, Aashiq Muhamed, Mai Alkhamissi, Thamar Solorio, and Mona T. Diab. 2026. Beyond understanding: Evaluating the pragmatic gap in LLMs’ cultural processing of figurative language. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 7238–7265, Rabat, Morocco. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2510.23828)
7. Zouheir Banou, Sanaa El Filali, El Habib Benlahmar, Laila Eljiani, and Fatima-Zahra Alaoui. 2025. Developing a multi-layer ontology construction framework for Arabic language processing: Focus on figurative language potential. IEEE Access, 13:138535–138550. [↗](https://doi.org/10.1109/access.2025.3596130)
8. Joanne Boisson, Asahi Ushio, Hsuvas Borkakoty, Kiamehr Rezaee, Dimosthenis Antypas, Zara Siddique, Nina White, and Jose Camacho-Collados. 2024. How are metaphors processed by language models? the case of analogies. In Proceedings of the 28th Conference on Computational Natural Language Learning, pages 365–387, Miami, FL, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Joanne+Boisson%2C+Asahi+Ushio%2C+Hsuvas+Borkakoty%2C+Kiamehr+Rezaee%2C+Dimosthenis+Antypas%2C+Zara+Siddique%2C+Nina+White%2C+and+Jose+Camacho%2DCollados%2E+2024%2E+How+are+metaphors+processed+by+language+models%3F+the+case)
9. Minjin Choi, Sunkyung Lee, Eunseong Choi, Heesoo Park, Junhyuk Lee, Dongwon Lee, and Jongwuk Lee. 2021. MelBERT: Metaphor detection via contextualized late interaction using metaphorical identification theories. In Proceedings of NAACL, pages 1763–1773. [find](https://scholar.google.com/scholar?q=Minjin+Choi%2C+Sunkyung+Lee%2C+Eunseong+Choi%2C+Heesoo+Park%2C+Junhyuk+Lee%2C+Dongwon+Lee%2C+and+Jongwuk+Lee%2E+2021%2E+MelBERT%3A+Metaphor+detection+via+contextualized+late+interaction+using+metaphorical+identificatio)
10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of NAACL-HLT, pages 4171–4186.
11. Erik-Lân Do Dinh and Iryna Gurevych. 2016. Token-level metaphor detection using neural networks. In Proceedings of the Fourth Workshop on Metaphor in NLP, pages 28–33. [find](https://scholar.google.com/scholar?q=Erik%2DL%C3%A2n+Do+Dinh+and+Iryna+Gurevych%2E+2016%2E+Token%2Dlevel+metaphor+detection+using+neural+networks%2E+In+Proceedings+of+the+Fourth+Workshop+on+Metaphor+in+NLP%2C+pages+28%E2%80%9333%2E)
12. Xiaoyu He, Long Yu, Shengwei Tian, Qimeng Yang, and Jun Long. 2024. Sc-net: Multimodal metaphor detection using semantic conflicts. In Neurocomputing, volume 594. AAAI Press. [↗](https://doi.org/10.1016/j.neucom.2024.127825)
13. Kaidi Jia and Rongsheng Li. 2024. Metaphor detection with context enhancement and curriculum learning. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 2726–2737, Mexico City, Mexico. Association for Computational Linguistics. [↗](https://doi.org/10.48448/b2st-1274)
14. Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, and Ali Farhadi. 2024. Matryoshka representation learning. Preprint, arXiv:2205.13147. [↗](https://arxiv.org/abs/2205.13147)
15. Riwei Lai, Li Chen, Weixin Chen, and Rui Chen. 2026. Matryoshka representation learning for recommendation with layer- and hardness-adaptive negative sampling. ACM Transactions on Intelligent Systems and Technology, 17(4):88:1–88:25. [↗](https://doi.org/10.1145/3800942)
16. George Lakoff and Mark Johnson. 1980. The metaphorical structure of the human conceptual system. Cognitive Science, 4(2):195–208. [↗](https://doi.org/10.1016/s0364-0213(80)80017-6)
17. Samar Mohamed Magdy, Sang Yun Kwon, Fakhraddin Alwajih, Safaa Taher Abdelfadil, Shady Shehata, and Muhammad Abdul-Mageed. 2025. [JAWAHER: A multidialectal dataset of Arabic proverbs for LLM benchmarking](https://md.research.elamur.ai/free/en/papers/magdy-2025-jawaher-multidialectal-dataset-arabic-proverbs/). In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 12320– 12341, Albuquerque, New Mexico. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2503.00231)
18. Veronica Mangiaterra, Chiara Barattieri di San Pietro, Paolo Canal, and Valentina Bambini. 2026. Metaphors’ journeys across time and genre: tracking the evolution of literary metaphors with temporal embeddings. Preprint, arXiv:2602.13701. [↗](https://arxiv.org/abs/2602.13701)
19. Omer Nacar and Anis Koubaa. 2024. [Enhancing semantic similarity understanding in arabic nlp with nested embedding learning](https://md.research.elamur.ai/free/en/papers/nacar-2024-enhancing-semantic-similarity-understanding-arabic/). Preprint, arXiv:2407.21139. [↗](https://arxiv.org/abs/2407.21139)
20. Elisa Sanchez-Bayona and Rodrigo Agerri. 2025. Metaphor and large language models: When surface features matter more than deep understanding. In Findings of the Association for Computational Linguistics: ACL 2025, pages 17462–17477, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Elisa+Sanchez%2DBayona+and+Rodrigo+Agerri%2E+2025%2E+Metaphor+and+large+language+models%3A+When+surface+features+matter+more+than+deep+understanding%2E+In+Findings+of+the+Association+for+Computational+Linguisti)
21. Elisa Sanchez-Bayona and Rodrigo Agerri. 2026. Meta4xnli: A cross-lingual parallel corpus for metaphor detection and interpretation. [↗](https://doi.org/10.1162/coli.a.20)
22. Shutova and Ekaterina. 2010. Models of metaphor in NLP. Proceedings of the 48th Annual Meeting of the ACL, pages 688–697. [find](https://scholar.google.com/scholar?q=Shutova+and+Ekaterina%2E+2010%2E+Models+of+metaphor+in+NLP%2E+Proceedings+of+the+48th+Annual+Meeting+of+the+ACL%2C+pages+688%E2%80%93697%2E)
23. Armin Tourajmehr, Mohammad Reza Modarres, and Yadollah Yaghoobzadeh. 2025. Evaluating the creativity of LLMs in Persian literary text generation. In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 14762–14774, Suzhou, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Armin+Tourajmehr%2C+Mohammad+Reza+Modarres%2C+and+Yadollah+Yaghoobzadeh%2E+2025%2E+Evaluating+the+creativity+of+LLMs+in+Persian+literary+text+generation%2E+In+Findings+of+the+Association+for+Computational+Lingu)
24. Aäron van den Oord, Yazhe Li, and Oriol Vinyals. 2018. [Representation learning with contrastive predictive coding](https://md.research.elamur.ai/free/en/papers/oord-2018-representation-learning-contrastive-predictive-coding/). ArXiv, abs/1807.03748. [↗](https://arxiv.org/abs/1807.03748)
25. Tongzhou Wang and Phillip Isola. 2020. Understanding contrastive representation learning through alignment and uniformity on the hypersphere. In Proceedings of the 37th International Conference on Machine Learning, ICML’20. JMLR.org. [↗](https://doi.org/10.48550/arxiv.2005.10242)
26. Biao Zhang, Lixin Chen, Tong Liu, and Bo Zheng. 2025a. SMEC:rethinking matryoshka representation learning for retrieval embedding compression. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 26209–26222, Suzhou, China. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Biao+Zhang%2C+Lixin+Chen%2C+Tong+Liu%2C+and+Bo+Zheng%2E+2025a%2E+SMEC%3Arethinking+matryoshka+representation+learning+for+retrieval+embedding+compression%2E+In+Proceedings+of+the+2025+Conference+on+Empirical+Method)
27. Dongyu Zhang, Xingyuan Lu, Mulin Zhuang, Senqi Yang, and Hongjun Chen. 2025b. Multimodal metaphor recognition based on chain-of-cognition prompting. Cognitive Systems Research, 91:101356. [↗](https://doi.org/10.1016/j.cogsys.2025.101356)
28. Linhao Zhang, Li Jin, Guangluan Xu, Xiaoyu Li, Cai Xu, Kaiwen Wei, Nayu Liu, and Haonan Liu. 2024. CAMEL: Capturing metaphorical alignment with context disentangling for multimodal emotion recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 9341–9349. AAAI Press. [find](https://scholar.google.com/scholar?q=Linhao+Zhang%2C+Li+Jin%2C+Guangluan+Xu%2C+Xiaoyu+Li%2C+Cai+Xu%2C+Kaiwen+Wei%2C+Nayu+Liu%2C+and+Haonan+Liu%2E+2024%2E+CAMEL%3A+Capturing+metaphorical+alignment+with+context+disentangling+for+multimodal+emotion+recognition)
29. Zibin, A., Binhaidara, N., Al-Shahwan, and H. et al. 2025. Metaphor interpretation in jordanian arabic, emirati arabic and classical arabic: artificial intelligence vs. humans. Cognitive Systems Research, 12:942. [↗](https://doi.org/10.1057/s41599-025-05282-0)

*Updated 1 November 2026*