---
title: "Learning Word Embeddings from Glosses: A Multi-Loss"
description: "The paper addresses Arabic reverse dictionary modeling: given a definition (gloss), produce an embedding that retrieves the… 15 references."
authors: ["Engy Ibrahim", "Farhah Adel", "Marwan Torki", "Nagwa El-Makky"]
year: 2025
venue: "Proceedings of The Third Arabic Natural Language Processing Conference"
source: "https://aclanthology.org/2025.arabicnlp-main.31/"
doi: "10.18653/v1/2025.arabicnlp-main.31"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ibrahim-2025-learning-word-embeddings-glosses-multi/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ibrahim-2025-learning-word-embeddings-glosses-multi/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/18/) / Learning Word Embeddings from Glosses: A Multi-Loss Framework for Arabic Reverse Dictionary Tasks

# Learning Word Embeddings from Glosses: A Multi-Loss Framework for Arabic Reverse Dictionary Tasks

**Authors:** Engy Ibrahim, Farhah Adel, [Marwan Torki](https://md.research.elamur.ai/free/en/authors/marwan-torki/) and [Nagwa El-Makky](https://md.research.elamur.ai/free/en/authors/nagwa-el-makky/) · **Year:** 2025 · **Venue:** [Proceedings of The Third Arabic Natural Language Processing Conference](https://md.research.elamur.ai/free/en/venues/arabicnlp/) ([ArabicNLP 2025](https://md.research.elamur.ai/free/en/venues/arabicnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.arabicnlp-main.31/) · [PDF](https://aclanthology.org/2025.arabicnlp-main.31.pdf) · [DOI 10.18653/v1/2025.arabicnlp-main.31](https://doi.org/10.18653/v1/2025.arabicnlp-main.31) · [OpenAlex](https://openalex.org/W4416037661)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper addresses Arabic reverse dictionary modeling: given a definition (gloss), produce an embedding that retrieves the intended word. The authors use the Arabic Reverse Dictionary Shared Task data (Al-Matham et al., 2023) and target the ELECTRA word vectors. There are two subtasks. In the first, glosses are in Arabic. In the second, glosses are in English, and the authors translate them into Arabic so that a single Arabic pipeline serves both. MARBERTv2 and the AraBART encoder are fine-tuned separately, and their outputs are averaged into an ensemble. The main argument is that plain MSE regression does not make the correct word the nearest candidate. They therefore train with three combined objectives:
- a batch-level contrastive term,
- a term that matches the pairwise similarity structure of predictions to that of targets,
- a term that aligns the mean of predictions with the mean of targets.

On the primary rank metric, the system clearly outperforms earlier shared-task entries in both subtasks. Cosine similarity and MSE, however, are worse than those baselines. The work shows that matching the training objective to the retrieval goal matters as much as the choice of encoder.

## Key points
- Dataset sizes: Arabic dictionary subset 45,200 / 6,400 / 6,410 (train/dev/test); cross-lingual mapping subset 2,862 / 301 / 1,213.
- Test rank results:
  - Subtask 1: 0.0508, versus 0.242 for the best prior system (Rosetta Stone).
  - Subtask 2: 0.0278, versus 0.127.
- Trade-off: Subtask 1 cosine similarity is 0.370 against Rosetta Stone's 0.645, so the gains are specific to rank.
- Training setup: NT-Xent with temperature 0.07, AdamW at learning rate 5e-5, batch size 100, 10 epochs with early stopping, weight decay 1e-4. The final embedding is the mean of the MARBERTv2 and AraBART outputs.
- Loss ablation, measured by mean pairwise cosine among predictions (target value 0.327):
  - Contrastive alone collapses it to 0.0097.
  - Adding the structural term raises it to 0.292, but predictions and targets form separate t-SNE clusters, and dev rank worsens to 0.121.
  - Adding the center term (0.282) makes the two clouds overlap and improves all metrics.

## Interesting ideas
- Instead of regressing toward a target vector with MSE, train with an in-batch contrastive loss so the correct item must outscore every distractor; this directly optimizes what a retrieval task measures.
- Penalize the gap between the prediction–prediction similarity matrix and the target–target one, so the model keeps the relational geometry of the target space and not just individual points.
- A cheap diagnostic: compare average pairwise similarity, and the mean vector, of predictions against targets to detect collapse or a shifted cloud that per-item metrics can hide.
- Translate foreign-language input into the language the model already handles, rather than building a separate cross-lingual model, which keeps one pipeline for both settings.
- Report several metrics because they can disagree: here rank improved sharply while cosine and MSE dropped, revealing what the objective actually trades away.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (world): When mapping a descriptive phrase ("the bird that cannot fly") onto a WordNet/Wikidata entry, a ranking objective would make the correct concept outrank its neighbours rather than merely land near it.
- **I2** (learn): When distilling from an LLM teacher tier, matching how examples relate to each other (not only per-example outputs) could keep student representations of related verb classes or frames coherently grouped.
- **I3** (learn): Evaluation reports could include such distribution-level checks next to accuracy, flagging a retrained component whose outputs drifted as a whole while scores looked fine.
- **I4** (back): Non-English text could be machine-translated into English before parsing, so the existing tree → frames → world chain is reused, with the translation step recorded for explainability.
- **I5** (learn): Training runs should log competing metrics, for example attachment accuracy against frame-level correctness, so that a gain on one is not mistaken for overall improvement.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Alexandria University](https://md.research.elamur.ai/free/en/universities/alexandria-university/) — University, Alexandria, Egypt · 42 works · 287 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 3,066 works

## Cite

Cite as: Engy Ibrahim, Farhah Adel, Marwan Torki and Nagwa El-Makky. 2025. Learning Word Embeddings from Glosses: A Multi-Loss Framework for Arabic Reverse Dictionary Tasks. Proceedings of The Third Arabic Natural Language Processing Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ibrahim2025learning,
  title = {{Learning Word Embeddings from Glosses: A Multi-Loss Framework for Arabic Reverse Dictionary Tasks}},
  author = {Engy Ibrahim, Farhah Adel, Marwan Torki and Nagwa El-Makky},
  booktitle = {Proceedings of The Third Arabic Natural Language Processing Conference},
  year = {2025},
  doi = {10.18653/v1/2025.arabicnlp-main.31},
  url = {https://doi.org/10.18653/v1/2025.arabicnlp-main.31}
}
```

- [← previous LC-Eval: A Bilingual Multi-Task Evaluation Benchmark for Long-Context Understanding](https://md.research.elamur.ai/free/en/papers/jubair-2025-lc-eval-bilingual-multi-task/)
- [next → Lemmatizing Dialectal Arabic with Sequence-to-Sequence Models](https://md.research.elamur.ai/free/en/papers/saeed-2025-lemmatizing-dialectal-arabic-sequence-sequence/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/18/)

## References

15 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Muhammad Abdul-Mageed, AbdelRahim Elmadany, and El Moatez Billah Nagoudi. 2020. [Arbert & marbert: Deep bidirectional transformers for arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). arXiv preprint arXiv:2101.01785. [↗](https://arxiv.org/abs/2101.01785)
2. Rawan Al-Matham, Waad Alshammari, Abdulrahman AlOsaimy, Sarah Alhumoud, Asma Wazrah, Afrah Altamimi, Halah Alharbi, and Abdullah Alaifi. 2023. [KSAA-RD shared task: Arabic reverse dictionary](https://md.research.elamur.ai/free/en/papers/matham-2023-ksaa-rd-shared-task-arabic/). In Proceedings of ArabicNLP 2023, pages 450–460, Singapore (Hybrid). Association for Computational Linguistics.
3. Mikel Artetxe, Vedanuj Goswami, Shruti Bhosale, Angela Fan, and Luke Zettlemoyer. 2023. Revisiting machine translation for cross-lingual classification. arXiv preprint arXiv:2305.14240. [↗](https://arxiv.org/abs/2305.14240)
4. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101. [↗](https://arxiv.org/abs/1711.05101)
5. Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. [Efficient estimation of word representations in vector space](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/). arXiv preprint arXiv:1301.3781. [↗](https://arxiv.org/abs/1301.3781)
6. Abdelrahim Qaddoumi. 2023. [Abed at ksaa-rd shared task: Enhancing arabic word embedding with modified bert multilingual](https://md.research.elamur.ai/free/en/papers/qaddoumi-2023-abed-ksaa-rd-shared-task/). In Proceedings of ArabicNLP 2023, pages 472–476.
7. Serry Sibaee, Samar Ahmad, Ibrahim Khurfan, Vian Sabeeh, Ahmed Bahaaulddin, Hanan Belhaj, and Abdullah Alharbi. 2023. [Qamosy at arabic reverse dictionary shared task: Semi decoder architecture for reverse dictionary with sbert encoder](https://md.research.elamur.ai/free/en/papers/sibaee-2023-qamosy-arabic-reverse-dictionary-shared/). In Proceedings of ArabicNLP 2023, pages 467–471.
8. Roger Brown and David McNeill. 1966. The “tip of the tongue” phenomenon. Journal of verbal learning and verbal behavior, 5(4):325–337. [find](https://scholar.google.com/scholar?q=Roger+Brown+and+David+McNeill%2E+1966%2E+The+%E2%80%9Ctip+of+the+tongue%E2%80%9D+phenomenon%2E+Journal+of+verbal+learning+and+verbal+behavior%2C+5%284%29%3A325%E2%80%93337%2E)
9. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. [A simple framework for contrastive learning of visual representations](https://md.research.elamur.ai/free/en/papers/chen-2020-simple-framework-contrastive-learning-visual/). In International conference on machine learning, pages 1597–1607. PmLR.
10. Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. 2020. Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555. [↗](https://arxiv.org/abs/2003.10555)
11. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pages 4171–4186. [↗](https://doi.org/10.18653/v1/n19-1423)
12. Moussa Kamal Eddine, Nadi Tomeh, Nizar Habash, Joseph Le Roux, and Michalis Vazirgiannis. 2022. [Arabart: a pretrained arabic sequence-to-sequence model for abstractive summarization](https://md.research.elamur.ai/free/en/papers/eddine-2022-arabart-pretrained-arabic-sequence-sequence/). arXiv preprint arXiv:2203.10945. [↗](https://arxiv.org/abs/2203.10945)
13. Ahmed ElBakry, Mohamed Gabr, Muhammad El-Nokrashy, and Badr AlKhamissi. 2023. [Rosetta stone at ksaa-rd shared task: A hop from language modeling to word–definition alignment](https://md.research.elamur.ai/free/en/papers/elbakry-2023-rosetta-stone-ksaa-rd-shared/). arXiv preprint arXiv:2310.15823. [↗](https://arxiv.org/abs/2310.15823)
14. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. [Simcse: Simple contrastive learning of sentence embeddings](https://md.research.elamur.ai/free/en/papers/gao-2021-simcse-simple-contrastive-learning-sentence/). arXiv preprint arXiv:2104.08821. [↗](https://arxiv.org/abs/2104.08821)
15. Felix Hill, Kyunghyun Cho, Anna Korhonen, and Yoshua Bengio. 2016. [Learning to understand phrases by embedding the dictionary](https://md.research.elamur.ai/free/en/papers/hill-2016-learning-understand-phrases-embedding-dictionary/). Transactions of the Association for Computational Linguistics, 4:17– 30.

*Updated 2 November 2026*