---
title: "NLU-STR at SemEval-2024 Task 1: Generative-based"
description: "The paper describes a SemEval-2024 Task 1 (SemRel-2024) system for semantic textual relatedness… 51 references, cited by 3 papers here."
authors: ["Sanad Malaysha", "Mustafa Jarrar", "Mohammed Khalilia"]
year: 2024
venue: "Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)"
source: "https://aclanthology.org/2024.semeval-1.128/"
doi: "10.18653/v1/2024.semeval-1.128"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/malaysha-2024-nlu-str-semeval-2024-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/malaysha-2024-nlu-str-semeval-2024-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/12/) / NLU-STR at SemEval-2024 Task 1: Generative-based Augmentation and Encoder-based Scoring for Semantic Textual Relatedness

# NLU-STR at SemEval-2024 Task 1: Generative-based Augmentation and Encoder-based Scoring for Semantic Textual Relatedness

**Authors:** [Sanad Malaysha](https://md.research.elamur.ai/free/en/authors/sanad-malaysha/), [Mustafa Jarrar](https://md.research.elamur.ai/free/en/authors/mustafa-jarrar/) and [Mohammed Khalilia](https://md.research.elamur.ai/free/en/authors/mohammed-khalilia/) · **Year:** 2024 · **Venue:** [Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2024](https://md.research.elamur.ai/free/en/venues/semeval/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.semeval-1.128/) · [PDF](https://aclanthology.org/2024.semeval-1.128.pdf) · [DOI 10.18653/v1/2024.semeval-1.128](https://doi.org/10.18653/v1/2024.semeval-1.128) · [OpenAlex](https://openalex.org/W4401042710)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [3](https://md.research.elamur.ai/free/en/papers/malaysha-2024-nlu-str-semeval-2024-task/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper describes a SemEval-2024 Task 1 (SemRel-2024) system for semantic textual relatedness (STR) in Arabic. STR is a looser notion than similarity: two sentences count as related if they share a topic, a scene or a context, even when their meanings differ. The authors entered two tracks. In supervised Track A (Algerian and Moroccan dialects) they fine-tuned AraBERTv2 and ArBERTv2 as single-output regressors on concatenated sentence pairs. For Moroccan only, they enlarged the training set with Google Gemini. Gemini rewrote one sentence of a pair, the rewrite was matched with the untouched partner, and the pair's original score was kept. In unsupervised Track B (MSA), they scored pairs by cosine similarity of mean-pooled embeddings from frozen models. Results:
- **MSA:** first place (Spearman 0.49).
- **Moroccan:** fifth place (0.83), helped by augmentation.
- **Algerian:** twelfth place (0.53), below the official baseline.

The work shows that cheap LLM-generated paraphrases can help a low-resource dialect. It also shows that the benefit depends on which encoder is being trained.

## Key points
- **Data sizes:** Algerian has 1,261 train / 583 test pairs. Moroccan has 924 train / 425 test pairs. MSA has only 595 test pairs and no labeled training data.
- **Augmentation:** Gemini added 757 filtered Moroccan pairs, giving 1,681 training instances. ChatGPT was rejected after a pilot of twenty prompts, because both LLMs handled Algerian poorly and only Gemini understood Moroccan well.
- **Track A setup:**
  - Input format: [CLS]S1[SEP]S2[SEP].
  - Output: one regression neuron trained with MSE loss.
  - Hyper-parameters: 10 epochs, lr 2e-5, batch 16, max length 512.
  - Preprocessing: AraBERT normalization.
- **Effect of augmentation on Moroccan:**
  - ArBERTv2 improved from 0.77 to 0.83 on test (0.82 to 0.88 on dev).
  - AraBERTv2 dropped from 0.79 to 0.77 on test (0.84 to 0.79 on dev).
- **Track B setup:** CLS, average, max and min pooling were compared, and AraBERTv2 with average pooling worked best. It beat the mBERT baseline of 0.42, while ArBERTv2 reached only 0.34. The authors attribute the weakness of raw cosine scores to the anisotropy of BERT embedding spaces.

## Interesting ideas
- Each sentence of a scored pair is rewritten separately and the label is inherited, so one labeled pair yields extra training pairs without new annotation.
- LLM output is filtered by spotting the model's recognizable refusal or "I'm just a language model" replies, while meaning drift is left to human review.
- Before committing to a generator, a tiny pilot of about twenty prompts shows which LLM actually understands the target variety.
- The same augmented data improved one encoder and degraded another, so the value of augmentation should be measured per model rather than assumed.
- Relatedness differs from similarity: two sentences about Earth's speeds are related despite saying different things, and plain embedding cosine struggles to capture this.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): paraphrase sentences whose frames are already verified, then check that the parser and prag recover the same who-did-what-to-whom frame, giving silver data plus a consistency test.
- **I2** (learn): LLM teacher tiers could auto-discard boilerplate refusals by format matching and route only content-level doubts to a costlier check.
- **I3** (learn): run a small probe set per domain (fables, maths problems) to choose which teacher tier is trusted there, instead of assuming one LLM is good everywhere.
- **I4** (learn): when adding silver data, evaluate each parser (perceptron vs gparse) separately on a held-out set, keeping the unaugmented model as a control.
- **I5** (world): relatedness could be computed explainably from shared entities and topics in the world graph, with no reliance on pooled vectors.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Birzeit University](https://md.research.elamur.ai/free/en/universities/birzeit-university/) — University, Birzeit, Palestine · 36 works · 798 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

## Cite

Cite as: Sanad Malaysha, Mustafa Jarrar and Mohammed Khalilia. 2024. NLU-STR at SemEval-2024 Task 1: Generative-based Augmentation and Encoder-based Scoring for Semantic Textual Relatedness. Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{malaysha2024nlu,
  title = {{NLU-STR at SemEval-2024 Task 1: Generative-based Augmentation and Encoder-based Scoring for Semantic Textual Relatedness}},
  author = {Sanad Malaysha, Mustafa Jarrar and Mohammed Khalilia},
  booktitle = {Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024)},
  year = {2024},
  doi = {10.18653/v1/2024.semeval-1.128},
  url = {https://doi.org/10.18653/v1/2024.semeval-1.128}
}
```

- [← previous MODOS at ArAIEval Shared Task: Multimodal Propagandistic Memes Classification Using Weighted SAM, CLIP and ArabianGPT](https://md.research.elamur.ai/free/en/papers/haouhat-2024-modos-araieval-shared-task-multimodal/)
- [next → No Context Needed: Contextual Quandary In Idiomatic Reasoning With Pre-Trained Language Models](https://md.research.elamur.ai/free/en/papers/cheng-2024-no-context-needed-contextual-quandary/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/12/)

## References

51 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mohamed Abdalla, Krishnapriya Vishnubhotla, and Saif M. Mohammad. 2023. What makes sentences semantically related? A textual relatedness dataset and empirical study. In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2023, Dubrovnik, Croatia, May 2-6, 2023, pages 782–796. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mohamed+Abdalla%2C+Krishnapriya+Vishnubhotla%2C+and+Saif+M%2E+Mohammad%2E+2023%2E+What+makes+sentences+semantically+related%3F+A+textual+relatedness+dataset+and+empirical+study%2E+In+Proceedings+of+the+17th+Confere)
2. Muhammad Abdul-Mageed, AbdelRahim A. Elmadany, and El Moatez Billah Nagoudi. 2021. [ARBERT & MARBERT: deep bidirectional transformers for arabic](https://md.research.elamur.ai/free/en/papers/abdul-mageed-2021-arbert-marbert-deep-bidirectional-transformers/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 7088–7105. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2101.01785)
3. Moustafa Al-Hajj and Mustafa Jarrar. 2021a. Arabgloss-bert: Fine-tuning bert on context-gloss pairs for wsd. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2021), pages 40–48, Online. INCOMA Ltd. [find](https://scholar.google.com/scholar?q=Moustafa+Al%2DHajj+and+Mustafa+Jarrar%2E+2021a%2E+Arabgloss%2Dbert%3A+Fine%2Dtuning+bert+on+context%2Dgloss+pairs+for+wsd%2E+In+Proceedings+of+the+International+Conference+on+Recent+Advances+in+Natural+Language+Proce)
4. Moustafa Al-Hajj and Mustafa Jarrar. 2021b. [Lu-bzu at semeval-2021 task 2: Word2vec and lemma2vec performance in arabic word-in-context disambiguation](https://md.research.elamur.ai/free/en/papers/hajj-2021-lu-bzu-semeval-2021-task/). In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 748– 755, Online. Association for Computational Linguistics.
5. Diana Alhafi, Anton Deik, and Mustafa Jarrar. 2019. Usability evaluation of lexicographic e-services. In The 2019 IEEE/ACS 16th International Conference on Computer Systems and Applications (AICCSA), pages 1–7. IEE. [find](https://scholar.google.com/scholar?q=Diana+Alhafi%2C+Anton+Deik%2C+and+Mustafa+Jarrar%2E+2019%2E+Usability+evaluation+of+lexicographic+e%2Dservices%2E+In+The+2019+IEEE%2FACS+16th+International+Conference+on+Computer+Systems+and+Applications+%28AICCSA%29%2C+)
6. Wissam Antoun, Fady Baly, and Hazem Hajj. 2020. [Arabert: Transformer-based model for arabic language understanding](https://md.research.elamur.ai/free/en/papers/antoun-2020-arabert-transformer-based-model-arabic/). In Proceedings of the 4th Workshop on Open-Source Arabic Corpora and Processing Tools, with a Shared Task on Offensive Language Detection, pages 9–15. [↗](https://doi.org/10.48550/arxiv.2003.00104)
7. Shima Asaadi, Saif M. Mohammad, and Svetlana Kiritchenko. 2019. Big bird: A large, fine-grained, bigram relatedness dataset for examining semantic composition. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, NAACL-HLT 2019, Minneapolis, MN, USA, June 2-7, 2019, Volume 1 (Long and Short Papers), pages 505–516. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Shima+Asaadi%2C+Saif+M%2E+Mohammad%2C+and+Svetlana+Kiritchenko%2E+2019%2E+Big+bird%3A+A+large%2C+fine%2Dgrained%2C+bigram+relatedness+dataset+for+examining+semantic+composition%2E+In+Proceedings+of+the+2019+Conference+of)
8. Alejandro Fuster Baggetto and Víctor Fresno. 2022. Is anisotropy really the cause of BERT embeddings not being semantic? In Findings of the Association for Computational Linguistics: EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 4271–4281. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Alejandro+Fuster+Baggetto+and+V%C3%ADctor+Fresno%2E+2022%2E+Is+anisotropy+really+the+cause+of+BERT+embeddings+not+being+semantic%3F+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2022%2C+Abu+D)
9. Luisa Bentivogli, Raffaella Bernardi, Marco Marelli, Stefano Menini, Marco Baroni, and Roberto Zamparelli. 2016. SICK through the semeval glasses. lesson learned from the evaluation of compositional distributional semantic models on full sentences through semantic relatedness and textual entailment. Lang. Resour. Evaluation, 50(1):95–124. [↗](https://doi.org/10.1007/s10579-015-9332-5)
10. Michele Bevilacqua, Tommaso Pasini, Alessandro Raganato, Roberto Navigli, et al. 2021. Recent trends in word sense disambiguation: A survey. In Proceedings of the Thirtieth International Joint Conference on Artificial Intelligence, IJCAI-21, pages 4330–4338. [find](https://scholar.google.com/scholar?q=Michele+Bevilacqua%2C+Tommaso+Pasini%2C+Alessandro+Raganato%2C+Roberto+Navigli%2C+et+al%2E+2021%2E+Recent+trends+in+word+sense+disambiguation%3A+A+survey%2E+In+Proceedings+of+the+Thirtieth+International+Joint+Confere)
11. Zugang Chen, Jia Song, and Yaping Yang. 2018. An approach to measuring semantic relatedness of geographic terminologies using a thesaurus and lexical database sources. ISPRS Int. J. Geo Inf., 7(3):98. [↗](https://doi.org/10.3390/ijgi7030098)
12. Kareem Darwish, Nizar Habash, Mourad Abbas, Hend Al-Khalifa, Huseein T. Al-Natsheh, Houda Bouamor, Karim Bouzoubaa, Violetta Cavalli-Sforza, Samhaa R. El-Beltagy, Wassim El-Hajj, Mustafa Jarrar, and Hamdy Mubarak. 2021. A panoramic survey of natural language processing in the arab worlds. Commun. ACM, 64(4):72–81. [find](https://scholar.google.com/scholar?q=Kareem+Darwish%2C+Nizar+Habash%2C+Mourad+Abbas%2C+Hend+Al%2DKhalifa%2C+Huseein+T%2E+Al%2DNatsheh%2C+Houda+Bouamor%2C+Karim+Bouzoubaa%2C+Violetta+Cavalli%2DSforza%2C+Samhaa+R%2E+El%2DBeltagy%2C+Wassim+El%2DHajj%2C+Mustafa+Jarrar%2C+and+H)
13. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022. [Language-agnostic BERT sentence embedding](https://md.research.elamur.ai/free/en/papers/feng-2020-language-agnostic-bert-sentence-embedding/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), ACL 2022, Dublin, Ireland, May 22-27, 2022, pages 878– 891. Association for Computational Linguistics.
14. Sana Ghanem, Mustafa Jarrar, Radi Jarrar, and Ibrahim Bounhas. 2023. [A benchmark and scoring algorithm for enriching arabic synonyms](https://md.research.elamur.ai/free/en/papers/ghanem-2023-benchmark-scoring-algorithm-enriching-arabic/). In Proceedings of the 12th International Global Wordnet Conference (GWC2023), pages 215–222. Global Wordnet Association.
15. Mirna El Ghosh, Nicolas Delestre, Jean-Philippe Kotowicz, Cecilia Zanni-Merk, and Habib Abdulrab. 2023. Reltopic: A graph-based semantic relatedness measure in topic ontologies and its applicability for topic labeling of old press articles. Semantic Web, 14(2):293–321. [find](https://scholar.google.com/scholar?q=Mirna+El+Ghosh%2C+Nicolas+Delestre%2C+Jean%2DPhilippe+Kotowicz%2C+Cecilia+Zanni%2DMerk%2C+and+Habib+Abdulrab%2E+2023%2E+Reltopic%3A+A+graph%2Dbased+semantic+relatedness+measure+in+topic+ontologies+and+its+applicability+f)
16. Edouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tomás Mikolov. 2018. [Learning word vectors for 157 languages](https://md.research.elamur.ai/free/en/papers/grave-2018-learning-word-vectors-157-languages/). In Proceedings of the Eleventh International Conference on Language Resources and Evaluation, LREC 2018, Miyazaki, Japan, May 7-12, 2018. European Language Resources Association (ELRA).
17. Karim El Haff, Mustafa Jarrar, Tymaa Hammouda, and Fadi Zaraket. 2022. Curras + baladi: Towards a levantine corpus. In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France. [↗](https://doi.org/10.48550/arxiv.2205.09692)
18. Michael Alexander Kirkwood Halliday and Ruqaiya Hasan. 2014. Cohesion in english. 9. Routledge. [↗](https://doi.org/10.4324/9781315836010)
19. Ali Muttaleb Hasan, Noorhuzaimi Mohd Noor, Taha H Rassem, and Ahmed Muttaleb Hasan. 2020. Knowledge-based semantic relatedness measure using semantic features. International Journal, 9(2). [↗](https://doi.org/10.30534/ijatcse/2020/02922020)
20. Mustafa Jarrar. 2011. Building a formal arabic ontology (invited paper). In Proceedings of the Experts Meeting on Arabic Ontologies and Semantic Networks. ALECSO, Arab League. [find](https://scholar.google.com/scholar?q=Mustafa+Jarrar%2E+2011%2E+Building+a+formal+arabic+ontology+%28invited+paper%29%2E+In+Proceedings+of+the+Experts+Meeting+on+Arabic+Ontologies+and+Semantic+Networks%2E+ALECSO%2C+Arab+League%2E)
21. Mustafa Jarrar. 2021. [The arabic ontology - an arabic wordnet with ontologically clean content](https://md.research.elamur.ai/free/en/papers/jarrar-2022-arabic-ontology-arabic-wordnet-ontologically/). Applied Ontology Journal, 16(1):1–26. [↗](https://doi.org/10.3233/ao-200241)
22. Mustafa Jarrar and Hamzeh Amayreh. 2019. An arabic-multilingual database with a lexicographic search engine. In The 24th International Conference on Applications of Natural Language to Information Systems (NLDB 2019), volume 11608 of LNCS, pages 234–246. Springer. [find](https://scholar.google.com/scholar?q=Mustafa+Jarrar+and+Hamzeh+Amayreh%2E+2019%2E+An+arabic%2Dmultilingual+database+with+a+lexicographic+search+engine%2E+In+The+24th+International+Conference+on+Applications+of+Natural+Language+to+Information+Sys)
23. Mustafa Jarrar, Ahmet Birim, Mohammed Khalilia, Mustafa Erden, and Sana Ghanem. 2023a. [Arbanking77: Intent detection neural model and a new dataset in modern and dialectical arabic](https://md.research.elamur.ai/free/en/papers/jarrar-2023-arbanking77-intent-detection-neural-model/). In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 276–287. ACL.
24. Mustafa Jarrar, Nizar Habash, Faeq Alrimawi, Diyam Akra, and Nasser Zalmout. 2017. Curras: An annotated corpus for the palestinian arabic dialect. Journal Language Resources and Evaluation, 51(3):745– 775. [↗](https://doi.org/10.1007/s10579-016-9370-7)
25. Mustafa Jarrar, Mohammed Khalilia, and Sana Ghanem. 2022. [Wojood: Nested arabic named entity corpus and recognition using bert](https://md.research.elamur.ai/free/en/papers/jarrar-2022-wojood-nested-arabic-named-entity/). In Proceedings of the International Conference on Language Resources and Evaluation (LREC 2022), Marseille, France.
26. Mustafa Jarrar, Sanad Malaysha, Tymaa Hammouda, and Mohammad Khalilia. 2023b. [Salma: Arabic sense-annotated corpus and wsd benchmarks](https://md.research.elamur.ai/free/en/papers/jarrar-2023-salma-arabic-sense-annotated-corpus/). In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 359–369. ACL.
27. Mustafa Jarrar, Fadi Zaraket, Tymaa Hammouda, Daanish Masood Alavi, and Martin Waahlisch. 2023c. Lisan: Yemeni, irqi, libyan, and sudanese arabic dialect copora with morphological annotations. In The 20th IEEE/ACS International Conference on Computer Systems and Applications (AICCSA). IEEE. [find](https://scholar.google.com/scholar?q=Mustafa+Jarrar%2C+Fadi+Zaraket%2C+Tymaa+Hammouda%2C+Daanish+Masood+Alavi%2C+and+Martin+Waahlisch%2E+2023c%2E+Lisan%3A+Yemeni%2C+irqi%2C+libyan%2C+and+sudanese+arabic+dialect+copora+with+morphological+annotations%2E+In+The+)
28. Jacob Devlin Ming-Wei Chang Kenton and Lee Kristina Toutanova. 2019. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of NAACL-HLT, pages 4171–4186.
29. Svetlana Kiritchenko and Saif M. Mohammad. 2017. [Best-worst scaling more reliable than rating scales: A case study on sentiment intensity annotation](https://md.research.elamur.ai/free/en/papers/kiritchenko-2017-best-worst-scaling-more-reliable/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ACL 2017, Vancouver, Canada, July 30 - August 4, Volume 2: Short Papers, pages 465–470. Association for Computational Linguistics.
30. Xiaotao Li, Shujuan You, and Wai Chen. 2021. Enhancing accuracy of semantic relatedness measurement by word single-meaning embeddings. IEEE Access, 9:117424–117433. [↗](https://doi.org/10.1109/access.2021.3107445)
31. Guan-Ting Lin and Manuel Giambi. 2021. Context-gloss augmentation for improving word sense disambiguation. arXiv preprint arXiv:2110.07174, abs/2110.07174. [↗](https://arxiv.org/abs/2110.07174)
32. Haneen Liqreina, Mustafa Jarrar, Mohammed Khalilia, Ahmed Oumar El-Shangiti, and Muhammad Abdul-Mageed. 2023. [Arabic fine-grained entity recognition](https://md.research.elamur.ai/free/en/papers/liqreina-2023-arabic-fine-grained-entity-recognition/). In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 310–323. ACL.
33. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized BERT pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). CoRR, abs/1907.11692. [↗](https://arxiv.org/abs/1907.11692)
34. Sanad Malaysha, Mustafa Jarrar, and Mohammed Khalilia. 2023. [Context-gloss augmentation for improving arabic target sense verification](https://md.research.elamur.ai/free/en/papers/malaysha-2023-context-gloss-augmentation-improving-arabic/). In Proceedings of the 12th Global Wordnet Conference, GWC 2023, University of the Basque Country, Donostia -San Sebastian, Basque Country, Spain, 23 - 27 January 2023, pages 254–262. Global Wordnet Association.
35. Chenggang Mi and Shaoliang Xie. 2024. Language relatedness evaluation for multilingual neural machine translation. Neurocomputing, 570:127115. [↗](https://doi.org/10.1016/j.neucom.2023.127115)
36. Tomás Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013. [Efficient estimation of word representations in vector space](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/). In 1st International Conference on Learning Representations, ICLR 2013, Scottsdale, Arizona, USA, May 2-4, 2013, Workshop Track Proceedings. [↗](https://doi.org/10.48550/arxiv.1301.3781)
37. Jeremy Miles. 2005. R-squared, adjusted r-squared. Encyclopedia of statistics in behavioral science. [↗](https://doi.org/10.1002/0470013192.bsa526)
38. George A Miller and Walter G Charles. 1991. Contextual correlates of semantic similarity. Language and cognitive processes, 6(1):1–28. [↗](https://doi.org/10.1080/01690969108406936)
39. Sewon Min, Xinxi Lyu, Ari Holtzman, Mikel Artetxe, Mike Lewis, Hannaneh Hajishirzi, and Luke Zettlemoyer. 2022. [Rethinking the role of demonstrations: What makes in-context learning work?](https://md.research.elamur.ai/free/en/papers/min-2022-rethinking-role-demonstrations-what-makes/) In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, EMNLP 2022, Abu Dhabi, United Arab Emirates, December 7-11, 2022, pages 11048–11064. Association for Computational Linguistics.
40. Usman Naseem, Imran Razzak, Shah Khalid Khan, and Mukesh Prasad. 2021. A comprehensive survey on word representation models: From classical to state-of-the-art word representation language models. ACM Trans. Asian Low Resour. Lang. Inf. Process., 20(5):74:1–74:35. [↗](https://doi.org/10.1145/3434237)
41. Amal Nayouf, Mustafa Jarrar, Fadi zaraket, Tymaa Hammouda, and Mohamad-Bassam Kurdy. 2023. [Nâbra: Syrian arabic dialects with morphological annotations](https://md.research.elamur.ai/free/en/papers/nayouf-2023-nabra-syrian-arabic-dialects-morphological/). In Proceedings of the 1st Arabic Natural Language Processing Conference (ArabicNLP), Part of the EMNLP 2023, pages 12–23. ACL.
42. Nedjma Ousidhoum, Shamsuddeen Hassan Muhammad, Mohamed Abdalla, Idris Abdulmumin, Ibrahim Said Ahmad, Sanchit Ahuja, Alham Fikri Aji, Vladimir Araujo, Abinew Ali Ayele, Pavan Baswani, Meriem Beloucif, Chris Biemann, Sofia Bourhim, Christine De Kock, Genet Shanko Dekebo, Oumaima Hourrane, Gopichand Kanumolu, Lokesh Madasu, Samuel Rutunda, Manish Shrivastava, Thamar Solorio, Nirmal Surange, Hailegnaw Getaneh Tilaye, Krishnapriya Vishnubhotla, Genta Winata, Seid Muhie Yimam, and Saif M. Mohammad. 2024a. Semrel2024: A collection of semantic textual relatedness datasets for 14 languages. [find](https://scholar.google.com/scholar?q=Nedjma+Ousidhoum%2C+Shamsuddeen+Hassan+Muhammad%2C+Mohamed+Abdalla%2C+Idris+Abdulmumin%2C+Ibrahim+Said+Ahmad%2C+Sanchit+Ahuja%2C+Alham+Fikri+Aji%2C+Vladimir+Araujo%2C+Abinew+Ali+Ayele%2C+Pavan+Baswani%2C+Meriem+Beloucif%2C)
43. Nedjma Ousidhoum, Shamsuddeen Hassan Muhammad, Mohamed Abdalla, Idris Abdulmumin, Ibrahim Said Ahmad, Sanchit Ahuja, Alham Fikri Aji, Vladimir Araujo, Meriem Beloucif, Christine De Kock, Oumaima Hourrane, Manish Shrivastava, Thamar Solorio, Nirmal Surange, Krishnapriya Vishnubhotla, Seid Muhie Yimam, and Saif M. Mohammad. 2024b. SemEval-2024 task 1: Semantic textual relatedness. In Proceedings of the 18th International Workshop on Semantic Evaluation (SemEval-2024). [find](https://scholar.google.com/scholar?q=Nedjma+Ousidhoum%2C+Shamsuddeen+Hassan+Muhammad%2C+Mohamed+Abdalla%2C+Idris+Abdulmumin%2C+Ibrahim+Said+Ahmad%2C+Sanchit+Ahuja%2C+Alham+Fikri+Aji%2C+Vladimir+Araujo%2C+Meriem+Beloucif%2C+Christine+De+Kock%2C+Oumaima+Hourr)
44. Rakia Saidi, Fethi Jarray, Jeongwoo Jay Kang, and Didier Schwab. 2022. GPT-2 contextual data augmentation for word sense disambiguation. In Proceedings of the 36th Pacific Asia Conference on Language, Information and Computation, PACLIC 2022, Manila, Philippines, October 20-22, 2022, pages 455–462. De La Salle University. [find](https://scholar.google.com/scholar?q=Rakia+Saidi%2C+Fethi+Jarray%2C+Jeongwoo+Jay+Kang%2C+and+Didier+Schwab%2E+2022%2E+GPT%2D2+contextual+data+augmentation+for+word+sense+disambiguation%2E+In+Proceedings+of+the+36th+Pacific+Asia+Conference+on+Language%2C)
45. George Tsatsaronis, Iraklis Varlamis, and Michalis Vazirgiannis. 2014. Text relatedness based on a word thesaurus. CoRR, abs/1401.5699. [↗](https://arxiv.org/abs/1401.5699)
46. Peter D. Turney, Yair Neuman, Dan Assaf, and Yohai Cohen. 2011. [Literal and metaphorical sense identification through concrete and abstract context](https://md.research.elamur.ai/free/en/papers/turney-2011-literal-metaphorical-sense-identification-through/). In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, EMNLP 2011, 27-31 July 2011, John McIntyre Conference Centre, Edinburgh, UK, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 680–690. ACL.
47. Linrui Zhang and Dan Moldovan. 2019. Multi-task learning for semantic relatedness and textual entailment. Journal of Software Engineering and Applications, 12(6):199–214. [↗](https://doi.org/10.4236/jsea.2019.126012)
48. Shuai Zhao, Tianyu Zhang, Man Hu, Wen Chang, and Fucheng You. 2022. AP-BERT: enhanced pre-trained model through average pooling. Appl. Intell., 52(14):15929–15937. [↗](https://doi.org/10.1007/s10489-022-03190-3)
49. Jeffrey Pennington, Richard Socher, and Christopher D. Manning. 2014. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, EMNLP 2014, October 25-29, 2014, Doha, Qatar, A meeting of SIGDAT, a Special Interest Group of the ACL, pages 1532–1543. ACL.
50. Nils Reimers and Iryna Gurevych. 2019. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pages 3980–3990. Association for Computational Linguistics.
51. Ahmed Sabir, Francesc Moreno, and Lluís Padró. 2019. Semantic relatedness based re-ranker for text spotting. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, China, November 3-7, 2019, pages 3449–3455. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1909.07950)

*Updated 1 November 2026*