---
title: "LuxEmbedder — paper by Philippy et al. (2025)"
description: "The paper addresses weak sentence embeddings for Luxembourgish, a West-Germanic language with… 19 references, cited by 8 papers here."
authors: ["Fred Philippy", "Siwen Guo", "Jacques Klein", "Tegawende Bissyande"]
year: 2025
venue: "Proceedings of the 31st International Conference on Computational Linguistics"
source: "https://aclanthology.org/2025.coling-main.753/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/philippy-2025-luxembedder-cross-lingual-approach-enhanced/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/philippy-2025-luxembedder-cross-lingual-approach-enhanced/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/10/) / LuxEmbedder: A Cross-Lingual Approach to Enhanced Luxembourgish Sentence Embeddings

# LuxEmbedder: A Cross-Lingual Approach to Enhanced Luxembourgish Sentence Embeddings

**Authors:** [Fred Philippy](https://md.research.elamur.ai/free/en/authors/fred-philippy/), [Siwen Guo](https://md.research.elamur.ai/free/en/authors/siwen-guo/), [Jacques Klein](https://md.research.elamur.ai/free/en/authors/jacques-klein/) and [Tegawende Bissyande](https://md.research.elamur.ai/free/en/authors/tegawende-bissyande/) · **Year:** 2025 · **Venue:** [Proceedings of the 31st International Conference on Computational Linguistics](https://md.research.elamur.ai/free/en/venues/coling/) ([COLING 2025](https://md.research.elamur.ai/free/en/venues/coling/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.coling-main.753/) · [PDF](https://aclanthology.org/2025.coling-main.753.pdf) · [OpenAlex](https://openalex.org/W4405090284)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [8](https://md.research.elamur.ai/free/en/papers/philippy-2025-luxembedder-cross-lingual-approach-enhanced/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses weak sentence embeddings for Luxembourgish, a West-Germanic language with roughly 400,000 speakers. Cross-lingual embedding models depend on parallel text, and Luxembourgish has very little. Machine translation and generative LLMs are too weak in this language to produce synthetic pairs. The authors therefore built LuxAlign, a human-written parallel corpus mined from the trilingual news site RTL.lu. They first matched articles across languages, then extracted sentence pairs using LaBSE. Fine-tuning LaBSE on this corpus produced LuxEmbedder. They also released ParaLux, a small paraphrase-detection benchmark whose hard negatives were drafted by an LLM and then repaired by a human. LuxEmbedder leads all open-source models on four tasks and beats the proprietary Cohere and OpenAI models on three of them. A separate alignment study suggests that putting a low-resource language into parallel training data improves alignment for other low-resource languages too, more than training on high-resource pairs alone. This makes a case for collecting parallel data for small languages directly.

## Key points
- **LuxAlign** has 25,996 LB-EN and 86,293 LB-FR sentence pairs. Articles were matched across languages with OpenAI text-embedding-3-small, and sentences were then aligned with LaBSE. **ParaLux** has 312 items, each an anchor sentence, a paraphrase, and an adversarial negative.
- **Training setup:** the base model is LaBSE, trained with a contrastive loss for 3 epochs. Batch size was 16 and the learning rate a constant 1e-6. Negatives are built by matching a Luxembourgish sentence with the translated version of some other randomly chosen sentence. The best checkpoint was chosen on a 1% dev split.
- **Evaluation tasks:**
  - zero-shot classification on SIB-200, using label templates;
  - cross-lingual transfer from six high-resource languages to the Luxembourgish SIB-200 test set;
  - Tatoeba bitext mining for LB-EN, LB-NL and LB-DE, using the Tatoeba Translation Challenge test sets;
  - ParaLux.
- **Results against the base model:**
  - On the two monolingual tasks, LaBSE's accuracies of 43.24 and 38.14 rise to 65.59 and 52.24.
  - Cross-lingual transfer goes from 80.88 to 83.39. Only text-embedding-3-large scores higher on this task, at 86.25.
- **Alignment study:** alignment was measured with CKA on Flores-200 after fine-tuning LaBSE on 20,000 pairs each of LB-EN, LB-FR, or EN-FR. The 10 languages with the most and least LaBSE training data served as the high- and low-resource groups. High-resource alignment improved similarly in all three cases, while low-resource alignment improved more when Luxembourgish was in the training pairs.

## Interesting ideas
- **Two-stage mining from a multilingual publisher:** match whole documents across language editions by embedding similarity first, then align sentences only inside matched pairs, which yields clean human-written parallel text without any translation system.
- **Running the parallel-mining pipeline on a single language:** sentences inside one article that a cross-lingual aligner scores as near-equivalent become paraphrase candidates, ready for manual selection.
- **LLM-drafted, human-corrected hard negatives:** a generator proposes minimally altered non-paraphrases and a person fixes them, giving a cheap benchmark of near-miss distractors.
- **Averaging zero-shot results over several label templates:** scoring with five different template wordings removes dependence on one lucky phrasing.
- **Measuring spillover with a representation-similarity metric:** CKA on parallel data across held-out language groups shows how training on one pair affects languages that were never trained on.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (back): The same document-then-sentence mining over bilingual sources could build checked sentence pairs for testing translation through the meta-world.
- **I2** (frames): Such in-article paraphrase pairs make a natural test that two different surface forms produce the same who-did-what-to-whom frames.
- **I3** (learn): Near-miss sentences that change one role or fact could serve as negative controls, checking that Mova's answers actually shift when the meaning changes.
- **I4** (answer): School-test evaluation could pose each question in several paraphrased forms and report the mean, exposing readers that hinge on exact wording.
- **I5** (learn): A similar before-and-after comparison on untouched treebank genres or dialects could reveal whether a new training batch quietly helps or harms other areas.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Luxembourg](https://md.research.elamur.ai/free/en/universities/university-of-luxembourg/) — University, Luxembourg, Luxembourg · 28 works · 337 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 882 works

## Cite

Cite as: Fred Philippy, Siwen Guo, Jacques Klein and Tegawende Bissyande. 2025. LuxEmbedder: A Cross-Lingual Approach to Enhanced Luxembourgish Sentence Embeddings. Proceedings of the 31st International Conference on Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{philippy2025luxembedder,
  title = {{LuxEmbedder: A Cross-Lingual Approach to Enhanced Luxembourgish Sentence Embeddings}},
  author = {Fred Philippy, Siwen Guo, Jacques Klein and Tegawende Bissyande},
  booktitle = {Proceedings of the 31st International Conference on Computational Linguistics},
  year = {2025},
  url = {https://aclanthology.org/2025.coling-main.753/}
}
```

- [← previous LogicPro: Improving Complex Logical Reasoning via Program-Guided Learning](https://md.research.elamur.ai/free/en/papers/jiang-2025-logicpro-improving-complex-logical-reasoning/)
- [next → M2RC-EVAL: Massively Multilingual Repository-level Code Completion Evaluation](https://md.research.elamur.ai/free/en/papers/liu-2025-m2rc-eval-massively-multilingual-repository/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/10/)

## References

19 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. David Adelani, Hannah Liu, Xiaoyu Shen, Nikita Vassilyev, Jesujoba Alabi, Yanke Mao, Haonan Gao, and En-Shiun Lee. 2024. [SIB-200: A Simple, Inclusive, and Big Evaluation Dataset for Topic Classification in 200+ Languages and Dialects](https://md.research.elamur.ai/free/en/papers/adelani-2023-sib-200-simple-inclusive-big/). In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 226–245, St. Julian’s, Malta. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2309.07445)
2. Mikel Artetxe and Holger Schwenk. 2019. [Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond](https://md.research.elamur.ai/free/en/papers/artetxe-2018-massively-multilingual-sentence-embeddings-zero/). Transactions of the Association for Computational Linguistics, 7:597– 610. [↗](https://doi.org/10.1162/tacl_a_00288)
3. Laurie Burchell, Alexandra Birch, Nikolay Bogoychev, and Kenneth Heafield. 2023. [An Open Dataset and Model for Language Identification](https://md.research.elamur.ai/free/en/papers/burchell-2023-open-dataset-model-language-identification/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 865–879, Toronto, Canada. Association for Computational Linguistics.
4. Muthu Chidambaram, Yinfei Yang, Daniel Cer, Steve Yuan, Yunhsuan Sung, Brian Strope, and Ray Kurzweil. 2019. Learning Cross-Lingual Sentence Representations via a Multi-task Dual-Encoder Model. In Proceedings of the 4th Workshop on Representation Learning for NLP (RepL4NLP-2019), pages 250–259, Florence, Italy. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Muthu+Chidambaram%2C+Yinfei+Yang%2C+Daniel+Cer%2C+Steve+Yuan%2C+Yunhsuan+Sung%2C+Brian+Strope%2C+and+Ray+Kurzweil%2E+2019%2E+Learning+Cross%2DLingual+Sentence+Representations+via+a+Multi%2Dtask+Dual%2DEncoder+Model%2E+In+Pro)
5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
6. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022. [Language-agnostic BERT Sentence Embedding](https://md.research.elamur.ai/free/en/papers/feng-2020-language-agnostic-bert-sentence-embedding/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 878–891, Dublin, Ireland. Association for Computational Linguistics.
7. Kevin Heffernan, Onur Çelebi, and Holger Schwenk. 2022. [Bitext Mining Using Distilled Sentence Representations for Low-Resource Languages](https://md.research.elamur.ai/free/en/papers/heffernan-2022-bitext-mining-distilled-sentence-representations/). In Findings of the Association for Computational Linguistics: EMNLP 2022, pages 2101–2112, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics.
8. Simon Kornblith, Mohammad Norouzi, Honglak Lee, and Geoffrey Hinton. 2019. [Similarity of Neural Network Representations Revisited](https://md.research.elamur.ai/free/en/papers/kornblith-2019-similarity-neural-network-representations-revisited/). In Proceedings of the 36th International Conference on Machine Learning, pages 3519–3529. PMLR. We are grateful to RTL Luxembourg for providing the raw data necessary for our research. Their support significantly facilitated our efforts. 11373 [↗](https://doi.org/10.48550/arxiv.1905.00414)
9. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). In Advances in Neural Information Processing Systems, volume 33, pages 9459– 9474. Curran Associates, Inc.
10. Cedric Lothritz, Bertrand Lebichot, Kevin Allix, Lisa Veiber, Tegawende Bissyande, Jacques Klein, Andrey Boytsov, Clément Lefebvre, and Anne Goujon. 2022. LuxemBERT: Simple and Practical Data Augmentation in Language Model Pre-Training for Luxembourgish. In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 5080–5089, Marseille, France. European Language Resources Association. [find](https://scholar.google.com/scholar?q=Cedric+Lothritz%2C+Bertrand+Lebichot%2C+Kevin+Allix%2C+Lisa+Veiber%2C+Tegawende+Bissyande%2C+Jacques+Klein%2C+Andrey+Boytsov%2C+Cl%C3%A9ment+Lefebvre%2C+and+Anne+Goujon%2E+2022%2E+LuxemBERT%3A+Simple+and+Practical+Data+Augmenta)
11. Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. [MTEB: Massive text embedding benchmark](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-mteb-massive-text-embedding-benchmark/). In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014–2037, Dubrovnik, Croatia. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2023.eacl-main.148)
12. NLLB Team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman, Semarley Jarrett, Kaushik Ram Sadagopan, Dirk Rowe, Shannon Spruit, Chau Tran, Pierre Andrews, Necip Fazil Ayan, Shruti Bhosale, Sergey Edunov, Angela Fan, Cynthia Gao, Vedanuj Goswami, Francisco Guzmán, Philipp Koehn, Alexandre Mourachko, Christophe Ropers, Safiyyah Saleem, Holger Schwenk, and Jeff Wang. 2022. No Language Left Behind: Scaling Human-Centered Machine Translation. arXiv preprint. ArXiv:2207.04672 [cs]. [↗](https://arxiv.org/abs/2207.04672)
13. Nils Reimers and Iryna Gurevych. 2020. [Making Monolingual Sentence Embeddings Multilingual using Knowledge Distillation](https://md.research.elamur.ai/free/en/papers/reimers-2020-making-monolingual-sentence-embeddings-multilingual/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 4512–4525, Online. Association for Computational Linguistics.
14. Yewei Song, Saad Ezzini, Jacques Klein, Tegawende Bissyande, Clément Lefebvre, and Anne Goujon. 2023. Letz Translate: Low-Resource Machine Translation for Luxembourgish. In 2023 5th International Conference on Natural Language Processing (IC-NLP), pages 165–170. [find](https://scholar.google.com/scholar?q=Yewei+Song%2C+Saad+Ezzini%2C+Jacques+Klein%2C+Tegawende+Bissyande%2C+Cl%C3%A9ment+Lefebvre%2C+and+Anne+Goujon%2E+2023%2E+Letz+Translate%3A+Low%2DResource+Machine+Translation+for+Luxembourgish%2E+In+2023+5th+International+Conf)
15. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models. Multilingual MT. In Proceedings of the Fifth Conference on Machine Translation, pages 1174–1182, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Nandan+Thakur%2C+Nils+Reimers%2C+Andreas+R%C3%BCckl%C3%A9%2C+Abhishek+Srivastava%2C+and+Iryna+Gurevych%2E+2021%2E+BEIR%3A+A+Heterogeneous+Benchmark+for+Zero%2Dshot+Evaluation+of+Information+Retrieval+Models%2E+Multilingual+MT%2E+I)
16. Yaushian Wang, Ashley Wu, and Graham Neubig. 2022. English Contrastive Learning Can Learn Universal Cross-lingual Sentence Embeddings. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 9122–9133, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yaushian+Wang%2C+Ashley+Wu%2C+and+Graham+Neubig%2E+2022%2E+English+Contrastive+Learning+Can+Learn+Universal+Cross%2Dlingual+Sentence+Embeddings%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+in+Nat)
17. Yinfei Yang, Daniel Cer, Amin Ahmad, Mandy Guo, Jax Law, Noah Constant, Gustavo Hernandez Abrego, Steve Yuan, Chris Tar, Yun-hsuan Sung, Brian Strope, and Ray Kurzweil. 2020. [Multilingual Universal Sentence Encoder for Semantic Retrieval](https://md.research.elamur.ai/free/en/papers/yang-2020-multilingual-universal-sentence-encoder-semantic/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pages 87–94, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.acl-demos.12)
18. Xinyu Zhang, Nandan Thakur, Odunayo Ogundepo, Ehsan Kamalloo, David Alfonso-Hermelo, Xiaoguang Li, Qun Liu, Mehdi Rezagholizadeh, and Jimmy Lin. 2023. [MIRACL: A multilingual retrieval dataset covering 18 diverse languages](https://md.research.elamur.ai/free/en/papers/zhang-2023-miracl-multilingual-retrieval-dataset-covering/). Transactions of the Association for Computational Linguistics, 11:1114–1131. [↗](https://doi.org/10.1162/tacl_a_00595)
19. Zhong Zhou, Matthias Sperber, and Alexander Waibel. 2018. Massively Parallel Cross-Lingual Learning in Low-Resource Target Language Translation. In Proceedings of the Third Conference on Machine Translation: Research Papers, pages 232–243, Brussels, Belgium. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zhong+Zhou%2C+Matthias+Sperber%2C+and+Alexander+Waibel%2E+2018%2E+Massively+Parallel+Cross%2DLingual+Learning+in+Low%2DResource+Target+Language+Translation%2E+In+Proceedings+of+the+Third+Conference+on+Machine+Trans)

*Updated 1 November 2026*