---
title: "Bilingual BSARD: Extending Statutory Article Retrieval to Dutch"
description: "The authors start from BSARD, an existing French resource of citizen legal questions paired with… 44 references, cited by 5 papers here."
authors: ["Ehsan Lotfi", "Nikolay Banar", "Nerses Yuzbashyan", "Walter Daelemans"]
year: 2025
venue: "Proceedings of the 1st Regulatory NLP Workshop (RegNLP 2025)"
source: "https://aclanthology.org/2025.regnlp-1.3/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/lotfi-2025-bilingual-bsard-extending-statutory-article/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lotfi-2025-bilingual-bsard-extending-statutory-article/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/11/) / Bilingual BSARD: Extending Statutory Article Retrieval to Dutch

# Bilingual BSARD: Extending Statutory Article Retrieval to Dutch

**Authors:** [Ehsan Lotfi](https://md.research.elamur.ai/free/en/authors/ehsan-lotfi/), [Nikolay Banar](https://md.research.elamur.ai/free/en/authors/nikolay-banar/), Nerses Yuzbashyan and [Walter Daelemans](https://md.research.elamur.ai/free/en/authors/walter-daelemans/) · **Year:** 2025 · **Venue:** Proceedings of the 1st Regulatory NLP Workshop (RegNLP 2025) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.regnlp-1.3/) · [PDF](https://aclanthology.org/2025.regnlp-1.3.pdf) · [OpenAlex](https://openalex.org/W4405263034)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [5](https://md.research.elamur.ai/free/en/papers/lotfi-2025-bilingual-bsard-extending-statutory-article/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses the lack of a Dutch benchmark for statutory article retrieval in multilingual jurisdictions such as Belgium. The authors start from BSARD, an existing French resource of citizen legal questions paired with Belgian law articles, and build bBSARD, a bilingual French–Dutch version. They scraped the Dutch and French versions of the same articles from Justel, the official Belgian legislation database, and checked that both sides came from the same legislative version. They then translated the questions into Dutch with GPT-4o, giving the model the relevant articles as context, and had a native speaker review a sample. The resulting parallel corpus was used to benchmark lexical retrievers, zero-shot dense embedders and fine-tuned small encoders in both languages. BM25 remains a strong baseline. Commercial embedding models lead in the zero-shot setting, but fine-tuned small language-specific models can match or beat them. Scores are generally higher for French than for Dutch. The resource gives Dutch legal retrieval and RAG work a reliable evaluation set.

## Key points
- 22,417 of the 22,633 BSARD articles (99%) were aligned across French and Dutch from 52 Justel pages per language. Most of the 216 missing articles come from the Walloon Code of Environment (126) and the Military Penal Code (66). Only one missing article is cited by a question.
- There are 1,108 questions (886 train / 222 test), translated with GPT-4o at temperature 0. In a 100-question human check, 2% had legally inaccurate wording and 6% had minor issues such as overly literal phrasing.
- Articles average about 143 words (French) and 142 words (Dutch); questions average about 15 and 14 words. 1,611 distinct articles are relevant, and the median is two relevant articles per question.
- voyage-3 and embedding-3-large are the best zero-shot models. Fine-tuned RobBERT-2023 base surpasses them on MAP, MRR and nDCG for Dutch, and FlauBERT base is the top fine-tuned model for French. Language-specific models clearly beat multilingual XLM-Roberta.
- The dataset is released as clips/bBSARD on HuggingFace (cc-by-nc-sa-4.0), and the evaluation code is on GitHub (MIT).

## Interesting ideas
- The two language sides were aligned only after manually confirming that both came from the same enforcement date, so the pairs are true translations of one version and not drifting texts.
- The LLM translator received the gold relevant articles as context and ran at temperature 0, which steers terminology toward the domain and makes outputs reproducible.
- A small random sample of machine output was audited by a native speaker and labeled with graded error types (legally wrong vs minor), which yields an honest error rate for the whole set.
- BM25 is kept as a serious baseline, and it beats many zero-shot dense models, showing that a simple lexical method can expose an overrated sophisticated system.
- A parallel benchmark lets the same questions be scored in two languages, isolating language and resource effects. Mixed French+Dutch fine-tuning (50+50 epochs) also helped XLM-Roberta on Dutch more than Dutch-only training.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (back): When building parallel or paraphrase pairs to test text → graph → text, check that both sides describe the same source state, so that differences in the output reflect the system and not changes in the data.
- **I2** (learn): LLM teacher tiers could be given the supporting passage or gold frames when producing silver data, run deterministically, so that annotations stay consistent and can be regenerated.
- **I3** (learn): Silver treebanks or frame annotations could be audited the same way, with a fixed-size random sample sorted into severity categories, to estimate noise before training on them.
- **I4** (answer): Every QA reader evaluation could include a shallow lexical-overlap baseline as a negative control, so that gains are shown to come from the tree, frames and world reasoning rather than surface word matching.
- **I5** (tree): gparse could be trained on a mix of a well-resourced treebank and a smaller UD dialect or converted treebank, then compared with single-source training on parallel test sentences to measure the transfer.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Antwerp](https://md.research.elamur.ai/free/en/universities/university-of-antwerp/) — University, Antwerp, Belgium · 17 works · 3,490 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works · [🤝 Political Science and International Relations](https://md.research.elamur.ai/free/en/shelves/social-sciences/political-science-and-international-relations/) · 182 works

🧭 Topics (2): [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 473 works · [⚖️ Legal AI](https://md.research.elamur.ai/free/en/topics/legal-ai/) · 134 works

## Cite

Cite as: Ehsan Lotfi, Nikolay Banar, Nerses Yuzbashyan and Walter Daelemans. 2025. Bilingual BSARD: Extending Statutory Article Retrieval to Dutch. Proceedings of the 1st Regulatory NLP Workshop (RegNLP 2025).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{lotfi2025bilingual,
  title = {{Bilingual BSARD: Extending Statutory Article Retrieval to Dutch}},
  author = {Ehsan Lotfi, Nikolay Banar, Nerses Yuzbashyan and Walter Daelemans},
  booktitle = {Proceedings of the 1st Regulatory NLP Workshop (RegNLP 2025)},
  year = {2025},
  url = {https://aclanthology.org/2025.regnlp-1.3/}
}
```

- [← previous Beyond Gold Standards: Epistemic Ensemble of LLM Judges for Formal Mathematical Reasoning](https://md.research.elamur.ai/free/en/papers/zhang-2025-gold-standards-epistemic-ensemble-llm/)
- [next → BiMediX2 : Bio-Medical EXpert LMM for Diverse Medical Modalities](https://md.research.elamur.ai/free/en/papers/mullappilly-2025-bimedix2-bio-medical-expert-lmm/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/11/)

## References

44 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Charu C Aggarwal and ChengXiang Zhai. 2012. A survey of text clustering algorithms. Mining text data, pages 77–128. [↗](https://doi.org/10.1007/978-1-4614-3223-4_4)
2. Paheli Bhattacharya, Kripabandhu Ghosh, Saptarshi Ghosh, Arindam Pal, Parth Mehta, Arnab Bhattacharya, and Prasenjit Majumder. 2019. Fire 2019 aila track: Artificial intelligence for legal assistance. In Proceedings of the 11th annual meeting of the forum for information retrieval evaluation, pages 4–6. Paheli Bhattacharya, Parth Mehta, Kripabandhu Ghosh, Saptarshi Ghosh, Arindam Pal, Arnab Bhattacharya, 282 GB and 270 GB before filtering/cleaning. [find](https://scholar.google.com/scholar?q=Paheli+Bhattacharya%2C+Kripabandhu+Ghosh%2C+Saptarshi+Ghosh%2C+Arindam+Pal%2C+Parth+Mehta%2C+Arnab+Bhattacharya%2C+and+Prasenjit+Majumder%2E+2019%2E+Fire+2019+aila+track%3A+Artificial+intelligence+for+legal+assistance%2E)
3. and Prasenjit Majumder. 2021. Fire 2020 aila track: Artificial intelligence for legal assistance. In Proceedings of the 12th Annual Meeting of the Forum for Information Retrieval Evaluation, FIRE ’20, page 1–3, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=and+Prasenjit+Majumder%2E+2021%2E+Fire+2020+aila+track%3A+Artificial+intelligence+for+legal+assistance%2E+In+Proceedings+of+the+12th+Annual+Meeting+of+the+Forum+for+Information+Retrieval+Evaluation%2C+FIRE+%E2%80%9920%2C)
4. Randy Goebel, Yoshinobu Kano, Mi-Young Kim, Juliano Rabelo, Ken Satoh, and Masaharu Yoshioka. 2024. Overview and discussion of the competition on legal information, extraction/entailment (coliee) 2023. The Review of Socionetwork Strategies, 18(1):27–47. [↗](https://doi.org/10.1007/s12626-023-00152-0)
5. Piotr Bojanowski, Edouard Grave, Armand Joulin, and Tomas Mikolov. 2017. [Enriching word vectors with subword information](https://md.research.elamur.ai/free/en/papers/bojanowski-2017-enriching-word-vectors-subword-information/). Transactions of the association for computational linguistics, 5:135–146. [↗](https://doi.org/10.1162/tacl_a_00051)
6. Édouard Grave, Piotr Bojanowski, Prakhar Gupta, Armand Joulin, and Tomáš Mikolov. 2018. [Learning word vectors for 157 languages](https://md.research.elamur.ai/free/en/papers/grave-2018-learning-word-vectors-157-languages/). In Proceedings of the Eleventh International Conference on Language Resources and Evaluation (LREC 2018).
7. Marius Büttner and Ivan Habernal. 2024. Answering legal questions from laymen in german civil law system. In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2015–2027. [↗](https://doi.org/10.48448/4j9q-9x69)
8. Matthew Henderson, Rami Al-Rfou, Brian Strope, Yun-Hsuan Sung, László Lukács, Ruiqi Guo, Sanjiv Kumar, Balint Miklos, and Ray Kurzweil. 2017. Efficient natural language response suggestion for smart reply. arXiv preprint arXiv:1705.00652. [↗](https://arxiv.org/abs/1705.00652)
9. Gautier Izacard, Mathilde Caron, Lucas Hosseini, Sebastian Riedel, Piotr Bojanowski, Armand Joulin, and Edouard Grave. 2022. [Unsupervised dense information retrieval with contrastive learning](https://md.research.elamur.ai/free/en/papers/izacard-2021-unsupervised-dense-information-retrieval-contrastive/). Transactions on Machine Learning Research. [↗](https://doi.org/10.48550/arxiv.2112.09118)
10. Andong Chen, Feng Yao, Xinyan Zhao, Yating Zhang, Changlong Sun, Yun Liu, and Weixing Shen. 2023. Equals: A real-world dataset for legal question answering via reading chinese laws. In Proceedings of the Nineteenth International Conference on Artificial Intelligence and Law, pages 71–80. [find](https://scholar.google.com/scholar?q=Andong+Chen%2C+Feng+Yao%2C+Xinyan+Zhao%2C+Yating+Zhang%2C+Changlong+Sun%2C+Yun+Liu%2C+and+Weixing+Shen%2E+2023%2E+Equals%3A+A+real%2Dworld+dataset+for+legal+question+answering+via+reading+chinese+laws%2E+In+Proceedings+of+)
11. Vladimir Karpukhin, Barlas Oguz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. [Dense passage retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769–6781. [↗](https://doi.org/10.18653/v1/2020.emnlp-main.550)
12. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. [Bge m3-embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation](https://md.research.elamur.ai/free/en/papers/chen-2024-m3-embedding-multi-linguality/). Preprint, arXiv:2402.03216. [↗](https://arxiv.org/abs/2402.03216)
13. Mi-Young Kim, Juliano Rabelo, Randy Goebel, Masaharu Yoshioka, Yoshinobu Kano, and Ken Satoh. 2022. Coliee 2022 summary: methods for legal document retrieval and entailment. In JSAI International Symposium on Artificial Intelligence, pages 51–67. Springer. [find](https://scholar.google.com/scholar?q=Mi%2DYoung+Kim%2C+Juliano+Rabelo%2C+Randy+Goebel%2C+Masaharu+Yoshioka%2C+Yoshinobu+Kano%2C+and+Ken+Satoh%2E+2022%2E+Coliee+2022+summary%3A+methods+for+legal+document+retrieval+and+entailment%2E+In+JSAI+International+Symp)
14. Pieter Delobelle and François Remy. 2024. Robbert-2023: Keeping dutch language models up-to-date at a lower cost thanks to model conversion. Computational Linguistics in the Netherlands Journal, 13:193–203. [find](https://scholar.google.com/scholar?q=Pieter+Delobelle+and+Fran%C3%A7ois+Remy%2E+2024%2E+Robbert%2D2023%3A+Keeping+dutch+language+models+up%2Dto%2Ddate+at+a+lower+cost+thanks+to+model+conversion%2E+Computational+Linguistics+in+the+Netherlands+Journal%2C+13%3A19)
15. Hang Le, Loïc Vial, Jibril Frej, Vincent Segonne, Maximin Coavoux, Benjamin Lecouteux, Alexandre Allauzen, Benoît Crabbé, Laurent Besacier, and Didier Schwab. 2020. [Flaubert: Unsupervised language model pre-training for french](https://md.research.elamur.ai/free/en/papers/le-2019-flaubert-unsupervised-language-model-pre/). In Proceedings of The 12th Language Resources and Evaluation Conference, pages 2479–2490, Marseille, France. European Language Resources Association. [↗](https://doi.org/10.5281/zenodo.3562902)
16. Pieter Delobelle, Thomas Winters, and Bettina Berendt. 2020. [RobBERT: a Dutch RoBERTa-based Language Model](https://md.research.elamur.ai/free/en/papers/delobelle-2020-robbert-dutch-roberta-based-language/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 3255–3265, Online. Association for Computational Linguistics.
17. European Union. 2012. Charter of fundamental rights of the european union. [https://eur-lex.europa.eu/legal-content/](https://eur-lex.europa.eu/legal-content/) EN/TXT/?uri=CELEX:12012P/TXT. [Accessed: 5 October 2024]. [↗](https://eur-lex.europa.eu/legal-content/)
18. Chaofan Li, MingHao Qin, Shitao Xiao, Jianlyu Chen, Kun Luo, Yingxia Shao, Defu Lian, and Zheng Liu. 2024. Making text embedders few-shot learners. arXiv preprint arXiv:2409.15700. [↗](https://arxiv.org/abs/2409.15700)
19. Jean-Philippe Fauconnier. 2015. French word embeddings. [find](https://scholar.google.com/scholar?q=Jean%2DPhilippe+Fauconnier%2E+2015%2E+French+word+embeddings%2E)
20. Ilya Loshchilov and Frank Hutter. 2017. Decoupled weight decay regularization. In International Conference on Learning Representations. [↗](https://doi.org/10.48550/arxiv.1711.05101)
21. Fangxiaoyu Feng, Yinfei Yang, Daniel Cer, Naveen Arivazhagan, and Wei Wang. 2022. [Language-agnostic bert sentence embedding](https://md.research.elamur.ai/free/en/papers/feng-2020-language-agnostic-bert-sentence-embedding/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 878–891.
22. Antoine Louis, Vageesh Saxena, Gijs van Dijck, and Gerasimos Spanakis. 2024a. Colbert-xm: A modular multi-vector representation model for zero-shot multilingual information retrieval. arXiv preprint arXiv:2402.15059. [↗](https://arxiv.org/abs/2402.15059)
23. Gregor Geigle, Nils Reimers, Andreas Rücklé, and Iryna Gurevych. 2021. Tweac: transformer with extendable qa agent classifiers. arXiv preprint arXiv:2104.07081. [↗](https://arxiv.org/abs/2104.07081)
24. Antoine Louis and Gerasimos Spanakis. 2022. A statutory article retrieval dataset in french. In Proceedings of the 60th Annual Meeting of the Association for Juliano Rabelo, Mi-Young Kim, Randy Goebel, Masaharu Yoshioka, Yoshinobu Kano, and Ken Satoh. 2021. Coliee 2020: methods for legal document retrieval and entailment. In New Frontiers in Artificial Intelligence: JSAI-isAI 2020 Workshops, JURISIN, LENLS 2020 Workshops, Virtual Event, November 15–17, 2020, Revised Selected Papers 12, pages 196– 210. Springer. Computational Linguistics (Volume 1: Long Papers), pages 6789–6803. [find](https://scholar.google.com/scholar?q=Antoine+Louis+and+Gerasimos+Spanakis%2E+2022%2E+A+statutory+article+retrieval+dataset+in+french%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+Association+for+Juliano+Rabelo%2C+Mi%2DYoung+Kim%2C+Randy+Goebel)
25. Antoine Louis, Gijs van Dijck, and Gerasimos Spanakis. 2024b. [Interpretable long-form legal question answering with retrieval-augmented large language models](https://md.research.elamur.ai/free/en/papers/louis-2023-interpretable-long-form-legal-question/). In Proceedings of the AAAI Conference on Artificial Intelligence, volume 38, pages 22266– 22275.
26. Nils Reimers and Iryna Gurevych. 2019. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Conference on Empirical Methods in Natural Language Processing.
27. Andrew Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and Christopher Potts. 2011. [Learning word vectors for sentiment analysis](https://md.research.elamur.ai/free/en/papers/maas-2011-learning-word-vectors-sentiment-analysis/). In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologies, pages 142–150.
28. François Remy, Pieter Delobelle, Bettina Berendt, Kris Demuynck, and Thomas Demeester. 2023. Tik-totok: Translating language models one token at a time: An embedding initialization strategy for efficient language adaptation. arXiv preprint arXiv:2310.03477. [↗](https://arxiv.org/abs/2310.03477)
29. Louis Martin, Benjamin Muller, Pedro Javier Ortiz Suárez, Yoann Dupont, Laurent Romary, Éric Villemonte de la Clergerie, Djamé Seddah, and Benoît Sagot. 2020. [Camembert: a tasty french language model](https://md.research.elamur.ai/free/en/papers/martin-2019-camembert-tasty-french-language-model/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.
30. Stephen E. Robertson, Steve Walker, Susan Jones, Micheline Hancock-Beaulieu, and Mike Gatford. 1994. [Okapi at TREC-3](https://md.research.elamur.ai/free/en/papers/robertson-1994-okapi-trec-3/). In Proceedings of The Third Text REtrieval Conference, TREC 1994, Gaithersburg, Maryland, USA, November 2-4, 1994, volume 500-225 of NIST Special Publication, pages 109– 126. National Institute of Standards and Technology (NIST).
31. Tomas Mikolov, Kai Chen, Gregory S. Corrado, and Jeffrey Dean. 2013a. [Efficient estimation of word representations in vector space](https://md.research.elamur.ai/free/en/papers/mikolov-2013-efficient-estimation-word-representations-vector/). In International Conference on Learning Representations. [↗](https://doi.org/10.48550/arxiv.1301.3781)
32. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b. [Distributed representations of words and phrases and their compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). Advances in neural information processing systems, 26. [↗](https://doi.org/10.48550/arxiv.1310.4546)
33. Gerard Salton and Chung-Shu Yang. 1973. On the specification of term values in automatic indexing. Journal of documentation, 29(4):351–372. [↗](https://doi.org/10.1108/eb026562)
34. Elvis Saravia, Hsien-Chi Toby Liu, Yen-Hao Huang, Junlin Wu, and Yi-Shin Chen. 2018. [Carer: Contextualized affect representations for emotion recognition](https://md.research.elamur.ai/free/en/papers/saravia-2018-carer-contextualized-affect-representations-emotion/). In Proceedings of the 2018 conference on empirical methods in natural language processing, pages 3687–3697.
35. Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. [Mteb: Massive text embedding benchmark](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-mteb-massive-text-embedding-benchmark/). In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014–2037.
36. OpenAI. 2024. New embedding models and api updates. Accessed: 2024-10-31. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2024%2E+New+embedding+models+and+api+updates%2E+Accessed%3A+2024%2D10%2D31%2E)
37. Karen Sparck Jones. 1972. A statistical interpretation of term specificity and its application in retrieval. Journal of documentation, 28(1):11–21. [↗](https://doi.org/10.1108/eb026526)
38. James O’Neill, Polina Rozenshtein, Ryuichi Kiryo, Motoko Kubota, and Danushka Bollegala. 2021. I wish i would have loved this one, but i didn’t–a multilingual dataset for counterfactual detection in product review. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 7092–7108. [find](https://scholar.google.com/scholar?q=James+O%E2%80%99Neill%2C+Polina+Rozenshtein%2C+Ryuichi+Kiryo%2C+Motoko+Kubota%2C+and+Danushka+Bollegala%2E+2021%2E+I+wish+i+would+have+loved+this+one%2C+but+i+didn%E2%80%99t%E2%80%93a+multilingual+dataset+for+counterfactual+detection+in+p)
39. Saba Sturua, Isabelle Mohr, Mohammad Kalim Akram, Michael Günther, Bo Wang, Markus Krimmel, Feng Wang, Georgios Mastrapas, Andreas Koukounas, Nan Wang, et al. 2024. jina-embeddings-v3: Multilingual embeddings with task lora. arXiv preprint arXiv:2409.10173. [↗](https://arxiv.org/abs/2409.10173)
40. Weihang Su, Yiran Hu, Anzhe Xie, Qingyao Ai, Zibing Que, Ning Zheng, Yun Liu, Weixing Shen, and Yiqun Liu. 2024. Stard: A chinese statute retrieval dataset with real queries issued by non-professionals. arXiv preprint arXiv:2406.15313. [↗](https://arxiv.org/abs/2406.15313)
41. Keqin Peng, Liang Ding, Qihuang Zhong, Li Shen, Xuebo Liu, Min Zhang, Yuanxin Ouyang, and Dacheng Tao. 2023. [Towards making the most of ChatGPT for machine translation](https://md.research.elamur.ai/free/en/papers/peng-2023-making-most-chatgpt-machine-translation/). In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5622–5633, Singapore. Association for Computational Linguistics. [↗](https://doi.org/10.13140/rg.2.2.24416.97283)
42. Nandan Thakur, Nils Reimers, Andreas Rücklé, Abhishek Srivastava, and Iryna Gurevych. 2021. BEIR: A heterogeneous benchmark for zero-shot evaluation of information retrieval models. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track (Round 2). [find](https://scholar.google.com/scholar?q=Nandan+Thakur%2C+Nils+Reimers%2C+Andreas+R%C3%BCckl%C3%A9%2C+Abhishek+Srivastava%2C+and+Iryna+Gurevych%2E+2021%2E+BEIR%3A+A+heterogeneous+benchmark+for+zero%2Dshot+evaluation+of+information+retrieval+models%2E+In+Thirty%2Dfifth+Co)
43. Juliano Rabelo, Randy Goebel, Mi-Young Kim, Yoshinobu Kano, Masaharu Yoshioka, and Ken Satoh. 2022. Overview and discussion of the competition on legal information extraction/entailment (coliee) 2021. The Review of Socionetwork Strategies, 16(1):111– 133. [↗](https://doi.org/10.1007/s12626-022-00105-z)
44. Stéphan Tulkens, Chris Emmery, and Walter Daelemans. 2016. Evaluating unsupervised dutch word [find](https://scholar.google.com/scholar?q=St%C3%A9phan+Tulkens%2C+Chris+Emmery%2C+and+Walter+Daelemans%2E+2016%2E+Evaluating+unsupervised+dutch+word)

*Updated 1 November 2026*