---
title: "Findings of the BEA 2026 Shared Task on Vocabulary Difficulty"
description: "The task offers a public benchmark for L1-aware difficulty estimation, with uses in item writing and… 65 references, cited by 9 papers here."
authors: ["Mariano Felice", "Lucy Skidmore"]
year: 2026
venue: "Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)"
source: "https://aclanthology.org/2026.bea-1.65/"
doi: "10.18653/v1/2026.bea-1.65"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/felice-2026-findings-bea-2026-shared-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/felice-2026-findings-bea-2026-shared-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/2/) / Findings of the BEA 2026 Shared Task on Vocabulary Difficulty Prediction for English Learners

# Findings of the BEA 2026 Shared Task on Vocabulary Difficulty Prediction for English Learners

**Authors:** [Mariano Felice](https://md.research.elamur.ai/free/en/authors/mariano-felice/) and Lucy Skidmore · **Year:** 2026 · **Venue:** [Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)](https://md.research.elamur.ai/free/en/venues/bea/) ([BEA 2026](https://md.research.elamur.ai/free/en/venues/bea/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.bea-1.65/) · [PDF](https://aclanthology.org/2026.bea-1.65.pdf) · [DOI 10.18653/v1/2026.bea-1.65](https://doi.org/10.18653/v1/2026.bea-1.65) · [OpenAlex](https://openalex.org/W7166844970)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [9](https://md.research.elamur.ai/free/en/papers/felice-2026-findings-bea-2026-shared-task/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Predicting how hard an English word is for learners usually depends on costly pretesting and expert judgement. Earlier shared tasks on lexical complexity did not target L2 learners or consider their mother tongue. This BEA 2026 task treats the problem as regression. For each vocabulary test item, which appears in the learner's L1 (Spanish, German or Mandarin), a system must output a psychometric difficulty score for the English target word. The data come from the British Council's Knowledge-based Vocabulary Lists (KVL), where difficulty was calibrated with Rasch-type GLMM models fitted to millions of learner answers. The task had two tracks. The closed track allowed only the provided per-L1 data. The open track placed no restrictions on data or models. Twenty-three teams took part, and RMSE was the official ranking metric. Most systems beat the XLM-RoBERTa-base baseline, and the winners beat it by a wide margin. The strongest systems were ensembles that mixed linguistic features with fine-tuned model outputs. Open-track systems did better, which suggests that training jointly across L1s helps. The task offers a public benchmark for L1-aware difficulty estimation, with uses in item writing and adaptive vocabulary learning.

## Key points
- **Data:** the 'Extended KVL Dataset for NLP' has 6,768 parallel items per L1, or 20,304 items in total.
  - A further 748 items per L1 (2,244 in total) were released as test data.
  - The development set has 677 items per L1.
  - The difficulty scores come from about 3.3 million responses by more than 100,000 learners, modelled with RPRI Rasch models in a GLMM framework. A lower GLMM_score means a harder word.
- **Baseline:** XLM-RoBERTa-base, fine-tuned on an input in the order learners see the item: L1 source word, L1 context, English clue, then the target word.
  - The closed-track baselines were trained per L1.
  - A single open-track baseline was trained on all L1s combined.
- **Participation:** 23 teams from 16 countries.
  - 22 teams entered the closed track (159 runs) and 9 entered the open track (71 runs).
  - 19 teams published system papers, and 17 of these fine-tuned encoder transformers.
  - XLM-RoBERTa-large was used by eight teams.
- **Metrics:** RMSE was primary. Pearson correlation was added because it does not depend on scale, which allows comparison across L1s.
- **Error patterns:** systems tended to underestimate difficulty and found polysemous words hardest.
  - Models transferred reasonably well between Spanish and German.
  - Transfer to and from Mandarin was poor.

## Interesting ideas
- The input sequence was built to follow the order of the real test item as learners see it, so the model reads the item the way a human does.
- Track rules separated using an LLM only to extract features (such as imageability ratings or translations) from using it to produce the prediction itself, which counted as open-track.
- Per-item error analysis exposed polysemy as a main source of failure, and several teams independently flagged the same features (spelling, polysemy, cross-lingual signals).
- The organisers propose checking features that interpretable models rank as important directly against real learner responses, instead of trusting model behaviour alone.
- Typologically close language pairs transferred well while Mandarin did not, so joint multilingual training helps unevenly.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (answer): School-test readers could present each question in the same order a student sees it, then check whether that ordering changes accuracy.
- **I2** (learn): LLM teacher tiers could be labelled in the same way, as feature-only versus label-producing, so evaluations report how much of a result depends on the LLM's own judgements.
- **I3** (world): WordNet sense counts in the compiled ontology could serve as an explainable difficulty or ambiguity signal that tells later stages where errors are most likely.
- **I4** (learn): Each self-induced repair rule could be validated against held-out gold behaviour, not only against the agreement of the witness parsers.
- **I5** (back): Translation through the meta-world should be evaluated separately for each language pair, without assuming that gains on European languages carry over to distant ones.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

Also on the byline: [British Council](https://ror.org/00t3pr326).

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) · 499 works

🧭 Topics (3): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,220 works · [🖍️ Grammatical error correction & learner language](https://md.research.elamur.ai/free/en/topics/grammatical-error-correction/) · 156 works

## Cite

Cite as: Mariano Felice and Lucy Skidmore. 2026. Findings of the BEA 2026 Shared Task on Vocabulary Difficulty Prediction for English Learners. Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{felice2026findings,
  title = {{Findings of the BEA 2026 Shared Task on Vocabulary Difficulty Prediction for English Learners}},
  author = {Mariano Felice and Lucy Skidmore},
  booktitle = {Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.bea-1.65},
  url = {https://doi.org/10.18653/v1/2026.bea-1.65}
}
```

- [← previous FaithLM: Towards Faithful Explanations for Large Language Models](https://md.research.elamur.ai/free/en/papers/chuang-2026-faithlm-faithful-explanations-large-language/)
- [next → HERMES: KV Cache as Hierarchical Memory for Efficient Streaming Video Understanding](https://md.research.elamur.ai/free/en/papers/zhang-2026-hermes-kv-cache-as-hierarchical/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/2/)

## References

65 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Abid Al Hossain and Kamruzzaman Khan Alve. 2026. Failure at BEA 2026 Shared Task 1: One pipeline, three L1s: A unified language-agnostic system for vocabulary difficulty prediction. In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Abid+Al+Hossain+and+Kamruzzaman+Khan+Alve%2E+2026%2E+Failure+at+BEA+2026+Shared+Task+1%3A+One+pipeline%2C+three+L1s%3A+A+unified+language%2Dagnostic+system+for+vocabulary+difficulty+prediction%2E+In+Proceedings+of+)
2. Desislava Aleksandrova and Vincent Pouliot. 2023. CEFR-based contextual lexical complexity classifier in English and French. In Proceedings of the 18th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2023), pages 518–527, Toronto, Canada. Association for Computational Linguistics. [↗](https://doi.org/10.48448/y38k-9m49)
3. David Alfter and Elena Volodina. 2018. Towards single word lexical complexity prediction. In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 79–88, New Orleans, Louisiana. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=David+Alfter+and+Elena+Volodina%2E+2018%2E+Towards+single+word+lexical+complexity+prediction%2E+In+Proceedings+of+the+Thirteenth+Workshop+on+Innovative+Use+of+NLP+for+Building+Educational+Applications%2C+page)
4. Samah AlKhuzaey, Floriana Grasso, Terry R Payne, and Valentina Tamma. 2024. Text-based question difficulty prediction: A systematic review of automatic approaches. International Journal of Artificial Intelligence in Education, 34(3):862–914. [↗](https://doi.org/10.1007/s40593-023-00362-1)
5. Stefano Bannò, Kate M. Knill, and Mark J. F. Gales. 2025. [Exploiting the English vocabulary profile for L2 word-level vocabulary assessment with LLMs](https://md.research.elamur.ai/free/en/papers/banno-2025-exploiting-english-vocabulary-profile-l2/). In Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025), pages 632–646, Vienna, Austria. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2506.02758)
6. Luca Benedetto, Paolo Cremonesi, Andrew Caines, Paula Buttery, Andrea Cappelli, Andrea Giussani, and Roberto Turrin. 2023. A survey on recent approaches to question difficulty estimation from text. ACM Comput. Surv., 55(9). [↗](https://doi.org/10.1145/3556538)
7. Yves Bestgen. 2026. [SATLab at BEA 2026 Shared Task 1: Predicting the difficulty of English words for three L1 learners using primarily psycholinguistic features](https://md.research.elamur.ai/free/en/papers/bestgen-2026-satlab-bea-2026-shared-task/). In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics.
8. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. [BGE M3-Embedding: Multi-lingual, multi-functionality, multi-granularity text embeddings through self-knowledge distillation](https://md.research.elamur.ai/free/en/papers/chen-2024-m3-embedding-multi-linguality/). Preprint, arXiv:2402.03216. [↗](https://arxiv.org/abs/2402.03216)
9. Xiaobin Chen. 2026. TeamXBC at BEA 2026 Shared Task 1: How AI (and I) won the shared task: Vibe and agentic coding solutions for practical machine learning problems. In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Xiaobin+Chen%2E+2026%2E+TeamXBC+at+BEA+2026+Shared+Task+1%3A+How+AI+%28and+I%29+won+the+shared+task%3A+Vibe+and+agentic+coding+solutions+for+practical+machine+learning+problems%2E+In+Proceedings+of+the+21st+Worksho)
10. Seok Hyeon Cho, JunHyeok Choi, Sangeun Ji, and Sung Won Han. 2026. AIDA at BEA 2026 Shared Task 1: A two-stage framework for L1-aware vocabulary difficulty prediction with representation diversity and residual calibration. In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Seok+Hyeon+Cho%2C+JunHyeok+Choi%2C+Sangeun+Ji%2C+and+Sung+Won+Han%2E+2026%2E+AIDA+at+BEA+2026+Shared+Task+1%3A+A+two%2Dstage+framework+for+L1%2Daware+vocabulary+difficulty+prediction+with+representation+diversity+and)
11. Hyung Won Chung, Thibault Févry, Henry Tsai, Melvin Johnson, and Sebastian Ruder. 2021. [Rethinking embedding coupling in pre-trained language models](https://md.research.elamur.ai/free/en/papers/chung-2020-rethinking-embedding-coupling-pre-trained/). In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net. [↗](https://doi.org/10.48550/arxiv.2010.12821)
12. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. Unsupervised cross-lingual representation learning at scale. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440– 8451, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.acl-main.747)
13. Emma Dalbo. 2026. UGA Threshold at BEA 2026 Shared Task 1: Predicting vocabulary acquisition difficulty with hand-crafted SLA-based features. In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Emma+Dalbo%2E+2026%2E+UGA+Threshold+at+BEA+2026+Shared+Task+1%3A+Predicting+vocabulary+acquisition+difficulty+with+hand%2Dcrafted+SLA%2Dbased+features%2E+In+Proceedings+of+the+21st+Workshop+on+Innovative+Use+of+N)
14. Paul De Boeck. 2008. Random item IRT models. Psychometrika, 73(4):533–559. [↗](https://doi.org/10.1007/s11336-008-9092-x)
15. Jasper Degraeuwe. 2025. You shall know a word’s difficulty by the family it keeps: Word family features in personalised word difficulty classifiers for L2 Spanish. In Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025), pages 312–325, Vienna, Austria. Association for Computational Linguistics. [↗](https://doi.org/10.48448/r3ee-3f89)
16. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, and 1 others. 2024. [The LlaMA 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783. [↗](https://arxiv.org/abs/2407.21783)
17. Karen J. Dunn. 2024. Random-item Rasch models and explanatory extensions: A worked example using L2 vocabulary test item responses. Research Methods in Applied Linguistics, 3(3):100143. [↗](https://doi.org/10.1016/j.rmal.2024.100143)
18. Karen J. Dunn, Lucy Skidmore, and Thomas Rogers. 2026. When measurement meets machine learning: interpretability and scalability in modelling item difficulty for language assessment. Frontiers in Education, 11. [↗](https://doi.org/10.3389/feduc.2026.1740237)
19. Bradley Efron and Robert J. Tibshirani. 1993. An Introduction to the Bootstrap. Chapman & Hall, New York. [↗](https://doi.org/10.1007/978-1-4899-4541-9)
20. Thomas François and Núria Gala. 2024. Graded resources for learning and teaching foreign languages: An overview. ITL – International Journal of Applied Linguistics, 175(1):8–24. [↗](https://doi.org/10.1075/itl.00024.fra)
21. Jeroen Geertzen, Theodora Alexopoulou, and Anna Korhonen. 2013. Automatic linguistic annotation of large scale L2 databases: The EF-Cambridge Open Language Database (EFCAMDAT). In Proceedings of the 31st Second Language Research Forum (SLRF), pages 240–254. [find](https://scholar.google.com/scholar?q=Jeroen+Geertzen%2C+Theodora+Alexopoulou%2C+and+Anna+Korhonen%2E+2013%2E+Automatic+linguistic+annotation+of+large+scale+L2+databases%3A+The+EF%2DCambridge+Open+Language+Database+%28EFCAMDAT%29%2E+In+Proceedings+of+the+3)
22. Sian Gooding and Manuel Tragut. 2022. One size does not fit all: The case for personalised word complexity models. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 353–365, Seattle, United States. Association for Computational Linguistics. [↗](https://doi.org/10.48448/vdd7-v591)
23. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. [DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing](https://md.research.elamur.ai/free/en/papers/he-2021-debertav3-improving-deberta-using-electra/). In International Conference on Learning Representations.
24. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. [LoRA: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In International Conference on Learning Representations.
25. Albert Q. Jiang, Alexandre Sablayrolles, Arthur Mensch, and 1 others. 2023. Mistral 7B. arXiv preprint arXiv:2310.06825. [↗](https://arxiv.org/abs/2310.06825)
26. Nouran Khallaf and Serge Sharoff. 2026. UOL@IDEM at BEA 2026 Shared Task 1: Neural fusion and feature-rich modeling for L1-aware vocabulary difficulty prediction. In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2606.24501)
27. Tayyab Latif, Asifa Bibi, Sabur Butt, and Sidorov Grigori. 2026. NLP-Explorers at BEA 2026 Shared Task 1: DeBERTa–CatBoost weighted ensemble approach for L1-specific vocabulary difficulty prediction. In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Tayyab+Latif%2C+Asifa+Bibi%2C+Sabur+Butt%2C+and+Sidorov+Grigori%2E+2026%2E+NLP%2DExplorers+at+BEA+2026+Shared+Task+1%3A+DeBERTa%E2%80%93CatBoost+weighted+ensemble+approach+for+L1%2Dspecific+vocabulary+difficulty+prediction%2E+)
28. Batia Laufer and Zahava Goldstein. 2004. Testing vocabulary knowledge: Size, strength, and computer adaptiveness. Language learning, 54(3):399–436. [↗](https://doi.org/10.1111/j.0023-8333.2004.00260.x)
29. John Lee and Chak Yan Yeung. 2018. Personalizing lexical simplification. In Proceedings of the 27th International Conference on Computational Linguistics, pages 224–232, Santa Fe, New Mexico, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=John+Lee+and+Chak+Yan+Yeung%2E+2018%2E+Personalizing+lexical+simplification%2E+In+Proceedings+of+the+27th+International+Conference+on+Computational+Linguistics%2C+pages+224%E2%80%93232%2C+Santa+Fe%2C+New+Mexico%2C+USA%2E+Ass)
30. Wicaksono Leksono, Joanito Agili Lopo, Tsama-rah R. N. Nugraha, Ahmad Cahyono Adi, and Muhamad Oriza Nurfajri. 2026. [TOEBM at BEA 2026 Shared Task 1: Improving lexical difficulty prediction with context-aligned contrastive learning and ridge ensembling](https://md.research.elamur.ai/free/en/papers/muhamad-2026-toebm-bea-2026-shared-task/). In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics.
31. Zhe Li, Pauline Aguinalde, and Jinnie Shin. 2026. Jinnie’s Lab at BEA 2026 Shared Task 1: Precalibration of vocabulary item difficulty with multilingual transformers and multi-task learning. In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zhe+Li%2C+Pauline+Aguinalde%2C+and+Jinnie+Shin%2E+2026%2E+Jinnie%E2%80%99s+Lab+at+BEA+2026+Shared+Task+1%3A+Precalibration+of+vocabulary+item+difficulty+with+multilingual+transformers+and+multi%2Dtask+learning%2E+In+Procee)
32. Alexander H. Liu, Kartik Khandelwal, Sandeep Subramanian, Victor Jouault, Abhinav Rastogi, Adrien Sadé, Alan Jeffares, Albert Jiang, Alexandre Cahill, Alexandre Gavaudan, Alexandre Sablayrolles, Amélie Héliou, Amos You, Andy Ehrenberg, Andy Lo, Anton Eliseev, Antonia Calvi, Avinash Sooriyarachchi, Baptiste Bout, and 101 others. 2026. Ministral 3. Preprint, arXiv:2601.08584. [↗](https://arxiv.org/abs/2601.08584)
33. Marc Marone, Orion Weller, William Fleshman, Eugene Yang, Dawn Lawrie, and Benjamin Van Durme. 2025. [mmBERT: A modern multilingual encoder with annealed language learning](https://md.research.elamur.ai/free/en/papers/marone-2025-mmbert-modern-multilingual-encoder-annealed/). Preprint, arXiv:2509.06888. [↗](https://arxiv.org/abs/2509.06888)
34. Jonas Mayer Martins, Zhuojing Huang, Aaricia Herygers, and Lisa Beinborn. 2026. [BoostedCats at BEA 2026 Shared Task 1: What makes a word hard to learn? Modeling L1 influence on English vocabulary difficulty](https://md.research.elamur.ai/free/en/papers/martins-2026-boostedcats-bea-2026-shared-task/). In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics.
35. Karthik Mattu, Adit Dhall, Arshad Naguru, Shubh Sehgal, Thejas Nagesh Gowda, and Hakyung Sung. 2026. [find](https://scholar.google.com/scholar?q=Karthik+Mattu%2C+Adit+Dhall%2C+Arshad+Naguru%2C+Shubh+Sehgal%2C+Thejas+Nagesh+Gowda%2C+and+Hakyung+Sung%2E+2026%2E)
36. [SAAKTH at BEA 2026 Shared Task 1: L1-aware English vocabulary difficulty prediction with hybrid transformer and psycholinguistic features](https://md.research.elamur.ai/free/en/papers/mattu-2026-saakth-bea-2026-shared-task/). In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics.
37. Adam Nohejl, Akio Hayakawa, Yusuke Ide, and Taro Watanabe. 2025. A Japanese dataset and efficient multilingual LLM-based methods for Lexical Simplification and Lexical Complexity Prediction. Journal of Natural Language Processing, 32(4):1129–1188. [↗](https://doi.org/10.5715/jnlp.32.1129)
38. Adam Nohejl, Xuanxin Wu, Yusuke Ide, Maria Angelica Riera Machin, and Yi-Ning Chang. 2026. [Sakura at BEA 2026 Shared Task 1: What makes vocabulary difficult?](https://md.research.elamur.ai/free/en/papers/nohejl-2026-sakura-bea-2026-shared-task/) In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics.
39. Kai North, Tharindu Ranasinghe, Matthew Shardlow, and Marcos Zampieri. 2025. Deep learning approaches to lexical simplification: A survey. Journal of Intelligent Information Systems, 63:111–134. [↗](https://doi.org/10.1007/s10844-024-00882-9)
40. Kai North and Marcos Zampieri. 2023. Features of lexical complexity: insights from L1 and L2 speakers. Frontiers in Artificial Intelligence, 6. [↗](https://doi.org/10.3389/frai.2023.1236963)
41. Kai North, Marcos Zampieri, and Matthew Shardlow. 2023. Lexical complexity prediction: An overview. ACM Comput. Surv., 55(9). [↗](https://doi.org/10.1145/3557885)
42. Gustavo Paetzold and Lucia Specia. 2016. [SemEval 2016 task 11: Complex word identification](https://md.research.elamur.ai/free/en/papers/paetzold-2016-semeval-2016-task-11-complex/). In Proceedings of the 10th International Workshop on Semantic Evaluation (SemEval-2016), pages 560–569, San Diego, California. Association for Computational Linguistics.
43. Anubhab Parashar and Sandeep Mathias. 2026. [Token Titans at BEA 2026 Shared Task 1: Multilingual lexical complexity prediction via fine-tuned XLM-RoBERTa with ensemble decoding](https://md.research.elamur.ai/free/en/papers/parashar-2026-token-titans-bea-2026-shared/). In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics.
44. Sydney Peters, Nan Zhang, Hong Jiao, Ming Li, and Tianyi Zhou. 2025. Review of text-based approaches to item difficulty modeling in large-scale assessments. In Proceedings of the Artificial Intelligence in Measurement and Education Conference (AIME-Con): Coordinated Session Papers, pages 37–47, Wyndham Grand Pittsburgh, Downtown, Pittsburgh, Pennsylvania, United States. National Council on Measurement in Education (NCME). [find](https://scholar.google.com/scholar?q=Sydney+Peters%2C+Nan+Zhang%2C+Hong+Jiao%2C+Ming+Li%2C+and+Tianyi+Zhou%2E+2025%2E+Review+of+text%2Dbased+approaches+to+item+difficulty+modeling+in+large%2Dscale+assessments%2E+In+Proceedings+of+the+Artificial+Intelligen)
45. Vassili Philippov, Dmitrii Andreev, Pavel Katunin, and Anton Nikolaev. 2026. [Glite at BEA 2026 Shared Task 1: Holistic difficulty models dominate, feature engineering closes the gap in L1-aware vocabulary difficulty prediction](https://md.research.elamur.ai/free/en/papers/philippov-2026-glite-bea-2026-shared-task/). In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics.
46. Adrian Pineda, Sabur Butt, and Héctor Gibrán Ceballos Cancino. 2026. [Data Asgardians at BEA 2026 Shared Task 1: A hybrid transformer–feature ensemble for L1-aware English vocabulary difficulty prediction](https://md.research.elamur.ai/free/en/papers/pineda-2026-data-asgardians-bea-2026-shared/). In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics.
47. Liudmila Prokhorenkova, Gleb Gusev, Aleksandr Vorobev, Anna Veronika Dorogush, and Andrey Gulin. 2018. Catboost: unbiased boosting with categorical features. In Proceedings of the 32nd International Conference on Neural Information Processing Systems, NIPS’18, pages 6639–6649, Red Hook, NY, USA. Curran Associates Inc. [↗](https://doi.org/10.48550/arxiv.1706.09516)
48. Ricardo Rei, Craig Stewart, Ana C Farinha, and Alon Lavie. 2020. [COMET: A neural framework for MT evaluation](https://md.research.elamur.ai/free/en/papers/rei-2020-comet-neural-framework-mt-evaluation/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 2685–2702, Online. Association for Computational Linguistics.
49. Santiago Robaina, Aiala Rosá, and Luis Chiruzzo. 2026. [RETUYT-INCO at BEA 2026 Shared Task 1: Feature-enriched mDeBERTa for word difficulty prediction](https://md.research.elamur.ai/free/en/papers/robaina-2026-retuyt-inco-bea-2026-shared/). In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics.
50. Norbert Schmitt, Karen Dunn, Barry O’Sullivan, Laurence Anthony, and Benjamin Kremmel. 2021. Introducing Knowledge-based Vocabulary Lists (KVL). Tesol Journal, 12(4). [↗](https://doi.org/10.1002/tesj.622)
51. Norbert Schmitt, Karen Dunn, Barry O’Sullivan, Laurence Anthony, and Benjamin Kremmel. 2024. Knowledge-based Vocabulary Lists. University of Toronto Press, Toronto. [find](https://scholar.google.com/scholar?q=Norbert+Schmitt%2C+Karen+Dunn%2C+Barry+O%E2%80%99Sullivan%2C+Laurence+Anthony%2C+and+Benjamin+Kremmel%2E+2024%2E+Knowledge%2Dbased+Vocabulary+Lists%2E+University+of+Toronto+Press%2C+Toronto%2E)
52. Norbert Schmitt, Paul Nation, and Benjamin Kremmel. 2020. Moving the field of vocabulary assessment forward: The need for more rigorous test development and validation. Language Teaching, 53(1):109–120. [↗](https://doi.org/10.26686/wgtn.12552059.v1)
53. Burr Settles, Geoffrey T. LaFlair, and Masato Hagiwara. 2020. Machine learning–driven language assessment. Transactions of the Association for Computational Linguistics, 8:247–263. [↗](https://doi.org/10.1162/tacl_a_00310)
54. Matthew Shardlow, Fernando Alva-Manchego, Riza Batista-Navarro, Stefan Bott, Saul Calderon Ramirez, Rémi Cardon, Thomas François, Akio Hayakawa, Andrea Horbach, Anna Hülsing, Yusuke Ide, Joseph Marvin Imperial, Adam Nohejl, Kai North, Laura Occhipinti, Nelson Peréz Rojas, Nishat Raihan, Tharindu Ranasinghe, Martin Solis Salazar, and 3 others. 2024. [find](https://scholar.google.com/scholar?q=Matthew+Shardlow%2C+Fernando+Alva%2DManchego%2C+Riza+Batista%2DNavarro%2C+Stefan+Bott%2C+Saul+Calderon+Ramirez%2C+R%C3%A9mi+Cardon%2C+Thomas+Fran%C3%A7ois%2C+Akio+Hayakawa%2C+Andrea+Horbach%2C+Anna+H%C3%BClsing%2C+Yusuke+Ide%2C+Joseph+Marvin)
55. The BEA 2024 shared task on the multilingual lexical simplification pipeline. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), pages 571–589, Mexico City, Mexico. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=The+BEA+2024+shared+task+on+the+multilingual+lexical+simplification+pipeline%2E+In+Proceedings+of+the+19th+Workshop+on+Innovative+Use+of+NLP+for+Building+Educational+Applications+%28BEA+2024%29%2C+pages+571%E2%80%935)
56. Matthew Shardlow, Richard Evans, Gustavo Henrique Paetzold, and Marcos Zampieri. 2021. [SemEval-2021 task 1: Lexical complexity prediction](https://md.research.elamur.ai/free/en/papers/shardlow-2021-semeval-2021-task-1-lexical/). In Proceedings of the 15th International Workshop on Semantic Evaluation (SemEval-2021), pages 1–16, Online. Association for Computational Linguistics. [↗](https://doi.org/10.60692/9gce1-6ak91)
57. Avinash Kumar Sharma. 2026. [EduNLP at BEA 2026 Shared Task 1: Multi-model ensemble with feature-augmented transformers for vocabulary difficulty prediction](https://md.research.elamur.ai/free/en/papers/sharma-2026-edunlp-bea-2026-shared-task/). In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics.
58. Lucy Skidmore, Mariano Felice, and Karen Dunn. 2025. Transformer architectures for vocabulary test item difficulty prediction. In Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025), pages 160– 174, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Lucy+Skidmore%2C+Mariano+Felice%2C+and+Karen+Dunn%2E+2025%2E+Transformer+architectures+for+vocabulary+test+item+difficulty+prediction%2E+In+Proceedings+of+the+20th+Workshop+on+Innovative+Use+of+NLP+for+Building)
59. Bernardo Stearns, John P. McCrae, Thomas Gaillat, and Jefkine Kafunah. 2026. [uogal at BEA 2026 Shared Task 1: Ensemble of multilingual encoders with NMT augmentation for L1-aware vocabulary difficulty prediction](https://md.research.elamur.ai/free/en/papers/stearns-2026-uogal-bea-2026-shared-task/). In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics.
60. Team GLM. 2024. [ChatGLM: A family of large language models from GLM-130B to GLM-4 all tools](https://md.research.elamur.ai/free/en/papers/glm-2024-chatglm-family-large-language-models/). arXiv preprint arXiv:2406.12793. [↗](https://arxiv.org/abs/2406.12793)
61. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. [Multilingual e5 text embeddings: A technical report](https://md.research.elamur.ai/free/en/papers/wang-2024-multilingual-e5-text-embeddings-technical/). Preprint, arXiv:2402.05672. [↗](https://arxiv.org/abs/2402.05672)
62. Georgina Jennifer Willoughby, Jordan Painter, Diptesh Kanojia, Emily Frances Wells, and Constantin Orasan. 2026. [SurreyCTS at BEA 2026 Shared Task 1: Semantic funnelling and entropy-based multilingual lexical difficulty prediction](https://md.research.elamur.ai/free/en/papers/willoughby-2026-surreycts-bea-2026-shared-task/). In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational Linguistics.
63. Victoria Yaneva, Kai North, Peter Baldwin, Le An Ha, Saed Rezayi, Yiyun Zhou, Sagnik Ray Choudhury, Polina Harik, and Brian Clauser. 2024. Findings from the first shared task on automated prediction of difficulty and response time for multiple-choice questions. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), pages 470–482, Mexico City, Mexico. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Victoria+Yaneva%2C+Kai+North%2C+Peter+Baldwin%2C+Le+An+Ha%2C+Saed+Rezayi%2C+Yiyun+Zhou%2C+Sagnik+Ray+Choudhury%2C+Polina+Harik%2C+and+Brian+Clauser%2E+2024%2E+Findings+from+the+first+shared+task+on+automated+prediction+o)
64. An Yang, Baosong Yang, Beichen Zhang, and 1 others. 2024. Qwen2.5: A party of foundation models. arXiv preprint arXiv:2412.15115. [↗](https://arxiv.org/abs/2412.15115)
65. Seid Muhie Yimam, Chris Biemann, Shervin Malmasi, Gustavo Paetzold, Lucia Specia, Sanja Štajner, Anaïs Tack, and Marcos Zampieri. 2018. A report on the complex word identification shared task 2018. In Proceedings of the Thirteenth Workshop on Innovative Use of NLP for Building Educational Applications, pages 66–78, New Orleans, Louisiana. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Seid+Muhie+Yimam%2C+Chris+Biemann%2C+Shervin+Malmasi%2C+Gustavo+Paetzold%2C+Lucia+Specia%2C+Sanja+%C5%A0tajner%2C+Ana%C3%AFs+Tack%2C+and+Marcos+Zampieri%2E+2018%2E+A+report+on+the+complex+word+identification+shared+task+2018%2E+In)

*Updated 1 November 2026*