--- title: "EduNLP at BEA 2026 Shared Task 1 — paper by Sharma (2026)" description: "Most test words never appear in training, so models must generalize to unseen vocabulary. 8 references, cited by 1 paper here." authors: ["Avinash Kumar Sharma"] year: 2026 venue: "Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)" source: "https://aclanthology.org/2026.bea-1.71/" doi: "10.18653/v1/2026.bea-1.71" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/sharma-2026-edunlp-bea-2026-shared-task/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sharma-2026-edunlp-bea-2026-shared-task/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/12/) / EduNLP at BEA 2026 Shared Task 1: Multi-Model Ensemble with Feature-Augmented Transformers for Vocabulary Difficulty Prediction # EduNLP at BEA 2026 Shared Task 1: Multi-Model Ensemble with Feature-Augmented Transformers for Vocabulary Difficulty Prediction **Authors:** Avinash Kumar Sharma · **Year:** 2026 · **Venue:** [Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)](https://md.research.elamur.ai/free/en/venues/bea/) ([BEA 2026](https://md.research.elamur.ai/free/en/venues/bea/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.bea-1.71/) · [PDF](https://aclanthology.org/2026.bea-1.71.pdf) · [DOI 10.18653/v1/2026.bea-1.71](https://doi.org/10.18653/v1/2026.bea-1.71) · [OpenAlex](https://openalex.org/W7166807079) **Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/sharma-2026-edunlp-bea-2026-shared-task/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary This system paper tackles the BEA 2026 shared task on estimating the difficulty of English vocabulary items for learners whose first language is Spanish, German or Mandarin. The targets are GLMM-calibrated scores from the Extended KVL Dataset for NLP. Most test words never appear in training, so models must generalize to unseen vocabulary. The author pairs hand-engineered signals (Zipf frequency, cognate similarity, length measures, POS flags) with fine-tuned xlm-roberta-base. The closed-track entry averages a per-L1 transformer with a LightGBM regressor. The open track adds two models: one trained jointly on all three L1s, and a variant that appends the 12 standardized features to the mean-pooled encoder output before an MLP head. The system beats the organizers' baselines in all six track–L1 settings, but a clear margin to the winning teams remains. The error analysis is useful in its own right. It identifies rare word senses and nominalized -ing forms as systematic weak spots. It also reports an overfitted Ridge blend as a cautionary negative result. ## Key points - Data: 6,091 train, 677 dev and 748 test items per L1 (ES, DE, CN). About 82.5% of test words are absent from the train and dev sets. - Best test RMSEs were 1.058 (closed, CN) and 0.992 (open, CN). Gains over baselines were 0.081–0.134 (closed) and 0.042–0.095 (open). The winners Glite (closed) and Sakura (open) scored considerably lower. - In dev ablations, dropping frequency raised RMSE by 0.08–0.12 and dropping cognate features raised it by 0.04–0.07. LightGBM alone outperformed the transformer on roughly 27% of words. - Nominalized -ing words (e.g., "dining") had an RMSE of 1.735 versus 1.101 for other words, about 58% worse. Predictions were also compressed: the predicted standard deviation was about 81% of the true value for ES. - A Ridge blend with interaction features looked strong on dev but failed on test (DE 1.397, CN 1.421), worse than the baseline. The author attributes this to tuning on a single split without cross-validation. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Indian Institute of Technology Madras](https://md.research.elamur.ai/free/en/universities/indian-institute-of-technology-madras/) — University, Chennai, India · 16 works · 2,499 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,649 works · [💭 Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) · 523 works ## Cite Cite as: Avinash Kumar Sharma. 2026. EduNLP at BEA 2026 Shared Task 1: Multi-Model Ensemble with Feature-Augmented Transformers for Vocabulary Difficulty Prediction. Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{sharma2026edunlp, title = {{EduNLP at BEA 2026 Shared Task 1: Multi-Model Ensemble with Feature-Augmented Transformers for Vocabulary Difficulty Prediction}}, author = {Avinash Kumar Sharma}, booktitle = {Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026)}, year = {2026}, doi = {10.18653/v1/2026.bea-1.71}, url = {https://doi.org/10.18653/v1/2026.bea-1.71} } ``` - [← previous EDU-CIRCUIT-HW: Evaluating Multimodal Large Language Models on Real-World University-Level STEM Student Handwritten Solutions](https://md.research.elamur.ai/free/en/papers/sun-2026-edu-circuit-hw-evaluating-multimodal/) - [next → Efficient Provably Secure Linguistic Steganography via Range Coding](https://md.research.elamur.ai/free/en/papers/yan-2026-efficient-provably-secure-linguistic-steganography/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/12/) ## References 8 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. [Unsupervised cross-lingual representation learning at scale](https://md.research.elamur.ai/free/en/papers/conneau-2020-unsupervised-cross-lingual-representation-learning/). In Proceedings of the 58th annual meeting of the association fo… 2. Mariano Felice and Lucy Skidmore. 2026. [Findings of the bea 2026 shared task on vocabulary difficulty prediction for english learners](https://md.research.elamur.ai/free/en/papers/felice-2026-findings-bea-2026-shared-task/). In Proceedings of the 21st Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2026), San Diego, California. Association for Computational L… 3. Guolin Ke, Qi Meng, Thomas Finley, Taifeng Wang, Wei Chen, Weidong Ma, Qiwei Ye, and Tie-Yan Liu. 2017. Lightgbm: A highly efficient gradient boosting decision tree. Advances in neural information processing systems, 30. [find](https://scholar.google.com/scholar?q=Guolin+Ke%2C+Qi+Meng%2C+Thomas+Finley%2C+Taifeng+Wang%2C+Wei+Chen%2C+Weidong+Ma%2C+Qiwei+Ye%2C+and+Tie%2DYan+Liu%2E+2017%2E+Lightgbm%3A+A+highly+efficient+gradient+boosting+decision+tree%2E+Advances+in+neural+information+pro) 4. Norbert Schmitt, Karen Dunn, Barry O’Sullivan, Laurence Anthony, and Benjamin Kremmel. 2021. Introducing knowledge-based vocabulary lists (kvl). Tesol Journal, 12(4). [find](https://scholar.google.com/scholar?q=Norbert+Schmitt%2C+Karen+Dunn%2C+Barry+O%E2%80%99Sullivan%2C+Laurence+Anthony%2C+and+Benjamin+Kremmel%2E+2021%2E+Introducing+knowledge%2Dbased+vocabulary+lists+%28kvl%29%2E+Tesol+Journal%2C+12%284%29%2E) 5. Norbert Schmitt, Karen Dunn, Barry O’Sullivan, Laurence Anthony, and Benjamin Kremmel. 2024. Knowledge-based vocabulary lists. University of Toronto Press. [find](https://scholar.google.com/scholar?q=Norbert+Schmitt%2C+Karen+Dunn%2C+Barry+O%E2%80%99Sullivan%2C+Laurence+Anthony%2C+and+Benjamin+Kremmel%2E+2024%2E+Knowledge%2Dbased+vocabulary+lists%2E+University+of+Toronto+Press%2E) 6. Lucy Skidmore, Mariano Felice, and Karen Dunn. 2025. Transformer architectures for vocabulary test item difficulty prediction. In Proceedings of the 20th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2025), pages 160– 174, Vienna, Austria. Association for Computational… [find](https://scholar.google.com/scholar?q=Lucy+Skidmore%2C+Mariano+Felice%2C+and+Karen+Dunn%2E+2025%2E+Transformer+architectures+for+vocabulary+test+item+difficulty+prediction%2E+In+Proceedings+of+the+20th+Workshop+on+Innovative+Use+of+NLP+for+Building) 7. Robyn Speer, Joshua Chin, and Catherine Havasi. 2017. Conceptnet 5.5: An open multilingual graph of general knowledge. In Proceedings of the AAAI conference on artificial intelligence, volume 31. [find](https://scholar.google.com/scholar?q=Robyn+Speer%2C+Joshua+Chin%2C+and+Catherine+Havasi%2E+2017%2E+Conceptnet+5%2E5%3A+An+open+multilingual+graph+of+general+knowledge%2E+In+Proceedings+of+the+AAAI+conference+on+artificial+intelligence%2C+volume+31%2E) 8. Walter JB Van Heuven, Pawel Mandera, Emmanuel Keuleers, and Marc Brysbaert. 2014. Subtlex-uk: A new and improved word frequency database for british english. Quarterly journal of experimental psychology, 67(6):1176–1190. \#\# Acknowledgements We thank Dr. Shabana K M, Senior Project Scientist at the W… [find](https://scholar.google.com/scholar?q=Walter+JB+Van+Heuven%2C+Pawel+Mandera%2C+Emmanuel+Keuleers%2C+and+Marc+Brysbaert%2E+2014%2E+Subtlex%2Duk%3A+A+new+and+improved+word+frequency+database+for+british+english%2E+Quarterly+journal+of+experimental+psycholo) *Updated 2 November 2026*