--- title: "Minanto at SemEval-2023 Task 2: Fine-tuning XLM-RoBERTa for" description: "The paper describes team Minanto's entry to SemEval-2023 Task 2 (MultiCoNER II), English… 1.07 points. 6 references, cited by 1 paper here." authors: ["Antonia Höfer", "Mina Mottahedin"] year: 2023 venue: "Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)" source: "https://aclanthology.org/2023.semeval-1.156/" doi: "10.18653/v1/2023.semeval-1.156" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/hofer-2023-minanto-semeval-2023-task-2/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hofer-2023-minanto-semeval-2023-task-2/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/13/) / Minanto at SemEval-2023 Task 2: Fine-tuning XLM-RoBERTa for Named Entity Recognition on English Data # Minanto at SemEval-2023 Task 2: Fine-tuning XLM-RoBERTa for Named Entity Recognition on English Data **Authors:** Antonia Höfer and Mina Mottahedin · **Year:** 2023 · **Venue:** [Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2023](https://md.research.elamur.ai/free/en/venues/semeval/2023/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2023.semeval-1.156/) · [PDF](https://aclanthology.org/2023.semeval-1.156.pdf) · [DOI 10.18653/v1/2023.semeval-1.156](https://doi.org/10.18653/v1/2023.semeval-1.156) · [OpenAlex](https://openalex.org/W4385570916) **Citations:** 10 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read ## Summary The paper describes team Minanto's entry to SemEval-2023 Task 2 (MultiCoNER II), English track. The task is to recognise complex, fine-grained named entities such as media titles, products and corporations. The label set has 33 classes grouped into six categories. Texts come from Wikipedia sentences, questions and search queries, and part of the test set contains injected typos. The authors converted the data into token/tag dictionaries and fine-tuned XLM-RoBERTa-base with Hugging Face, using a linear token-classification head. They also tried adding CoNLL 2003 ++ and WNUT 2016 after mapping those corpora's tags onto the MultiCoNER taxonomy. The final system reached a fine-grained macro F1 of 51.47%, ranking 28th of 34. Coarse-grained F1 was much higher at 70.75%. Per-class scores roughly followed how often each class appeared in training, though not always. The external data gave little or no benefit. After the deadline, the authors found that the English-only RoBERTa-base beat the cross-lingual model. The paper is useful mainly as a candid baseline and a lesson in lossy label harmonisation. ## Key points - Training setup: XLM-RoBERTa-base with batch size 16, three epochs and learning rate 3.0 × 10⁻⁵, evaluated by precision, recall and F1. - Final macro F1 scores were 51.47% overall, 53.43% on the clean subset and 47.0% on the noisy subset; the winning team scored 85.53%. - Strongest fine-grained classes were HumanSettlement (0.8396), Athlete (0.7466) and Artist (0.7381). The weakest were PrivateCorp (0.0197) and Symptom (0.1561), both rare in training at under 0.8% each. - At the coarse level, PER scored highest (F1 0.9019) and PROD lowest (0.5143). - The supplementary CoNLL 2003 ++ and WNUT 2016 corpora were relabelled into MultiCoNER tags, with MISC and "other" assigned to O, so they covered only eight fine classes. - On the dev set it lowered F1, so it was left out of the final model. - On the labelled test set it would have added about 1.07 points. ## Interesting ideas - Build an explicit crosswalk table from each foreign tagset to the target taxonomy, and send any unmappable label (like MISC) to O instead of guessing; this makes the loss visible and auditable. - Compare per-class F1 with the class's share of training data, and treat classes that underperform their frequency (OtherPROD vs Vehicle) as signs of vague or ambiguous categories rather than data scarcity. - Report scores separately on clean and deliberately noisy inputs, so robustness to typos becomes its own measured number. - Score the same predictions at both fine and coarse granularity, which shows whether errors are confusions between sibling classes or misses of the whole category. - The dev and test sets gave opposite verdicts on adding external data, which warns against trusting a single small dev split for inclusion decisions. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): when merging treebanks or silver sources with different label schemes, keep a documented mapping table and log how much each source loses in conversion. - **I2** (world): apply the same check to entity and ontology types, flagging types that score poorly despite many examples as candidates for splitting or redefinition. - **I3** (tokens): evaluate the tokenizer and tagger on a typo-injected copy of the test text to measure how much damage spelling noise causes upstream. - **I4** (frames): evaluate semantic roles both at the specific VerbNet role and at a coarse agent/patient level to locate where frame errors originate. - **I5** (learn): confirm data-inclusion choices on more than one held-out split, plus a negative control, before adopting them. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of Tübingen](https://md.research.elamur.ai/free/en/universities/university-of-tubingen/) — University, Tübingen, Germany · 78 works · 41,280 citations 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works 🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,379 works · [🏷️ Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/) · 214 works ## Cite Cite as: Antonia Höfer and Mina Mottahedin. 2023. Minanto at SemEval-2023 Task 2: Fine-tuning XLM-RoBERTa for Named Entity Recognition on English Data. Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{hofer2023minanto, title = {{Minanto at SemEval-2023 Task 2: Fine-tuning XLM-RoBERTa for Named Entity Recognition on English Data}}, author = {Antonia Höfer and Mina Mottahedin}, booktitle = {Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)}, year = {2023}, doi = {10.18653/v1/2023.semeval-1.156}, url = {https://doi.org/10.18653/v1/2023.semeval-1.156} } ``` - [← previous MerA: Merging Pretrained Adapters For Few-Shot Learning](https://md.research.elamur.ai/free/en/papers/he-2023-mera-merging-pretrained-adapters-few/) - [next → Mixed Distillation Helps Smaller Language Model Better Reasoning](https://md.research.elamur.ai/free/en/papers/li-2023-mixed-distillation-helps-smaller-language/) - [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/13/) ## References 6 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. 2020. [Unsupervised cross-lingual representation learning at scale](https://md.research.elamur.ai/free/en/papers/conneau-2020-unsupervised-cross-lingual-representation-learning/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8440– Besnik Fetahu, Zhiyu Chen, Sudipta Kar, Oleg Rokhlenko, and Shervin Malmasi. 2023a. Multi-CoNER v2: a Large Multilingual dataset for Fine-grained and Noisy Named Entity Recognition. 2. Besnik Fetahu, Anjie Fang, Oleg Rokhlenko, and Shervin Malmasi. 2021. Gazetteer Enhanced Named Entity Recognition for Code-Mixed Web Queries. In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 1677–1681. [find](https://scholar.google.com/scholar?q=Besnik+Fetahu%2C+Anjie+Fang%2C+Oleg+Rokhlenko%2C+and+Shervin+Malmasi%2E+2021%2E+Gazetteer+Enhanced+Named+Entity+Recognition+for+Code%2DMixed+Web+Queries%2E+In+Proceedings+of+the+44th+International+ACM+SIGIR+Confere) 3. Besnik Fetahu, Sudipta Kar, Zhiyu Chen, Oleg Rokhlenko, and Shervin Malmasi. 2023b. SemEval-2023 Task 2: Fine-grained Multilingual Named Entity Recognition (MultiCoNER 2). In Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Besnik+Fetahu%2C+Sudipta+Kar%2C+Zhiyu+Chen%2C+Oleg+Rokhlenko%2C+and+Shervin+Malmasi%2E+2023b%2E+SemEval%2D2023+Task+2%3A+Fine%2Dgrained+Multilingual+Named+Entity+Recognition+%28MultiCoNER+2%29%2E+In+Proceedings+of+the+17th+I) 4. Shervin Malmasi, Anjie Fang, Besnik Fetahu, Sudipta Kar, and Oleg Rokhlenko. 2022a. [MultiCoNER: a Large-scale Multilingual dataset for Complex Named Entity Recognition](https://md.research.elamur.ai/free/en/papers/malmasi-2022-multiconer-large-scale-multilingual-dataset/). 5. Shervin Malmasi, Anjie Fang, Besnik Fetahu, Sudipta Kar, and Oleg Rokhlenko. 2022b. [SemEval-2022 Task 11: Multilingual Complex Named Entity Recognition (MultiCoNER)](https://md.research.elamur.ai/free/en/papers/malmasi-2022-semeval-2022-task-11-multilingual/). In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022). Association for Computational Linguistics. 6. Tao Meng, Anjie Fang, Oleg Rokhlenko, and Shervin Malmasi. 2021. GEMNET: Effective gated gazetteer representations for recognizing complex entities in low-context input. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1499–1512. [find](https://scholar.google.com/scholar?q=Tao+Meng%2C+Anjie+Fang%2C+Oleg+Rokhlenko%2C+and+Shervin+Malmasi%2E+2021%2E+GEMNET%3A+Effective+gated+gazetteer+representations+for+recognizing+complex+entities+in+low%2Dcontext+input%2E+In+Proceedings+of+the+2021+Con) *Updated 2 November 2026*