--- title: "UMUTeam at SemEval-2023 Task 10 — paper by Pan et al. (2023)" description: "The paper describes UMUTeam's system for EDOS (Explainable Detection of Online Sexism, SemEval-2023 Task 10). 2.5 points. 2 references." authors: ["Ronghao Pan", "José Antonio García-Díaz", "Salud María Jiménez Zafra", "Rafael Valencia-García"] year: 2023 venue: "Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)" source: "https://aclanthology.org/2023.semeval-1.80/" doi: "10.18653/v1/2023.semeval-1.80" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/pan-2023-umuteam-semeval-2023-task-10/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pan-2023-umuteam-semeval-2023-task-10/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/17/) / UMUTeam at SemEval-2023 Task 10: Fine-grained detection of sexism in English # UMUTeam at SemEval-2023 Task 10: Fine-grained detection of sexism in English **Authors:** [Ronghao Pan](https://md.research.elamur.ai/free/en/authors/ronghao-pan/), [José Antonio García-Díaz](https://md.research.elamur.ai/free/en/authors/jose-antonio-garcia-diaz/), Salud María Jiménez Zafra and [Rafael Valencia-García](https://md.research.elamur.ai/free/en/authors/rafael-valencia-garcia/) · **Year:** 2023 · **Venue:** [Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2023](https://md.research.elamur.ai/free/en/venues/semeval/2023/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2023.semeval-1.80/) · [PDF](https://aclanthology.org/2023.semeval-1.80.pdf) · [DOI 10.18653/v1/2023.semeval-1.80](https://doi.org/10.18653/v1/2023.semeval-1.80) · [OpenAlex](https://openalex.org/W4385571799) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary The paper describes UMUTeam's system for EDOS (Explainable Detection of Online Sexism, SemEval-2023 Task 10). EDOS asks systems to flag sexist posts from Gab and Reddit and then explain them through a label hierarchy: a binary decision (Task A), four coarse categories (Task B) and eleven fine-grained vectors (Task C). The team took RoBERTa-large and continued its masked-language-model pretraining on the organizers' two million unlabeled Gab and Reddit posts. They then fine-tuned this domain-adapted model separately for each subtask, using only the official labeled corpus. Before training, the text was lightly cleaned: mentions and hashtags were replaced with placeholders, emojis were dropped and slang contractions were expanded. Hyperparameters were tuned with a TPE search. The system ranked 24th of 84 on Task A, 23rd of 69 on Task B and 13th of 63 on Task C. An ablation shows that domain adaptation always helped, and helped most on the sparse 11-way task. Error analysis traces most mistakes to rare or semantically overlapping classes. The work is a simple, reproducible baseline showing that unlabeled in-domain text pays off when labels are scarce. ## Key points - **Data:** The EDOS corpus has 20,000 posts, half from Gab and half from Reddit, split 14,000/2,000/4,000. About a quarter of the posts are sexist (4,854). Each post was labeled by three annotators, with expert adjudication of disagreements. - **Unlabeled data and pretraining:** The organizers supplied 1M Gab and 1M Reddit unlabeled posts. MLM continuation of RoBERTa-large (355M parameters) used line-by-line sentences, a block size of 128, 3 epochs and batch size 32. - **Fine-tuning search:** RayTune with TPE ran 10 trials. The chosen setting was about 15–20 epochs, batch 16, weight decay ≈0.18 and learning rate ≈1.19e-5, the same for all subtasks. - **Results (macro F1):** Task A scored 0.8495, about 2.5 points behind the winner. Task B scored 0.6395 (rank 23/69 in the abstract, 24/70 in the results section). Task C scored 0.4793. - **Ablation:** Compared with plain RoBERTa-large, domain adaptation raised Task A from 0.8426 to 0.8495, Task B from 0.6313 to 0.6395 and Task C from 0.4349 to 0.4793. The main confusions were between animosity, derogation and prejudiced discussions. ## Interesting ideas - Before fine-tuning on a small labeled set, keep training the encoder's MLM objective on plentiful unlabeled text from the same genre; the gain grew as the label set got finer and sparser. - Report a clean before/after comparison of the same backbone with and without adaptation, so the effect of one design choice is isolated. - Read row-normalized confusion matrices to see which class pairs collapse, then explain the collapses by class frequency or by real semantic overlap. - Use a nested label scheme (binary, then 4 classes, then 11), where errors at the coarse level reappear in the fine one. - Normalize noisy social text before modeling, by mapping mentions and hashtags to placeholders and expanding slang contractions like "y'all". ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): continue pretraining the gparse encoder on raw text of our target genres (fables, school textbooks) before treebank training, especially to help rare relations. - **I2** (learn): treat "no adaptation" as a standing control run next to every adapted parser or tagger, so each claimed improvement has its own baseline. - **I3** (frames): apply the same per-pair confusion analysis to speech-act and irony labels in prag, to decide whether a failure needs more data or a sharper label definition. - **I4** (world): ontology-backed labels could be scored at each depth, so we can tell whether a mistake happens at the parent class or only among siblings. - **I5** (tokens): a normalization pass in crate en could expand such forms for tagging while keeping the original span, so every analysis still points back to the raw text. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Universidad de Jaén](https://md.research.elamur.ai/free/en/universities/universidad-de-jaen/) — University, Jaén, Spain · 9 works · 1,671 citations - [Universidad de Murcia](https://md.research.elamur.ai/free/en/universities/universidad-de-murcia/) — University, Murcia, Spain · 6 works · 3 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works 🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,379 works ## Cite Cite as: Ronghao Pan, José Antonio García-Díaz, Salud María Jiménez Zafra and Rafael Valencia-García. 2023. UMUTeam at SemEval-2023 Task 10: Fine-grained detection of sexism in English. Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{pan2023umuteam, title = {{UMUTeam at SemEval-2023 Task 10: Fine-grained detection of sexism in English}}, author = {Ronghao Pan, José Antonio García-Díaz, Salud María Jiménez Zafra and Rafael Valencia-García}, booktitle = {Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)}, year = {2023}, doi = {10.18653/v1/2023.semeval-1.80}, url = {https://doi.org/10.18653/v1/2023.semeval-1.80} } ``` - [← previous Two Safaitic Inscriptions in Comparison to Geographical and Historical Sources about the Arabian Expedition of Aelius Gallus](https://md.research.elamur.ai/free/en/papers/magnelli-2023-two-safaitic-inscriptions-comparison-geographical/) - [next → Unified Syntactic Annotation of English in the CGEL Framework](https://md.research.elamur.ai/free/en/papers/reynolds-2023-unified-syntactic-annotation-english-cgel/) - [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/17/) ## References 2 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Nikolay Arefyev, Dmitrii Kharchev, and Artem Shelmanov. 2021. NB-MLM: Efficient domain adaptation of masked language models for sentiment analysis. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9114–9124, Online and Punta Cana, Dominican Republic. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Nikolay+Arefyev%2C+Dmitrii+Kharchev%2C+and+Artem+Shelmanov%2E+2021%2E+NB%2DMLM%3A+Efficient+domain+adaptation+of+masked+language+models+for+sentiment+analysis%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+M) 2. James Bergstra, Daniel Yamins, and David Cox. 2013. Making a science of model search: Hyperparameter optimization in hundreds of dimensions for vision [find](https://scholar.google.com/scholar?q=James+Bergstra%2C+Daniel+Yamins%2C+and+David+Cox%2E+2013%2E+Making+a+science+of+model+search%3A+Hyperparameter+optimization+in+hundreds+of+dimensions+for+vision) *Updated 2 November 2026*