--- title: "PCJ at SemEval-2023 Task 10 — paper by Pu and Zhou (2023)" description: "The paper presents team PCJ's system for SemEval-2023 Task 10, subtask A. The task is a binary decision on whether an… 19 references." authors: ["Chujun Pu", "Xiaobing Zhou"] year: 2023 venue: "Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)" source: "https://aclanthology.org/2023.semeval-1.59/" doi: "10.18653/v1/2023.semeval-1.59" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/pu-2023-pcj-semeval-2023-task-10/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pu-2023-pcj-semeval-2023-task-10/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/16/) / PCJ at SemEval-2023 Task 10: A Ensemble Model Based on Pre-trained Model for Sexism Detection and Classification in English # PCJ at SemEval-2023 Task 10: A Ensemble Model Based on Pre-trained Model for Sexism Detection and Classification in English **Authors:** Chujun Pu and Xiaobing Zhou · **Year:** 2023 · **Venue:** [Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2023](https://md.research.elamur.ai/free/en/venues/semeval/2023/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2023.semeval-1.59/) · [PDF](https://aclanthology.org/2023.semeval-1.59.pdf) · [DOI 10.18653/v1/2023.semeval-1.59](https://doi.org/10.18653/v1/2023.semeval-1.59) · [OpenAlex](https://openalex.org/W4385570477) **Citations:** 2 citations (OpenAlex, 2026-10-14) ## Summary The paper presents team PCJ's system for SemEval-2023 Task 10, subtask A. The task is a binary decision on whether an English post from Gab or Reddit is sexist. The authors fine-tune two large encoders, RoBERTa-large and sup-SimCSE-RoBERTa-large, and attach several classification heads to them: - a BiLSTM; - a BiLSTM followed by a BiGRU, with pooled outputs combined with the pooler output; - a head that concatenates the \ vectors of the last four hidden layers. The resulting four models are merged by majority voting. Training uses the organizers' 14,000 labelled posts, which are heavily skewed toward the non-sexist class. The authors split them 8:1:1 with stratification and clean only the training and validation parts. On the official evaluation the ensemble reached a macro F1 of 0.8449, placing 28th of 84 teams. The ensemble's development-phase score stayed close to its score on the internal split, while the single models drifted more. This suggests that voting mainly buys stability. A per-class analysis shows that the minority "sexist" class remains clearly harder. ## Key points - **Data:** 14,000 training posts, of which 3,398 are sexist, drawn from Gab and Reddit. The data was split 8:1:1 with stratified sampling. Emoji and "[xxx]" tokens were removed from train and validation, but the internal test part was left raw. - **Four models:** - RoBERTa-BiLSTM; - RoBERTa-BiLSTM-BiGRU, with average and max pooling plus the pooler output; - RoBERTa using the last-four-layer \ concatenation with max pooling; - SimCSE-RoBERTa-BiLSTM. - **Training setup:** BertAdam optimizer, learning rate 2e-5 for RoBERTa and 1e-5 for the other parts, dropout 0.2, pad size 64, batch size 32. - **Macro F1 on the internal test split:** plain RoBERTa 0.8327, +BiLSTM 0.8311, +BiLSTM+BiGRU 0.8467, SimCSE+BiLSTM 0.8359, ensemble 0.8480. - **Macro F1 on development-phase data:** RoBERTa 0.8164, ensemble 0.8489. - **Per-class results:** precision and recall exceed 0.9 for "not sexist" but sit around 0.70–0.80 for "sexist". The authors attribute the gap to class imbalance. ## Interesting ideas - Several differently built heads on related encoders are combined by simple majority vote, which mainly improved how well results held up on a second evaluation set rather than raising peak accuracy. - The [CLS] vectors from the last four encoder layers are concatenated instead of using only the top layer, on the premise that each layer captures text differently. - The authors deliberately keep their internal test split unpreprocessed, so it resembles the raw competition input and gives a more honest score. - They treat the gap between a model's score on its own split and on an external dev set as a generalization signal, separate from the absolute score. - Reporting precision and recall per class exposed that the macro score hid a weak minority class. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): This supports building silver data from agreement among structurally different witness parsers, and suggests measuring agreement's value by stability across held-out sets. - **I2** (tree): gparse could feed a learned mix of upper layers into its arc and label scorers and then check which layers help attachment decisions most. - **I3** (learn): Evaluation sets should pass through the full pipeline from raw text, so that cleanup steps applied in training do not inflate tokenizer or parser scores. - **I4** (learn): For each trained component, we could report the drop from in-domain to out-of-domain treebank as a standard metric alongside LAS. - **I5** (frames): Speech-act and irony detectors should be evaluated per label, because rare categories like irony are exactly where aggregate scores mislead. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Yunnan University](https://md.research.elamur.ai/free/en/universities/yunnan-university/) — University, Kunming, China · 20 works · 40 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works 🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,379 works ## Cite Cite as: Chujun Pu and Xiaobing Zhou. 2023. PCJ at SemEval-2023 Task 10: A Ensemble Model Based on Pre-trained Model for Sexism Detection and Classification in English. Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{pu2023pcj, title = {{PCJ at SemEval-2023 Task 10: A Ensemble Model Based on Pre-trained Model for Sexism Detection and Classification in English}}, author = {Chujun Pu and Xiaobing Zhou}, booktitle = {Proceedings of the 17th International Workshop on Semantic Evaluation (SemEval-2023)}, year = {2023}, doi = {10.18653/v1/2023.semeval-1.59}, url = {https://doi.org/10.18653/v1/2023.semeval-1.59} } ``` - [← previous Parsing “Early English Books Online” for Linguistic Search](https://md.research.elamur.ai/free/en/papers/kulick-2023-parsing-early-english-books-online/) - [next → Peer Teaching Method Integrated with Card Media: Its Implementation in Arabic Syntax Learning](https://md.research.elamur.ai/free/en/papers/rizal-2023-peer-teaching-method-integrated-card/) - [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/16/) ## References 19 references, 10 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Basant Agarwal and Namita Mittal. Text classification using machine learning methods-a survey. In Proceedings of the Second International Conference on Soft Computing for Problem Solving (SocProS 2012), December 28-30, 2012, pages 701–709. [find](https://scholar.google.com/scholar?q=Basant+Agarwal+and+Namita+Mittal%2E+Text+classification+using+machine+learning+methods%2Da+survey%2E+In+Proceedings+of+the+Second+International+Conference+on+Soft+Computing+for+Problem+Solving+%28SocProS+2012) 2. Shikha Bordia and Samuel R Bowman. 2019. [Identifying and reducing gender bias in word-level language models](https://md.research.elamur.ai/free/en/papers/bordia-2019-identifying-reducing-gender-bias-word/). arXiv preprint arXiv:1904.03035. [↗](https://arxiv.org/abs/1904.03035) 3. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805. [↗](https://arxiv.org/abs/1810.04805) 4. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. [Simcse: Simple contrastive learning of sentence embeddings](https://md.research.elamur.ai/free/en/papers/gao-2021-simcse-simple-contrastive-learning-sentence/). arXiv preprint arXiv:2104.08821. [↗](https://arxiv.org/abs/2104.08821) 5. Ganesh Jawahar, Benoît Sagot, and Djamé Seddah. 2019. [What does bert learn about the structure of language?](https://md.research.elamur.ai/free/en/papers/jawahar-2019-what-does-bert-learn-about/) In ACL 2019-57th Annual Meeting of the Association for Computational Linguistics. 6. Hannah Rose Kirk, Wenjie Yin, Bertie Vidgen, and Paul Röttger. 2023. [SemEval-2023 Task 10: Explainable Detection of Online Sexism](https://md.research.elamur.ai/free/en/papers/kirk-2023-semeval-2023-task-10-explainable/). In Proceedings of the 17th International Workshop on Semantic Evaluation, Toronto, Canada. Association for Computational Linguistics. 7. Qian Li, Hao Peng, Jianxin Li, Congying Xia, Renyu Yang, Lichao Sun, Philip S Yu, and Lifang He. 2020. A survey on text classification: From shallow to deep learning. arXiv preprint arXiv:2008.00364. [↗](https://arxiv.org/abs/2008.00364) 8. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692. [↗](https://arxiv.org/abs/1907.11692) 9. Hanjie Mai, Xiaobing Zhou, and Liqing Wang. A multi-task learning model for fine-grain dialogue social bias measurement. In Natural Language Processing and Chinese Computing: 11th CCF International Conference, NLPCC 2022, Guilin, China, September 24–25, 2022, Proceedings, Part II. [find](https://scholar.google.com/scholar?q=Hanjie+Mai%2C+Xiaobing+Zhou%2C+and+Liqing+Wang%2E+A+multi%2Dtask+learning+model+for+fine%2Dgrain+dialogue+social+bias+measurement%2E+In+Natural+Language+Processing+and+Chinese+Computing%3A+11th+CCF+International+Co) 10. SreeJagadeesh Malla and PJA Alphonse. 2021. Covid-19 outbreak: An ensemble pre-trained deep learning model for detecting informative tweets. Applied Soft Computing, 107:107495. [↗](https://doi.org/10.1016/j.asoc.2021.107495) 11. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. [Distributed representations of words and phrases and their compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). Advances in neural information processing systems, 26. [↗](https://doi.org/10.48550/arxiv.1310.4546) 12. Shervin Minaee, Nal Kalchbrenner, Erik Cambria, Narjes Nikzad, Meysam Chenaghlu, and Jianfeng Gao. 2021. Deep learning–based text classification: a comprehensive review. ACM computing surveys (CSUR), 54(3):1–40. [↗](https://doi.org/10.48550/arxiv.2004.03705) 13. Ji Ho Park, Jamin Shin, and Pascale Fung. 2018. [Reducing gender bias in abusive language detection](https://md.research.elamur.ai/free/en/papers/park-2018-reducing-gender-bias-abusive-language/). arXiv preprint arXiv:1808.07231. Xipeng Qiu, Tianxiang Sun, Yige Xu, Yunfan Shao, Ning Dai, and Xuanjing Huang. Pre-trained models for natural language processing: A survey. Science China Technological Sciences. [↗](https://arxiv.org/abs/1808.07231) 14. Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. 2018. Improving language understanding by generative pre-training. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Karthik+Narasimhan%2C+Tim+Salimans%2C+Ilya+Sutskever%2C+et+al%2E+2018%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E) 15. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in neural information processing systems, 30. 16. Bingkun Wang, Donghong Shan, Aiwan Fan, Lei Liu, and Jingli Gao. 2021. A sentiment classification method of web social media based on multidimensional and multilevel modeling. IEEE Transactions on Industrial Informatics, 18(2):1240–1249. [↗](https://doi.org/10.1109/tii.2021.3085663) 17. Qing Yu, Ziyin Wang, and Kaiwen Jiang. 2021. Research on text classification based on bert-bigru model. In Journal of Physics: Conference Series, volume 1746, page 012019. IOP Publishing. [find](https://scholar.google.com/scholar?q=Qing+Yu%2C+Ziyin+Wang%2C+and+Kaiwen+Jiang%2E+2021%2E+Research+on+text+classification+based+on+bert%2Dbigru+model%2E+In+Journal+of+Physics%3A+Conference+Series%2C+volume+1746%2C+page+012019%2E+IOP+Publishing%2E) 18. Tianyi Zhang, Felix Wu, Arzoo Katiyar, Kilian Q Weinberger, and Yoav Artzi. 2020. [Revisiting few-sample bert fine-tuning](https://md.research.elamur.ai/free/en/papers/zhang-2020-revisiting-few-sample-bert-fine/). arXiv preprint arXiv:2006.05987. [↗](https://arxiv.org/abs/2006.05987) 19. Yu Zhang and Qiang Yang. 2021. A survey on multi-task learning. IEEE Transactions on Knowledge and Data Engineering, 34(12):5586–5609. [↗](https://doi.org/10.1109/tkde.2021.3070203) *Updated 1 November 2026*