---
title: "PSK@EEUCA 2026 — paper by Pulipaka (2026)"
description: "This system paper addresses the EEUCA 2026 Shared Task on toxic behavior in gaming communities. 0.60 F1. 15 references."
authors: ["Srikar Kashyap Pulipaka"]
year: 2026
venue: "Proceedings of the 9th Workshop on Event Extraction and Understanding: Challenges and Applications (EEUCA 2026)"
source: "https://aclanthology.org/2026.eeuca-1.21/"
doi: "10.18653/v1/2026.eeuca-1.21"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/pulipaka-2026-psk-eeuca-2026-fine-tuning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pulipaka-2026-psk-eeuca-2026-fine-tuning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/40/) / PSK@EEUCA 2026: Fine-tuning Large Language Models with Synthetic Data Augmentation for Multi-class Toxicity Detection in Gaming Chat

# PSK@EEUCA 2026: Fine-tuning Large Language Models with Synthetic Data Augmentation for Multi-class Toxicity Detection in Gaming Chat

**Authors:** [Srikar Kashyap Pulipaka](https://md.research.elamur.ai/free/en/authors/srikar-kashyap-pulipaka/) · **Year:** 2026 · **Venue:** [Proceedings of the 9th Workshop on Event Extraction and Understanding: Challenges and Applications (EEUCA 2026)](https://md.research.elamur.ai/free/en/venues/eeuca/) ([Workshop on Event Extraction and Understanding: Challenges and Applications 2026](https://md.research.elamur.ai/free/en/venues/eeuca/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eeuca-1.21/) · [PDF](https://aclanthology.org/2026.eeuca-1.21.pdf) · [DOI 10.18653/v1/2026.eeuca-1.21](https://doi.org/10.18653/v1/2026.eeuca-1.21) · [OpenAlex](https://openalex.org/W7166858647)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
This system paper addresses the EEUCA 2026 Shared Task on toxic behavior in gaming communities. The task is to assign World of Tanks chat messages, drawn from the GameTox corpus, to one of six toxicity classes. The data is heavily skewed toward Non-toxic messages, and the rarest classes are almost absent. The author compared several approaches: an encoder baseline (XLM-RoBERTa Large), LoRA-tuned instruction LLMs of different sizes, two-stage and one-vs-rest classifiers, transfer from DOTA 2 data, ensembles, and post-hoc calibration. The winning setup is Llama 3.1 8B, quantized to 4 bits and adapted with LoRA. Its prompt includes short definitions of each class, and its training set gets a small share of LLM-generated paraphrases of minority-class messages. This system scored 0.6234 macro-F1 on the test set, ranking 4th of 35 teams. The paper's main analytical contribution is a "validation trap": models that look strong on validation by predicting mostly Non-toxic transfer poorly to the test split. This warns against relying on validation scores when annotation patterns shift between splits.

## Key points
- **Dataset:** The training set has 42,959 messages, 81% of them Non-toxic. Threats and Extremism together make up under 0.2%. About 40.2% of messages are exact duplicates and 13.4% repeat a text with a different label. Removing duplicates hurt performance (0.44 vs 0.60 F1), apparently because they act as implicit oversampling.
- **Final configuration:** Llama 3.1 8B with NF4 4-bit quantization, LoRA (rank 16, alpha 64), class-weighted cross-entropy, 4 epochs, and a maximum sequence length of 384.
- **Synthetic data:** The paraphrase pool held 10,464 filtered examples. The 5% setting used 1,921 of them, drawn only from Classes 2–5. This ratio gave a test F1 of 0.6232, compared with 0.5971 at 0% and 0.5851 at 10%. A small Class 2 boost raised the submitted score to 0.6234.
- **Validation trap:**
  - Gemma 3 12B: 0.66 validation vs. 0.52 test.
  - Two-stage hierarchical model: 0.67 vs. 0.47.
  - DOTA 2 transfer: 0.68 vs. 0.55.
- **Other findings:** Ensembles and calibration methods (Platt scaling, isotonic regression, temperature scaling) brought no gains. Other Offensive and Hate/Harassment were the hardest classes, each with roughly 0.43–0.44 test F1.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🛡️ Bias, fairness & safety](https://md.research.elamur.ai/free/en/topics/bias-fairness-safety/) · 517 works

## Cite

Cite as: Srikar Kashyap Pulipaka. 2026. PSK@EEUCA 2026: Fine-tuning Large Language Models with Synthetic Data Augmentation for Multi-class Toxicity Detection in Gaming Chat. Proceedings of the 9th Workshop on Event Extraction and Understanding: Challenges and Applications (EEUCA 2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{pulipaka2026psk,
  title = {{PSK@EEUCA 2026: Fine-tuning Large Language Models with Synthetic Data Augmentation for Multi-class Toxicity Detection in Gaming Chat}},
  author = {Srikar Kashyap Pulipaka},
  booktitle = {Proceedings of the 9th Workshop on Event Extraction and Understanding: Challenges and Applications (EEUCA 2026)},
  year = {2026},
  doi = {10.18653/v1/2026.eeuca-1.21},
  url = {https://doi.org/10.18653/v1/2026.eeuca-1.21}
}
```

- [← previous PseudoSeer: a Search Engine for Pseudocode](https://md.research.elamur.ai/free/en/papers/toksoz-2026-pseudoseer-search-engine-pseudocode/)
- [next → Psycholinguistic Profiles of Cognitive Distortions in Reddit Data](https://md.research.elamur.ai/free/en/papers/sharma-2026-psycholinguistic-profiles-cognitive-distortions-reddit/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/40/)

## References

15 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aashish Bhandari, Siddhant B Shah, Surendrabikram Thapa, Usman Naseem, and Mehwish Nasim. 2023. CrisisHateMM: Multimodal analysis of directed and undirected hate speech in text-embedded images from Russia–Ukraine conflict. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Reco… [find](https://scholar.google.com/scholar?q=Aashish+Bhandari%2C+Siddhant+B+Shah%2C+Surendrabikram+Thapa%2C+Usman+Naseem%2C+and+Mehwish+Nasim%2E+2023%2E+CrisisHateMM%3A+Multimodal+analysis+of+directed+and+undirected+hate+speech+in+text%2Dembedded+images+from+Ru)
2. Usman Naseem, Shuvam Shiwakoti, Siddhant Bikram Shah, Surendrabikram Thapa, and Qi Zhang. 2025. GameTox: A comprehensive dataset and analysis for enhanced toxicity detection in online gaming communities. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association… [find](https://scholar.google.com/scholar?q=Usman+Naseem%2C+Shuvam+Shiwakoti%2C+Siddhant+Bikram+Shah%2C+Surendrabikram+Thapa%2C+and+Qi+Zhang%2E+2025%2E+GameTox%3A+A+comprehensive+dataset+and+analysis+for+enhanced+toxicity+detection+in+online+gaming+communiti)
3. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. 2023. [QLoRA: Efficient finetuning of quantized LLMs](https://md.research.elamur.ai/free/en/papers/dettmers-2023-qlora-efficient-finetuning-quantized-llms/). In Advances in Neural Information Processing Systems, volume 36.
4. Anil Singh Parihar, Surendrabikram Thapa, and Sushruti Mishra. 2021. Hate speech detection using natural language processing: Applications and challenges. In 2021 5th International Conference on Trends in Electronics and Informatics (ICOEI), pages 1302– 1308. IEEE. [find](https://scholar.google.com/scholar?q=Anil+Singh+Parihar%2C+Surendrabikram+Thapa%2C+and+Sushruti+Mishra%2E+2021%2E+Hate+speech+detection+using+natural+language+processing%3A+Applications+and+challenges%2E+In+2021+5th+International+Conference+on+Trend)
5. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,…
6. Surendrabikram Thapa, Shuvam Shiwakoti, Siddhant Bikram Shah, Surabhi Adhikari, Hariram Veeramani, Mehwish Nasim, and Usman Naseem. 2025. Large language models (llm) in computational social science: prospects, current state, and challenges. Social Network Analysis and Mining, 15(1):4. [find](https://scholar.google.com/scholar?q=Surendrabikram+Thapa%2C+Shuvam+Shiwakoti%2C+Siddhant+Bikram+Shah%2C+Surabhi+Adhikari%2C+Hariram+Veeramani%2C+Mehwish+Nasim%2C+and+Usman+Naseem%2E+2025%2E+Large+language+models+%28llm%29+in+computational+social+science%3A+p)
7. Gemma Team. 2024. Gemma 2: Improving open language models at a practical size. arXiv preprint arXiv:2408.00118. [find](https://scholar.google.com/scholar?q=Gemma+Team%2E+2024%2E+Gemma+2%3A+Improving+open+language+models+at+a+practical+size%2E+arXiv+preprint+arXiv%3A2408%2E00118%2E)
8. Surendrabikram Thapa, Shuvam Shiwakoti, Siddhant Bikram Shah, Kritesh Rauniyar, Laxmi Thapa, Surabhi Adhikari, Kristina T. Johnson, Ali Hür-riyetoğlu, Hristo Tanev, and Usman Naseem. 2026. Understanding toxic behavior in gaming communities using ai to promote healthier digital spaces. In Proceeding… [find](https://scholar.google.com/scholar?q=Surendrabikram+Thapa%2C+Shuvam+Shiwakoti%2C+Siddhant+Bikram+Shah%2C+Kritesh+Rauniyar%2C+Laxmi+Thapa%2C+Surabhi+Adhikari%2C+Kristina+T%2E+Johnson%2C+Ali+H%C3%BCr%2Driyetog%CC%86lu%2C+Hristo+Tanev%2C+and+Usman+Naseem%2E+2026%2E+Understand)
9. Gemma Team. 2025. Gemma 3 technical report. arXiv preprint arXiv:2503.19786. [find](https://scholar.google.com/scholar?q=Gemma+Team%2E+2025%2E+Gemma+3+technical+report%2E+arXiv+preprint+arXiv%3A2503%2E19786%2E)
10. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. [LoRA: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In International Conference on Learning Representations.
11. Ali Hürriyetoğlu, Surendrabikram Thapa, Hristo Tanev, Laxmi Thapa, and Surabhi Adhikari. 2026. Overview of the workshop on event extraction and understanding: Challenges and applications. In Proceedings of the 9th Workshop on Event Extraction and Understanding: Challenges and Applications (EEUCA). [find](https://scholar.google.com/scholar?q=Ali+H%C3%BCrriyetog%CC%86lu%2C+Surendrabikram+Thapa%2C+Hristo+Tanev%2C+Laxmi+Thapa%2C+and+Surabhi+Adhikari%2E+2026%2E+Overview+of+the+workshop+on+event+extraction+and+understanding%3A+Challenges+and+applications%2E+In+Proceedi)
12. Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2022. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). In International Conference on Learning Representations.
13. Haewoon Kwak, Jeremy Blackburn, and Seungyeop Han. 2015. Exploring cyberbullying and other toxic behavior in team competition online games. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pages 3739– 3748. [find](https://scholar.google.com/scholar?q=Haewoon+Kwak%2C+Jeremy+Blackburn%2C+and+Seungyeop+Han%2E+2015%2E+Exploring+cyberbullying+and+other+toxic+behavior+in+team+competition+online+games%2E+In+Proceedings+of+the+33rd+Annual+ACM+Conference+on+Human+Fa)
14. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. [Focal loss for dense object detection](https://md.research.elamur.ai/free/en/papers/lin-2017-focal-loss-dense-object-detection/). In Proceedings of the IEEE International Conference on Computer Vision, pages 2980–2988.
15. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [RoBERTa: A robustly optimized BERT pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). arXiv preprint arXiv:1907.11692.

*Updated 1 November 2026*