---
title: "Segmentation Fault at SemEval-2026 Task 13: A"
description: "The test set includes generators and programming languages absent from training. 13 references."
authors: ["Lakshmi Priya Swaminatha Rao", "Dhannya Santhakumari Madhavan", "Sreya Kodeswaran", "Nithila R", "Kanmani R"]
year: 2026
venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)"
source: "https://aclanthology.org/2026.semeval-1.266/"
doi: "10.18653/v1/2026.semeval-1.266"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/rao-2026-segmentation-fault-semeval-2026-task/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rao-2026-segmentation-fault-semeval-2026-task/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/42/) / Segmentation Fault at SemEval-2026 Task 13: A Regularization-First Approach with Generator-Based Out-of-Distribution Splits for Detecting AI-Generated Code

# Segmentation Fault at SemEval-2026 Task 13: A Regularization-First Approach with Generator-Based Out-of-Distribution Splits for Detecting AI-Generated Code

**Authors:** [Lakshmi Priya Swaminatha Rao](https://md.research.elamur.ai/free/en/authors/lakshmi-priya-swaminatha-rao/), [Dhannya Santhakumari Madhavan](https://md.research.elamur.ai/free/en/authors/dhannya-santhakumari-madhavan/), Sreya Kodeswaran, Nithila R and Kanmani R · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.266/) · [PDF](https://aclanthology.org/2026.semeval-1.266.pdf) · [DOI 10.18653/v1/2026.semeval-1.266](https://doi.org/10.18653/v1/2026.semeval-1.266) · [OpenAlex](https://openalex.org/W7166881845)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
This system paper tackles SemEval-2026 Task 13 Subtask A, a binary task of labeling code snippets as human-written or machine-generated. The test set includes generators and programming languages absent from training. The authors found that models validated on the provided random split scored nearly perfectly but collapsed on the hidden test data. They attribute this to memorization of generator-specific style.

Their remedy is to hold out entire generator families (Llama, DeepSeek, Yi) for validation. They also fine-tune CodeBERT-base under strong regularization on a balanced 100K-sample subset. The regularization combines lexical augmentation, high dropout, label smoothing and weight decay, with early checkpoint selection.

They compare four systems: TF-IDF logistic regression, UniXcoder, vanilla CodeBERT and the regularized CodeBERT. The final model reaches 0.439 macro F1 on the test set, ranking 55th of 81 submissions. This is a 62% relative gain over the 0.27 of the unregularized neural models.

The study's main lesson is that validation design and data regularization matter more than data volume when detecting output from unseen generators. However, a large gap between validation and test performance remains.

## Key points
- **Generator-based OOD split:** 22 generators from six families (Qwen, Phi, CodeLlama, StarCoder, Granite, CodeGemma) are used for training. 13 generators from Llama, DeepSeek and Yi are reserved for validation, with no overlap between the two.
- **Regularization settings:**
  - dropout 0.3, label smoothing 0.15, weight decay 0.02;
  - gradient clipping 0.5 and 10% learning-rate warmup;
  - checkpoint chosen at step 1,000;
  - stochastic augmentation on 80% of samples: variable renaming, comment removal and formatting normalization.
- **Validation vs. test scores:**

  | System | Validation F1 | Test F1 |
  |---|---|---|
  | UniXcoder (150K) | 0.99 | 0.27 |
  | vanilla CodeBERT (500K) | 0.99 | 0.27 |
  | TF-IDF logistic regression | 0.920 | 0.340 |
  | final system (OOD validation) | 0.965 | 0.439 |

- **Training data skew:** the data is about 91% Python, with smaller shares of C++ and Java. Inputs are truncated at 256 tokens, which affects roughly 44% of training samples.
- **Limitations:** the authors note that individual components were not ablated separately and that regex-based renaming can break syntax. The model also struggles with short boilerplate snippets. Code is on GitHub (Sreya889/Semeval-Task-13-Subtask-A).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Sri Sivasubramaniya Nadar College of Engineering](https://md.research.elamur.ai/free/en/universities/sri-sivasubramaniya-nadar-college-of-engineering/) — University, Chennai, India · 10 works · 9 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

## Cite

Cite as: Lakshmi Priya Swaminatha Rao, Dhannya Santhakumari Madhavan, Sreya Kodeswaran, Nithila R and Kanmani R. 2026. Segmentation Fault at SemEval-2026 Task 13: A Regularization-First Approach with Generator-Based Out-of-Distribution Splits for Detecting AI-Generated Code. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{rao2026segmentation,
  title = {{Segmentation Fault at SemEval-2026 Task 13: A Regularization-First Approach with Generator-Based Out-of-Distribution Splits for Detecting AI-Generated Code}},
  author = {Lakshmi Priya Swaminatha Rao, Dhannya Santhakumari Madhavan, Sreya Kodeswaran, Nithila R and Kanmani R},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.266},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.266}
}
```

- [← previous SEEM-CZ: Annotation and Classification of Epistemic Markers in Czech](https://md.research.elamur.ai/free/en/papers/stepankova-2026-seem-cz-annotation-classification-epistemic/)
- [next → SELECting over Tokens: Curating Pre-training Data at Scale via Token Classification](https://md.research.elamur.ai/free/en/papers/tong-2026-selecting-over-tokens-curating-pre/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/42/)

## References

13 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. H. Ahmadian, J.E. Mottershead, and M.I. Friswell. Regularisation methods for finite element model updating. Mechanical Systems and Signal Processing, 12(1):47–64, 1998. ISSN 0888-3270. doi: https://doi.org/10.1006/mssp.1996.0133. URL https://www.sciencedirect.com/science/ article/pii/S088832709690… — [doi:10.1006/mssp.1996.0133](https://doi.org/10.1006/mssp.1996.0133) [↗](https://doi.org/10.1006/mssp.1996.0133)
2. Khalid Alemerien, Aram Al-Ghareeb, and Malek Zakarya Alksasbeh. Sentiment analysis of online reviews: A machine learning based approach with tf-idf vectorization. Journal of Mobile Multimedia, 20 (5):1089–1116, 2024. doi: 10.13052/jmm1550-4646. 2055. — [doi:10.13052/jmm1550-4646](https://doi.org/10.13052/jmm1550-4646) [find](https://scholar.google.com/scholar?q=Khalid+Alemerien%2C+Aram+Al%2DGhareeb%2C+and+Malek+Zakarya+Alksasbeh%2E+Sentiment+analysis+of+online+reviews%3A+A+machine+learning+based+approach+with+tf%2Didf+vectorization%2E+Journal+of+Mobile+Multimedia%2C+20+%285%29%3A)
3. Brett A. Becker, Paul Denny, James Finnie-Ansley, Andrew Luxton-Reilly, James Prather, and Eddie Antonio Santos. Programming is hard - or at least it used to be: Educational opportunities and challenges of ai code generation. SIGCSE 2023, New York, NY, USA, 2023. Association for Computing Machinery.… [find](https://scholar.google.com/scholar?q=Brett+A%2E+Becker%2C+Paul+Denny%2C+James+Finnie%2DAnsley%2C+Andrew+Luxton%2DReilly%2C+James+Prather%2C+and+Eddie+Antonio+Santos%2E+Programming+is+hard+%2D+or+at+least+it+used+to+be%3A+Educational+opportunities+and+challeng)
4. Zhangyin Feng et al. [CodeBERT: A pre-trained model for programming and natural languages](https://md.research.elamur.ai/free/en/papers/feng-2020-codebert-pre-trained-model-programming/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1536–1547. Association for Computational Linguistics, 2020.
5. Daya Guo et al. [UniXcoder: Unified cross-modal pre-training for code representation](https://md.research.elamur.ai/free/en/papers/guo-2022-unixcoder-unified-cross-modal-pre/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, pages 7212–7225. Association for Computational Linguistics, 2022.
6. Hamel Husain et al. [CodeSearchNet challenge: Evaluating the state of semantic code search](https://md.research.elamur.ai/free/en/papers/husain-2020-codesearchnet-challenge-evaluating-state-semantic/). arXiv preprint arXiv:1909.09436, 2019.
7. Daniil Orel, Dilshod Azizov, and Preslav Nakov. CoDet-m4: Detecting machine-generated code in multi-lingual, multi-generator and multi-domain settings. In Wanxiang Che, Joyce Nabende, Ekaterina Shutova, and Mohammad Taher Pilehvar, editors, Findings of the Association for Computational Linguistics:… [find](https://scholar.google.com/scholar?q=Daniil+Orel%2C+Dilshod+Azizov%2C+and+Preslav+Nakov%2E+CoDet%2Dm4%3A+Detecting+machine%2Dgenerated+code+in+multi%2Dlingual%2C+multi%2Dgenerator+and+multi%2Ddomain+settings%2E+In+Wanxiang+Che%2C+Joyce+Nabende%2C+Ekaterina+Shutov)
8. Daniil Orel, Indraneil Paul, Iryna Gurevych, and Preslav Nakov. Droid: A resource suite for ai-generated code detection. In Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, pages 31251–31277, 2025b. [find](https://scholar.google.com/scholar?q=Daniil+Orel%2C+Indraneil+Paul%2C+Iryna+Gurevych%2C+and+Preslav+Nakov%2E+Droid%3A+A+resource+suite+for+ai%2Dgenerated+code+detection%2E+In+Proceedings+of+the+2025+Conference+on+Empirical+Methods+in+Natural+Language+)
9. Daniil Orel, Dilshod Azizov, Indraneil Paul, Yuxia Wang, Iryna Gurevych, and Preslav Nakov. [SemEval-2026 task 13: Detecting machine-generated code with multiple programming languages, generators, and application scenarios](https://md.research.elamur.ai/free/en/papers/orel-2026-semeval-2026-task-13-detecting/). In Proceedings of the 20th International Workshop on Semantic Evaluation (Sem…
10. Wei Hung Pan, Ming Jie Chok, Jonathan Leong Shan Wong, Yung Xin Shin, Yeong Shian Poon, Zhou Yang, Chun Yong Chong, David Lo, and Mei Kuan Lim. Assessing ai detectors in identifying ai-generated code: Implications for education. ICSE-SEET ’24, page 1–11, New York, NY, USA, 2024. Association for Comp… — [doi:10.1145/3639474](https://doi.org/10.1145/3639474) [find](https://scholar.google.com/scholar?q=Wei+Hung+Pan%2C+Ming+Jie+Chok%2C+Jonathan+Leong+Shan+Wong%2C+Yung+Xin+Shin%2C+Yeong+Shian+Poon%2C+Zhou+Yang%2C+Chun+Yong+Chong%2C+David+Lo%2C+and+Mei+Kuan+Lim%2E+Assessing+ai+detectors+in+identifying+ai%2Dgenerated+code%3A)
11. Vinu Sankar Sadasivan, Aounon Kumar, Sriram Balasubramanian, Wenxiao Wang, and Soheil Feizi. [Can ai-generated text be reliably detected?](https://md.research.elamur.ai/free/en/papers/sadasivan-2023-ai-generated-text-reliably-detected/), 2025. URL https://arxiv.org/abs/2303.11156. [↗](https://arxiv.org/abs/2303.11156)
12. Jin Wen, Qiang Hu, Yuejun Guo, Maxime Cordy, and Yves Le Traon. Variable renaming-based adversarial test generation for code model: Benchmark and enhancement. 35(1), December 2025. ISSN 1049-331X. doi: 10.1145/3723353. URL https://doi.org/10.1145/3723353. — [doi:10.1145/3723353](https://doi.org/10.1145/3723353) [↗](https://doi.org/10.1145/3723353)
13. Shiwen Yu, Ting Wang, and Ji Wang. Data augmentation by program transformation. Journal of Systems and Software, 190: 111304, 2022. ISSN 0164-1212. doi: https://doi.org/10.1016/j.jss.2022.111304. URL https://www.sciencedirect.com/science/ article/pii/S0164121222000541. — [doi:10.1016/j.jss.2022.111304](https://doi.org/10.1016/j.jss.2022.111304) [↗](https://doi.org/10.1016/j.jss.2022.111304)

*Updated 1 November 2026*