---
title: "SemEval-2026 Task 13 — paper by S et al. (2026)"
description: "This paper describes team UdayThalavesh's system for SemEval-2026 Task 13 Subtask A, a binary task that separates… 17 references."
authors: ["Udaythalavesh S", "Rajalakshmi Sivanaiah", "Angel Deborah S"]
year: 2026
venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)"
source: "https://aclanthology.org/2026.semeval-1.132/"
doi: "10.18653/v1/2026.semeval-1.132"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/s-2026-semeval-2026-task-13-fine/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/s-2026-semeval-2026-task-13-fine/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/42/) / SemEval-2026 Task 13: Fine-tuned CodeBERT with Stratified Balancing, Dynamic Threshold Optimization, and Logit Bias Correction for Robust Multi-Language AI Code Detection

# SemEval-2026 Task 13: Fine-tuned CodeBERT with Stratified Balancing, Dynamic Threshold Optimization, and Logit Bias Correction for Robust Multi-Language AI Code Detection

**Authors:** Udaythalavesh S, [Rajalakshmi Sivanaiah](https://md.research.elamur.ai/free/en/authors/rajalakshmi-sivanaiah/) and [Angel Deborah S](https://md.research.elamur.ai/free/en/authors/angel-deborah-s/) · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.132/) · [PDF](https://aclanthology.org/2026.semeval-1.132.pdf) · [DOI 10.18653/v1/2026.semeval-1.132](https://doi.org/10.18653/v1/2026.semeval-1.132) · [OpenAlex](https://openalex.org/W7166806656)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
This paper describes team UdayThalavesh's system for SemEval-2026 Task 13 Subtask A, a binary task that separates human-written code from AI-generated code. The data come from AICD-Bench, which spans six programming languages and several generators. The authors fine-tuned microsoft/codebert-base on a class-balanced 200K subset drawn from about 500K training examples, which are roughly 65% human-written. Training ran on TPUs. After each epoch they searched for the decision threshold that maximised validation F1-macro. They also added a deterministic label flip step that inverts predictions after thresholding, meant to offset the model's skewed, overconfident probabilities. On validation the system reached F1-macro 0.874. On the hidden test set it scored only 0.53, ranking around 40th. The authors' error analysis attributes this drop to distribution shift across languages and generators, Python over-representation, and thresholds overfitted to validation. The paper's main value is as a cautionary case: decision-level tuning that helps in-distribution may not transfer to unseen code sources.

## Key points
- The model is microsoft/codebert-base (125M parameters), pretrained on CodeSearchNet, with a linear 768→2 head and 384-token inputs. Training used AdamW (lr 2e-5, weight decay 0.01) on TPUv3-8 with a global batch of 256 for 3 epochs.
- Balanced sampling took 100K human and 100K AI examples but did not control for programming language.
- The best validation threshold fell from 0.61 to 0.52 over the three epochs, with the threshold grid spanning 0.25–0.73.
- Ablation results on validation F1-macro:
  - removing the label flip: 0.765
  - removing balancing: 0.812
  - using a fixed 0.5 threshold: 0.847
  - training on the full 500K set: 0.836
  - replacing CodeBERT with BERT-base: 0.791
  - replacing CodeBERT with DeBERTa-v3: 0.862
- Validation reached 0.874, but the private leaderboard score was 0.53, ranking around 40th of more than 80 teams. Proposed future work includes CodeT5+, StarCoder, PEFT, and ensembles.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Sri Sivasubramaniya Nadar College of Engineering](https://md.research.elamur.ai/free/en/universities/sri-sivasubramaniya-nadar-college-of-engineering/) — University, Chennai, India · 10 works · 9 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

## Cite

Cite as: Udaythalavesh S, Rajalakshmi Sivanaiah and Angel Deborah S. 2026. SemEval-2026 Task 13: Fine-tuned CodeBERT with Stratified Balancing, Dynamic Threshold Optimization, and Logit Bias Correction for Robust Multi-Language AI Code Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{s2026semeval,
  title = {{SemEval-2026 Task 13: Fine-tuned CodeBERT with Stratified Balancing, Dynamic Threshold Optimization, and Logit Bias Correction for Robust Multi-Language AI Code Detection}},
  author = {Udaythalavesh S, Rajalakshmi Sivanaiah and Angel Deborah S},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.132},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.132}
}
```

- [← previous SemBench: A Universal Semantic Framework for LLM Evaluation](https://md.research.elamur.ai/free/en/papers/zubillaga-2026-sembench-universal-semantic-framework-llm/)
- [next → SemiAdapt: Semi-Supervised and Efficient LoRA-Based Domain Adaptation for Low-Resource Irish Machine Translation with Transformers](https://md.research.elamur.ai/free/en/papers/mcgiff-2026-semiadapt-semi-supervised-efficient-lora/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/42/)

## References

17 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Daniil Orel, Dilshod Azizov, Indraneil Paul, Yuxia Wang, Iryna Gurevych, and Preslav Nakov. 2026. [SemEval-2026 Task 13: Detecting Machine-Generated Code with Multiple Programming Languages, Generators, and Application Scenarios](https://md.research.elamur.ai/free/en/papers/orel-2026-semeval-2026-task-13-detecting/). In Proceedings of the 20th International Workshop on Semantic Evaluatio…
2. Daniil Orel, Dilshod Azizov, Indraneil Paul, Yuxia Wang, Iryna Gurevych, and Preslav Nakov. 2026. AICDBench: A Challenging Benchmark for AI-Generated Code Detection. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Daniil+Orel%2C+Dilshod+Azizov%2C+Indraneil+Paul%2C+Yuxia+Wang%2C+Iryna+Gurevych%2C+and+Preslav+Nakov%2E+2026%2E+AICDBench%3A+A+Challenging+Benchmark+for+AI%2DGenerated+Code+Detection%2E+In+Proceedings+of+the+19th+Confere)
3. Ting Liu, Daxin Jiang, and Ming Zhou. 2020. Code-BERT: A Pre-Trained Model for Programming and Natural Languages. In Findings of EMNLP 2020. [find](https://scholar.google.com/scholar?q=Ting+Liu%2C+Daxin+Jiang%2C+and+Ming+Zhou%2E+2020%2E+Code%2DBERT%3A+A+Pre%2DTrained+Model+for+Programming+and+Natural+Languages%2E+In+Findings+of+EMNLP+2020%2E)
4. Hamel Husain et al. 2019. [CodeSearchNet challenge: Evaluating the state of semantic code search](https://md.research.elamur.ai/free/en/papers/husain-2020-codesearchnet-challenge-evaluating-state-semantic/). arXiv:1909.09436.
5. Yue Wang, Weishi Wang, Shafiq Joty, and Steven C. H. Hoi. 2021. CodeT5: Identifier-aware Unified Pre-trained Encoder-Decoder Models for Code Understanding and Generation. In Proceedings of ACL 2021. [find](https://scholar.google.com/scholar?q=Yue+Wang%2C+Weishi+Wang%2C+Shafiq+Joty%2C+and+Steven+C%2E+H%2E+Hoi%2E+2021%2E+CodeT5%3A+Identifier%2Daware+Unified+Pre%2Dtrained+Encoder%2DDecoder+Models+for+Code+Understanding+and+Generation%2E+In+Proceedings+of+ACL+2021%2E)
6. Daya Guo, Shuai Lu, Nanyang Liu, Shengyu Fu, Shuo Ren, Alexey Svyatkovskiy, and Neel Sundaresan. 2022. [find](https://scholar.google.com/scholar?q=Daya+Guo%2C+Shuai+Lu%2C+Nanyang+Liu%2C+Shengyu+Fu%2C+Shuo+Ren%2C+Alexey+Svyatkovskiy%2C+and+Neel+Sundaresan%2E+2022%2E)
7. UniXCoder: Unified Multilingual Code Pre-training Transformer. In EMNLP 2022. [find](https://scholar.google.com/scholar?q=UniXCoder%3A+Unified+Multilingual+Code+Pre%2Dtraining+Transformer%2E+In+EMNLP+2022%2E)
8. Yue Wang, Hung Le, Sonse Shakhawat, Zhe Gan, Ahmed H. Awadallah, and Hao Peng. 2023. CodeT5+: Open Code Large Language Models for Code Understanding and Generation. arXiv:2305.07922. [find](https://scholar.google.com/scholar?q=Yue+Wang%2C+Hung+Le%2C+Sonse+Shakhawat%2C+Zhe+Gan%2C+Ahmed+H%2E+Awadallah%2C+and+Hao+Peng%2E+2023%2E+CodeT5%2B%3A+Open+Code+Large+Language+Models+for+Code+Understanding+and+Generation%2E+arXiv%3A2305%2E07922%2E)
9. Raymond Li et al. 2023. StarCoder: may the source be with you! arXiv:2305.06161. [find](https://scholar.google.com/scholar?q=Raymond+Li+et+al%2E+2023%2E+StarCoder%3A+may+the+source+be+with+you%21+arXiv%3A2305%2E06161%2E)
10. Loubna Ben Allal et al. 2023. [SantaCoder: don’t reach for the stars!](https://md.research.elamur.ai/free/en/papers/allal-2023-santacoder-don-t-reach-stars/) arXiv:2303.13424.
11. Eric Mitchell, Yoon Kim, Alexander W. Berglund, and Chelsea Finn. 2023. [DetectGPT: Zero-Shot Machine-Generated Text Detection using Probability Curvature](https://md.research.elamur.ai/free/en/papers/mitchell-2023-detectgpt-zero-shot-machine-generated/). In ICLR 2023.
12. John Kirchenbauer, Jonas Geiping, Yuxin Wen, Jonathan Katz, Ian Miers, and Tom Goldstein. 2023. [A Watermark for Large Language Models](https://md.research.elamur.ai/free/en/papers/kirchenbauer-2023-watermark-large-language-models/). In ICML 2023.
13. Daniil Orel, Indraneil Paul, Iryna Gurevych, and Preslav Nakov. 2025. Droid: A Resource Suite for AI-Generated Code Detection. In Proceedings of EMNLP 2025. [find](https://scholar.google.com/scholar?q=Daniil+Orel%2C+Indraneil+Paul%2C+Iryna+Gurevych%2C+and+Preslav+Nakov%2E+2025%2E+Droid%3A+A+Resource+Suite+for+AI%2DGenerated+Code+Detection%2E+In+Proceedings+of+EMNLP+2025%2E)
14. Marcin Buda, Atsuto Maki, and Maciej A. Zwolinski. 2018. A systematic study of the class imbalance problem in convolutional neural networks. Neural Networks, 106:249–259. [find](https://scholar.google.com/scholar?q=Marcin+Buda%2C+Atsuto+Maki%2C+and+Maciej+A%2E+Zwolinski%2E+2018%2E+A+systematic+study+of+the+class+imbalance+problem+in+convolutional+neural+networks%2E+Neural+Networks%2C+106%3A249%E2%80%93259%2E)
15. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On calibration of modern neural networks. In ICML. [find](https://scholar.google.com/scholar?q=Chuan+Guo%2C+Geoff+Pleiss%2C+Yu+Sun%2C+and+Kilian+Q%2E+Weinberger%2E+2017%2E+On+calibration+of+modern+neural+networks%2E+In+ICML%2E)
16. Zachary C. Lipton, Charles Elkan, and Balakrishnan Narayanaswamy. 2014. Optimal Thresholding of Classifiers to Maximize F1 Score. In Proceedings of ECML PKDD. [find](https://scholar.google.com/scholar?q=Zachary+C%2E+Lipton%2C+Charles+Elkan%2C+and+Balakrishnan+Narayanaswamy%2E+2014%2E+Optimal+Thresholding+of+Classifiers+to+Maximize+F1+Score%2E+In+Proceedings+of+ECML+PKDD%2E)
17. Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, [find](https://scholar.google.com/scholar?q=Zhangyin+Feng%2C+Daya+Guo%2C+Duyu+Tang%2C+Nan+Duan%2C+Xiaocheng+Feng%2C+Ming+Gong%2C+Linjun+Shou%2C+Bing+Qin%2C)

*Updated 1 November 2026*