---
title: "Stylometry at SemEval-2026 Task 13: Clustered Stylometric"
description: "Rather than fine-tuning large neural models, the authors build a lightweight stylometric pipeline. 15 references, cited by 1 paper here."
authors: ["Sruthi Santhanam", "Parthib Sarkar", "Yashvardhan Sharma"]
year: 2026
venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)"
source: "https://aclanthology.org/2026.semeval-1.172/"
doi: "10.18653/v1/2026.semeval-1.172"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/santhanam-2026-stylometry-semeval-2026-task-13/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/santhanam-2026-stylometry-semeval-2026-task-13/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/17/) / Stylometry at SemEval-2026 Task 13: Clustered Stylometric Modeling for Machine-Generated Code Detection

# Stylometry at SemEval-2026 Task 13: Clustered Stylometric Modeling for Machine-Generated Code Detection

**Authors:** Sruthi Santhanam, Parthib Sarkar and [Yashvardhan Sharma](https://md.research.elamur.ai/free/en/authors/yashvardhan-sharma/) · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.172/) · [PDF](https://aclanthology.org/2026.semeval-1.172.pdf) · [DOI 10.18653/v1/2026.semeval-1.172](https://doi.org/10.18653/v1/2026.semeval-1.172) · [OpenAlex](https://openalex.org/W7166876551)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/santhanam-2026-stylometry-semeval-2026-task-13/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses whether a program was written by a person or produced by an LLM, as posed in SemEval-2026 Task 13 Subtask A. Training data cover only a few languages and one algorithmic domain, while the test set adds unseen languages and domains. Rather than fine-tuning large neural models, the authors build a lightweight stylometric pipeline. Each snippet is represented by character-level TF-IDF n-grams plus standardized structural descriptors such as length, line count, comment, digit and punctuation ratios, and character diversity. K-means clustering then routes snippets to cluster-specific experts, acting as a simple mixture-of-experts. Each expert averages logistic regression and multinomial Naïve Bayes probabilities, and a decision threshold is tuned on validation data to maximize macro F1. The system scores 0.6125 macro F1 on the official test set and places 23rd. This shows that cheap, interpretable signals tied to verbosity, formatting and commenting carry useful cross-domain information for spotting generated code.

## Key points
- **Training data:** roughly 91% Python, with C++ and Java making up the rest. AI samples come from 34 generators. Because of memory limits, a stratified subset of 80,000 training samples was used.
- **Exploratory analysis:** generated code is longer than human code (mean 1052 vs. about 600 characters) and has more comments (comment ratio 0.105 vs. 0.03). Human code is more punctuation-dense.
- **Configuration:**
  - Features: char 3–5-grams, vocabulary capped at 120,000, min_df of 5.
  - Clustering: MiniBatch K-means with K = 4.
  - Classifiers: L2 logistic regression and Naïve Bayes with α = 0.5.
  - Threshold: searched over [0.70, 0.99].
  - Tools: scikit-learn on Colab.
- **Ablation:** removing threshold calibration causes the largest drop, to 0.338. A single logistic regression performs almost identically to the full ensemble (0.640 vs. 0.639).
- **Error analysis:** on 1000 validation samples the error rate is 24.1%. Long, regular human code is often mislabeled as machine-generated. Code: https://github.com/parthib-s/semeval-2026-task13-stylometry.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Birla Institute of Technology and Science, Pilani](https://md.research.elamur.ai/free/en/universities/birla-institute-of-technology-and-science-pilani/) — University, Pilāni, India · 16 works · 41,879 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

## Cite

Cite as: Sruthi Santhanam, Parthib Sarkar and Yashvardhan Sharma. 2026. Stylometry at SemEval-2026 Task 13: Clustered Stylometric Modeling for Machine-Generated Code Detection. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{santhanam2026stylometry,
  title = {{Stylometry at SemEval-2026 Task 13: Clustered Stylometric Modeling for Machine-Generated Code Detection}},
  author = {Sruthi Santhanam, Parthib Sarkar and Yashvardhan Sharma},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.172},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.172}
}
```

- [← previous Style Transfer as Bias Mitigation: Diffusion Models for Synthetic Mental Health Text for Arabic](https://md.research.elamur.ai/free/en/papers/mankarious-2026-style-transfer-bias-mitigation-diffusion/)
- [next → SUAT-BMI at MEDIQA-EVAL 2026: An Ensemble Approach to Language Models as Judges for Automatic Rating of Medical Responses](https://md.research.elamur.ai/free/en/papers/peng-2026-suat-bmi-mediqa-eval-2026/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/17/)

## References

15 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aylin Caliskan-Islam, Richard Harang, Andrew Liu, Arvind Narayanan, Clare Voss, Fabian Yamaguchi, and Rachel Greenstadt. 2015. De-anonymizing programmers via code stylometry. In Proceedings of the 24th USENIX Security Symposium (USENIX Security 2015), pages 255–270, Washington, D.C. USENIX Associati… [find](https://scholar.google.com/scholar?q=Aylin+Caliskan%2DIslam%2C+Richard+Harang%2C+Andrew+Liu%2C+Arvind+Narayanan%2C+Clare+Voss%2C+Fabian+Yamaguchi%2C+and+Rachel+Greenstadt%2E+2015%2E+De%2Danonymizing+programmers+via+code+stylometry%2E+In+Proceedings+of+the+24t)
2. Zhangyin Feng, Daya Guo, Duyu Tang, Nan Duan, Xiaocheng Feng, Ming Gong, Linjun Shou, Bing Qin, and Ting Liu. 2020. [Codebert: A pre-trained model for programming and natural languages](https://md.research.elamur.ai/free/en/papers/feng-2020-codebert-pre-trained-model-programming/). In Findings of EMNLP.
3. Sebastian Gehrmann, Hendrik Strobelt, and Alexander Rush. 2019. [Gltr: Statistical detection and visualization of generated text](https://md.research.elamur.ai/free/en/papers/gehrmann-2019-gltr-statistical-detection-visualization-generated/). Proceedings of ACL System Demonstrations.
4. Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, and Geoffrey E. Hinton. 1991. Adaptive mixtures of local experts. Neural Computation, 3(1):79–87. [find](https://scholar.google.com/scholar?q=Robert+A%2E+Jacobs%2C+Michael+I%2E+Jordan%2C+Steven+J%2E+Nowlan%2C+and+Geoffrey+E%2E+Hinton%2E+1991%2E+Adaptive+mixtures+of+local+experts%2E+Neural+Computation%2C+3%281%29%3A79%E2%80%9387%2E)
5. Ganesh Jawahar, Muhammad Abdul-Mageed, and Laks V.S. Lakshmanan. 2020. [Automatic detection of machine generated text: A critical survey](https://md.research.elamur.ai/free/en/papers/jawahar-2020-automatic-detection-machine-generated-text/). In Proceedings of COLING.
6. Moshe Koppel, Jonathan Schler, and Shlomo Argamon. 2009. Computational methods in authorship attribution. Journal of the American Society for Information Science and Technology, 60(1):9–26. [find](https://scholar.google.com/scholar?q=Moshe+Koppel%2C+Jonathan+Schler%2C+and+Shlomo+Argamon%2E+2009%2E+Computational+methods+in+authorship+attribution%2E+Journal+of+the+American+Society+for+Information+Science+and+Technology%2C+60%281%29%3A9%E2%80%9326%2E)
7. Stuart Lloyd. 1982. Least squares quantization in PCM. IEEE Transactions on Information Theory, 28(2):129–137. [find](https://scholar.google.com/scholar?q=Stuart+Lloyd%2E+1982%2E+Least+squares+quantization+in+PCM%2E+IEEE+Transactions+on+Information+Theory%2C+28%282%29%3A129%E2%80%93137%2E)
8. Andrew McCallum and Kamal Nigam. 1998. [A comparison of event models for naive bayes text classification](https://md.research.elamur.ai/free/en/papers/mccallum-1998-comparison-event-models-naive-bayes/). In AAAI Workshop.
9. Eric Mitchell, Yoonho Lee, Alexander Khazatsky, Christopher D. Manning, and Chelsea Finn. 2023. [DetectGPT: Zero-shot machine-generated text detection using probability curvature](https://md.research.elamur.ai/free/en/papers/mitchell-2023-detectgpt-zero-shot-machine-generated/). In Proceedings of the 40th International Conference on Machine Learning (ICML 2023), volume 202 of Proceedings of Machine…
10. Daniil Orel, Dilshod Azizov, Indraneil Paul, Yuxia Wang, Iryna Gurevych, and Preslav Nakov. 2026a. [AICD bench: A challenging benchmark for AI-generated code detection](https://md.research.elamur.ai/free/en/papers/orel-2026-aicd-bench-challenging-benchmark-ai/). In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Paper…
11. Daniil Orel, Dilshod Azizov, Indraneil Paul, Yuxia Wang, Iryna Gurevych, and Preslav Nakov. 2026b. [Semeval-2026 task 13: Detecting machine-generated code with multiple programming languages, generators, and application scenarios](https://md.research.elamur.ai/free/en/papers/orel-2026-semeval-2026-task-13-detecting/). In Proceedings of the 20th International Workshop on Semantic Evaluati…
12. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, and Ilya Sutskever. 2019. Language models are unsupervised multitask learners. In OpenAI Technical Report. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+and+Ilya+Sutskever%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+In+OpenAI+Technical+Report%2E)
13. Juan Ramos. 2003. Using tf-idf to determine word relevance in document queries. In ML Conference. [find](https://scholar.google.com/scholar?q=Juan+Ramos%2E+2003%2E+Using+tf%2Didf+to+determine+word+relevance+in+document+queries%2E+In+ML+Conference%2E)
14. Efstathios Stamatatos. 2009. A survey of modern authorship attribution methods. Journal of the American Society for Information Science and Technology, 60(3):538–556. [find](https://scholar.google.com/scholar?q=Efstathios+Stamatatos%2E+2009%2E+A+survey+of+modern+authorship+attribution+methods%2E+Journal+of+the+American+Society+for+Information+Science+and+Technology%2C+60%283%29%3A538%E2%80%93556%2E)
15. Yue Wang, Weishi Wang, Shafiq Joty, and Steven Hoi. 2021. Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. In EMNLP. [find](https://scholar.google.com/scholar?q=Yue+Wang%2C+Weishi+Wang%2C+Shafiq+Joty%2C+and+Steven+Hoi%2E+2021%2E+Codet5%3A+Identifier%2Daware+unified+pre%2Dtrained+encoder%2Ddecoder+models+for+code+understanding+and+generation%2E+In+EMNLP%2E)

*Updated 31 October 2026*