---
title: "PFW at SemEval-2026 Task 6: Multi-Seed DeBERTa Ensembles"
description: "The PFW team's system for SemEval-2026 Task 6 (CLARITY) labels answers in political interview question-answer pairs from… 16 references."
authors: ["Taleef Tamsal", "Jonathan Rusert"]
year: 2026
venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)"
source: "https://aclanthology.org/2026.semeval-1.197/"
doi: "10.18653/v1/2026.semeval-1.197"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/tamsal-2026-pfw-semeval-2026-task-6/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/tamsal-2026-pfw-semeval-2026-task-6/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/39/) / PFW at SemEval-2026 Task 6: Multi-Seed DeBERTa Ensembles for Political Response Clarity and Evasion Classification

# PFW at SemEval-2026 Task 6: Multi-Seed DeBERTa Ensembles for Political Response Clarity and Evasion Classification

**Authors:** [Taleef Tamsal](https://md.research.elamur.ai/free/en/authors/taleef-tamsal/) and [Jonathan Rusert](https://md.research.elamur.ai/free/en/authors/jonathan-rusert/) · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.197/) · [PDF](https://aclanthology.org/2026.semeval-1.197.pdf) · [DOI 10.18653/v1/2026.semeval-1.197](https://doi.org/10.18653/v1/2026.semeval-1.197) · [OpenAlex](https://openalex.org/W7166884216)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The PFW team's system for SemEval-2026 Task 6 (CLARITY) labels answers in political interview question-answer pairs from the QEvasion dataset (3,448 English training pairs). Subtask 1 assigns one of three clarity levels. Subtask 2 assigns one of nine heavily imbalanced evasion types.

Instead of prompting large language models, the authors fine-tune encoders: DeBERTa-xlarge for Subtask 1 and DeBERTa-v3-large for Subtask 2. To counter the strong seed-to-seed instability of fine-tuning, they combine 5-fold stratified cross-validation with 10 random seeds. This yields 50 models per architecture, whose logits are simply averaged. On the 237-item evaluation set, the system reached macro F1 of 0.76 and 0.50, placing mid-table behind LLM-based leaders.

The paper also observes that three post-hoc adjustments tuned on out-of-fold predictions raised cross-validation scores but lowered official ones. The authors cautiously call this an "optimization paradox." They recommend investing in more diverse models rather than tuning fixed predictions. The work shows that sub-1B, single-GPU models remain an affordable, reproducible baseline.

## Key points
- Official results: macro F1 0.76 on Subtask 1 (18th of 41) and 0.50 on Subtask 2 (12th of 33). The top systems scored 0.89 and 0.68.
- Seed variance is large. Single DeBERTa-v3-large models ranged from 0.601 to 0.698 on Subtask 1. Ensembling added +0.024 (Subtask 1) and +0.046 (Subtask 2) out-of-fold F1, and DeBERTa-xlarge rose from 0.663 to 0.679. Training all 100 models took about 50 GPU-hours on A100s.
- Every one of the three adjustments (learning weights for the ensemble, calibrating decision thresholds, and applying hierarchical masks) reduced evaluation F1 by between 0.02 and 0.10. Two of these drops fall within the ±0.06 confidence interval expected from sampling noise.
- A naive 5-shot Qwen2.5-32B-Instruct baseline showed strong label bias. It predicted Dodging for 48.7% of items and Explicit for only 4.5%, against a true 30.5%.
- Error analysis:
  - Ambivalent vs. Clear Reply confusion accounts for 74.4% of Subtask 1 out-of-fold errors.
  - Partial/half-answer is never predicted (F1 0.00).
  - Clarification reaches F1 0.655 despite being a minority class.
  - In later experiments, inverse-frequency class weighting gave +0.024, while focal loss did not help.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Purdue University Fort Wayne](https://md.research.elamur.ai/free/en/universities/purdue-university-fort-wayne/) — University, Fort Wayne, United States · 2 works · 1 citation

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 General Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/general-social-sciences/) · 227 works

## Cite

Cite as: Taleef Tamsal and Jonathan Rusert. 2026. PFW at SemEval-2026 Task 6: Multi-Seed DeBERTa Ensembles for Political Response Clarity and Evasion Classification. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{tamsal2026pfw,
  title = {{PFW at SemEval-2026 Task 6: Multi-Seed DeBERTa Ensembles for Political Response Clarity and Evasion Classification}},
  author = {Taleef Tamsal and Jonathan Rusert},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.197},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.197}
}
```

- [← previous PExA: Parallel Exploration Agent for Complex Text-to-SQL](https://md.research.elamur.ai/free/en/papers/parekh-2026-pexa-parallel-exploration-agent-complex/)
- [next → Phase 1 Implementation of LLM-generated Discharge Summaries showing high Adoption in a Dutch Academic Hospital](https://md.research.elamur.ai/free/en/papers/nadalini-2026-phase-1-implementation-llm-generated/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/39/)

## References

16 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Takuya Akiba, Shotaro Sano, Toshihiko Yanase, Takeru Ohta, and Masanori Koyama. 2019. [Optuna: A next-generation hyperparameter optimization framework](https://md.research.elamur.ai/free/en/papers/akiba-2019-optuna-next-generation-hyperparameter-optimization/).
2. In Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 2623–2631, Anchorage, Alaska, USA. ACM. [find](https://scholar.google.com/scholar?q=In+Proceedings+of+the+25th+ACM+SIGKDD+International+Conference+on+Knowledge+Discovery+and+Data+Mining%2C+pages+2623%E2%80%932631%2C+Anchorage%2C+Alaska%2C+USA%2E+ACM%2E)
3. Peter Bull. 2003. The Microanalysis of Political Communication: Claptrap and Ambiguity. Routledge, London, UK. [find](https://scholar.google.com/scholar?q=Peter+Bull%2E+2003%2E+The+Microanalysis+of+Political+Communication%3A+Claptrap+and+Ambiguity%2E+Routledge%2C+London%2C+UK%2E)
4. Steven E. Clayman. 2001. Answers and evasions. Language in Society, 30(3):403–442. [find](https://scholar.google.com/scholar?q=Steven+E%2E+Clayman%2E+2001%2E+Answers+and+evasions%2E+Language+in+Society%2C+30%283%29%3A403%E2%80%93442%2E)
5. Jesse Dodge, Gabriel Ilharco, Roy Schwartz, Ali Farhadi, Hannaneh Hajishirzi, and Noah A. Smith. 2020. Fine-tuning pretrained language models: Weight initializations, data orders, and early stopping. arXiv preprint arXiv:2002.06305. [find](https://scholar.google.com/scholar?q=Jesse+Dodge%2C+Gabriel+Ilharco%2C+Roy+Schwartz%2C+Ali+Farhadi%2C+Hannaneh+Hajishirzi%2C+and+Noah+A%2E+Smith%2E+2020%2E+Fine%2Dtuning+pretrained+language+models%3A+Weight+initializations%2C+data+orders%2C+and+early+stopping%2E+)
6. Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. 2017. On calibration of modern neural networks. In Proceedings of the 34th International Conference on Machine Learning, volume 70 of Proceedings of Machine Learning Research, pages 1321– 1330. PMLR. [find](https://scholar.google.com/scholar?q=Chuan+Guo%2C+Geoff+Pleiss%2C+Yu+Sun%2C+and+Kilian+Q%2E+Weinberger%2E+2017%2E+On+calibration+of+modern+neural+networks%2E+In+Proceedings+of+the+34th+International+Conference+on+Machine+Learning%2C+volume+70+of+Proceed)
7. Sandra Harris. 1991. Evasive action: How politicians respond to questions in political interviews. In Paddy Scannell, editor, Broadcast Talk, pages 76–99. Sage, London, UK. [find](https://scholar.google.com/scholar?q=Sandra+Harris%2E+1991%2E+Evasive+action%3A+How+politicians+respond+to+questions+in+political+interviews%2E+In+Paddy+Scannell%2C+editor%2C+Broadcast+Talk%2C+pages+76%E2%80%9399%2E+Sage%2C+London%2C+UK%2E)
8. Pengcheng He, Jianfeng Gao, and Weizhu Chen. 2023. [DeBERTaV3: Improving DeBERTa using ELECTRA-style pre-training with gradient-disentangled embedding sharing](https://md.research.elamur.ai/free/en/papers/he-2021-debertav3-improving-deberta-using-electra/). In International Conference on Learning Representations.
9. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. [DeBERTa: Decoding-enhanced BERT with disentangled attention](https://md.research.elamur.ai/free/en/papers/he-2020-deberta-decoding-enhanced-bert-disentangled/). In International Conference on Learning Representations.
10. Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. 2017. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems (NeurIPS). [find](https://scholar.google.com/scholar?q=Balaji+Lakshminarayanan%2C+Alexander+Pritzel%2C+and+Charles+Blundell%2E+2017%2E+Simple+and+scalable+predictive+uncertainty+estimation+using+deep+ensembles%2E+In+Advances+in+Neural+Information+Processing+Systems)
11. Tsung-Yi Lin, Priya Goyal, Ross Girshick, Kaiming He, and Piotr Dollár. 2017. [Focal loss for dense object detection](https://md.research.elamur.ai/free/en/papers/lin-2017-focal-loss-dense-object-detection/). In Proceedings of the IEEE International Conference on Computer Vision (ICCV), pages 2980– 2988, Venice, Italy. IEEE.
12. Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, Jake Vanderplas, Alexandre Passos, David Cournapeau, Matthieu Brucher, Matthieu Perrot, and Édouard Duchesnay. 2011. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12(85):2825–2830. [find](https://scholar.google.com/scholar?q=Mathieu+Blondel%2C+Peter+Prettenhofer%2C+Ron+Weiss%2C+Vincent+Dubourg%2C+Jake+Vanderplas%2C+Alexandre+Passos%2C+David+Cournapeau%2C+Matthieu+Brucher%2C+Matthieu+Perrot%2C+and+%C3%89douard+Duchesnay%2E+2011%2E+Scikit%2Dlearn%3A+Mach)
13. Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou, Chrysoula Zerva, and Giorgos Stamou. 2024. “I never said that”: A dataset, taxonomy and baselines on response clarity classification. In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 5204–5233, Miami, Florida… [find](https://scholar.google.com/scholar?q=Konstantinos+Thomas%2C+Giorgos+Filandrianos%2C+Maria+Lymperaiou%2C+Chrysoula+Zerva%2C+and+Giorgos+Stamou%2E+2024%2E+%E2%80%9CI+never+said+that%E2%80%9D%3A+A+dataset%2C+taxonomy+and+baselines+on+response+clarity+classification%2E+In+Fi)
14. Konstantinos Thomas, Giorgos Filandrianos, Maria Lymperaiou, Chrysoula Zerva, and Giorgos Stamou. 2026. [SemEval-2026 Task 6: CLARITY — unmasking political question evasions](https://md.research.elamur.ai/free/en/papers/thomas-2026-semeval-2026-task-6-clarity/). In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026). Association for Computational Linguis…
15. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, and 3 others. 2020. [Transf…](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
16. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, and 22 others. 2024. Qwen2.5 technical report. Preprint, arXiv:241… [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Yang%2C+Jiaxi+Yang%2C+Jin)

*Updated 1 November 2026*