--- title: "Duluth at SemEval-2026 Task 6: DeBERTa with LLM-Augmented" description: "The Duluth team built a system for SemEval-2026 Task 6 (CLARITY), which asks whether answers in U.S. presidential… 3 references." authors: ["Shujauddin Syed", "Ted Pedersen"] year: 2026 venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)" source: "https://aclanthology.org/2026.semeval-1.94/" doi: "10.18653/v1/2026.semeval-1.94" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/syed-2026-duluth-semeval-2026-task-6/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/syed-2026-duluth-semeval-2026-task-6/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/) / Duluth at SemEval-2026 Task 6: DeBERTa with LLM-Augmented Data for Unmasking Political Question Evasions # Duluth at SemEval-2026 Task 6: DeBERTa with LLM-Augmented Data for Unmasking Political Question Evasions **Authors:** Shujauddin Syed and [Ted Pedersen](https://md.research.elamur.ai/free/en/authors/ted-pedersen/) · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.94/) · [PDF](https://aclanthology.org/2026.semeval-1.94.pdf) · [DOI 10.18653/v1/2026.semeval-1.94](https://doi.org/10.18653/v1/2026.semeval-1.94) · [OpenAlex](https://openalex.org/W7166842174) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary The Duluth team built a system for SemEval-2026 Task 6 (CLARITY), which asks whether answers in U.S. presidential interviews actually respond to the question. Task 1 sorts question–answer pairs into three clarity labels: Clear Reply, Ambivalent and Clear Non-Reply. Task 2 uses nine finer evasion techniques. The team fine-tuned DeBERTa-V3-base with: - focal loss, - layer-wise learning rate decay, - two binary discourse features from the QEvasion data. The QEvasion training data is heavily skewed toward Ambivalent, so the authors added synthetic minority-class examples in two ways. Gemini 3 produced frame-based context-aware generations that fully balance the classes. Claude Sonnet 4.5 produced EDA-style lexical paraphrases that balance them only partly. With Gemini augmentation, the model reached 0.76 Macro F1 when evaluated on the Task 1 evaluation set and placed 8th among 40 teams. Most remaining errors come from mixing up Ambivalent and Clear Reply answers, which is also where human annotators tend to disagree. The study suggests that LLM-generated data can raise minority-class performance on subtle political discourse. Hedged or partial answers remain the main open problem. ## Key points - **Training data:** QEvasion has 3,448 training pairs (59.2% Ambivalent, 30.5% Clear Reply, 10.3% Clear Non-Reply) and a 308-sample test set. The final evaluation set has 237 samples. - **Augmentation:** - Gemini 3 (CASA-style) added 2,672 examples, giving 2,040 per class. - Claude Sonnet 4.5 (EDA-style) added 1,086 examples. - Synthetic examples were down-weighted in the loss: 0.7 for Gemini data, 0.5 for Claude data. - **Task 1 evaluation F1:** 0.69 with original data only, 0.74 with Claude augmentation and 0.76 with Gemini augmentation. The top system (TeleAI) reached 0.89, and the participant mean was 0.70. - **Task 2:** DeBERTa-V3-large with focal loss reached 0.45 in the development phase but fell to 0.28 in the evaluation phase (30th of 33). The authors attribute the drop to distribution shift. - **Error analysis and other models:** - Ambivalent–Clear Reply confusion accounts for 68% of test errors and 65% of evaluation errors. - Political DEBATE reached 0.57 test F1, and DeBERTa-v3-large-nli reached 0.66. - The majority-class baseline scored 0.27. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of Minnesota, Duluth](https://md.research.elamur.ai/free/en/universities/university-of-minnesota-duluth/) — University, Duluth, United States · 5 works · 1,160 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 General Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/general-social-sciences/) · 227 works 🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works ## Cite Cite as: Shujauddin Syed and Ted Pedersen. 2026. Duluth at SemEval-2026 Task 6: DeBERTa with LLM-Augmented Data for Unmasking Political Question Evasions. Proceedings of the 20th International Workshop on Semantic Evaluation (2026). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{syed2026duluth, title = {{Duluth at SemEval-2026 Task 6: DeBERTa with LLM-Augmented Data for Unmasking Political Question Evasions}}, author = {Shujauddin Syed and Ted Pedersen}, booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)}, year = {2026}, doi = {10.18653/v1/2026.semeval-1.94}, url = {https://doi.org/10.18653/v1/2026.semeval-1.94} } ``` - [← previous DualFact+: A Multimodal Fact Verification Framework for Procedural Video Captioning](https://md.research.elamur.ai/free/en/papers/oguz-2026-dualfact-multimodal-fact-verification-framework/) - [next → Dynamic Layer Selection for Efficient Tone Recognition in Self-Supervised Speech Models](https://md.research.elamur.ai/free/en/papers/obiang-2026-dynamic-layer-selection-efficient-tone/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/28/) ## References 3 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Table 5 shows the performance of the additional models on the development set (test scores were not submitted). [find](https://scholar.google.com/scholar?q=Table+5+shows+the+performance+of+the+additional+models+on+the+development+set+%28test+scores+were+not+submitted%29%2E) 2. Janet Bavelas, Alex Black, Lisa Bryson, and Jennifer Mullett. 1988. Political equivocation: A situational explanation. Journal of Language and Social Psychology, 7:137–145. [find](https://scholar.google.com/scholar?q=Janet+Bavelas%2C+Alex+Black%2C+Lisa+Bryson%2C+and+Jennifer+Mullett%2E+1988%2E+Political+equivocation%3A+A+situational+explanation%2E+Journal+of+Language+and+Social+Psychology%2C+7%3A137%E2%80%93145%2E) 3. Peter Bull. 1994. On identifying questions, replies, and non-replies in political interviews. Journal of Language and Social Psychology, 13:115–131. [find](https://scholar.google.com/scholar?q=Peter+Bull%2E+1994%2E+On+identifying+questions%2C+replies%2C+and+non%2Dreplies+in+political+interviews%2E+Journal+of+Language+and+Social+Psychology%2C+13%3A115%E2%80%93131%2E) *Updated 1 November 2026*