--- title: "Clutch or Cry at SemEval-2026 Task 12: Offline" description: "The paper presents an approach built for the abductive event reasoning challenge… 0.72 exact match. 5 references, cited by 1 paper here." authors: ["Aayush Prasad", "Rudra Trivedi", "Arshad Khatib", "Shrikant Malviya", "Naveen Kumar"] year: 2026 venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)" source: "https://aclanthology.org/2026.semeval-1.261/" doi: "10.18653/v1/2026.semeval-1.261" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/prasad-2026-clutch-or-cry-semeval-2026/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/prasad-2026-clutch-or-cry-semeval-2026/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/11/) / Clutch or Cry at SemEval-2026 Task 12: Offline Retrieval-Augmented Generation with Frozen DeBERTa for Abductive Event Reasoning # Clutch or Cry at SemEval-2026 Task 12: Offline Retrieval-Augmented Generation with Frozen DeBERTa for Abductive Event Reasoning **Authors:** Aayush Prasad, Rudra Trivedi, Arshad Khatib, [Shrikant Malviya](https://md.research.elamur.ai/free/en/authors/shrikant-malviya/) and Naveen Kumar · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.261/) · [PDF](https://aclanthology.org/2026.semeval-1.261.pdf) · [DOI 10.18653/v1/2026.semeval-1.261](https://doi.org/10.18653/v1/2026.semeval-1.261) · [OpenAlex](https://openalex.org/W7166868553) **Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read ## Summary The paper presents an approach built for the abductive event reasoning challenge posed as Task 12 of SemEval-2026. Given a document corpus, the system must infer which answer options explain the causes or consequences of a target event. Any subset of options may be correct, and scoring uses Exact Match, so a single wrong or missing choice zeroes out the instance. The authors' first attempts failed. Fine-tuning large models on full documents memorized the small training set and did not transfer to test data. Smaller models had short context windows, so truncation removed the causal clues and the models effectively guessed. The final design separates evidence finding from decision making. An offline dense retriever selects one most relevant sentence for each observation. A partially frozen deberta-v3-large then scores each option independently against that sentence. Despite the RAG label, the second stage is a classifier rather than a text generator. This setup reached 0.72 Exact Match on the official test set. It shows that tightly limiting context and the number of trainable parameters can help in low-resource, all-or-nothing evaluation settings. ## Key points - **Retrieval:** all-MiniLM-L6-v2 embeds the corpus and ranks passages by cosine similarity. Using top k = 1 worked best; k = 3 and k = 5 added noisy or contradictory evidence and lowered accuracy. - **Classifier:** microsoft/deberta-v3-large (435M parameters) with its bottom 12 of 24 layers frozen. BCEWithLogitsLoss replaces Cross-Entropy so each option is judged as a separate binary decision. Inputs follow the pattern [CLS] context [SEP] event [SEP] option. - **Training setup:** - AdamW, learning rate 5e-6, weight decay 0.01. - Batch size 1 with 16-step gradient accumulation. - Maximum 512 tokens. - Up to 8 epochs with early stopping on dev Exact Match. - bfloat16 and gradient checkpointing on one NVIDIA H100. - **Ablation:** - A large model on full documents reached about 95% training accuracy but performed poorly on test. - microsoft/deberta-v3-small with inputs cut to 500 characters peaked at 49.57% validation accuracy, then fell to 34.19%. - The final system scored 0.72 on test. - **Limitation:** single-sentence retrieval cannot capture evidence spread across several sentences. The authors plan contradiction-aware multi-sentence retrieval as future work. Code is on GitHub (aayush-decoder/semeval-2026-abductive-event-reasoning). *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Sardar Vallabhbhai National Institute of Technology Surat](https://md.research.elamur.ai/free/en/universities/sardar-vallabhbhai-national-institute-of-technology-surat/) — University, Surat, India · 5 works · 4 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works 🧭 Topics (2): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,685 works · [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 552 works ## Cite Cite as: Aayush Prasad, Rudra Trivedi, Arshad Khatib, Shrikant Malviya and Naveen Kumar. 2026. Clutch or Cry at SemEval-2026 Task 12: Offline Retrieval-Augmented Generation with Frozen DeBERTa for Abductive Event Reasoning. Proceedings of the 20th International Workshop on Semantic Evaluation (2026). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{prasad2026clutch, title = {{Clutch or Cry at SemEval-2026 Task 12: Offline Retrieval-Augmented Generation with Frozen DeBERTa for Abductive Event Reasoning}}, author = {Aayush Prasad, Rudra Trivedi, Arshad Khatib, Shrikant Malviya and Naveen Kumar}, booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)}, year = {2026}, doi = {10.18653/v1/2026.semeval-1.261}, url = {https://doi.org/10.18653/v1/2026.semeval-1.261} } ``` - [← previous Clozing the Gap: Exploring Why Language Model Surprisal Outperforms Cloze Surprisal](https://md.research.elamur.ai/free/en/papers/nair-2026-clozing-gap-exploring-why-language/) - [next → CoAct: Co-Active LLM Preference Learning with Human-AI Synergy](https://md.research.elamur.ai/free/en/papers/xu-2026-coact-co-active-llm-preference/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/11/) ## References 5 references, 5 of them with a card of ours. ↗ leads to the work. 1. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2021. [Deberta: Decoding-enhanced bert with disentangled attention](https://md.research.elamur.ai/free/en/papers/he-2020-deberta-decoding-enhanced-bert-disentangled/). In International Conference on Learning Representations. 2. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. [Retrieval-augmented generation for knowledge-intensive nlp tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). Advances in Neural Information Proce… 3. Yujia Qin and 1 others. 2023. [Language models can improve event prediction by few-shot abductive reasoning](https://md.research.elamur.ai/free/en/papers/shi-2023-language-models-improve-event-prediction/). arXiv preprint arXiv:2305.16646. 4. Nils Reimers and Iryna Gurevych. 2019. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. 5. Thomas Wolf and 1 others. 2020. [Transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45. *Updated 31 October 2026*