--- title: "FactUEP at SemEval-2026 Task 4 — paper by Sawinski (2026)" description: "This paper describes a system for SemEval-2026 Task 4 Track A. Given an anchor story and two… 7 references, cited by 1 paper here." authors: ["Marcin Sawinski"] year: 2026 venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)" source: "https://aclanthology.org/2026.semeval-1.147/" doi: "10.18653/v1/2026.semeval-1.147" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/sawinski-2026-factuep-semeval-2026-task-4/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sawinski-2026-factuep-semeval-2026-task-4/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/12/) / FactUEP at SemEval-2026 Task 4: Structured Narrative Similarity Scoring with Aspect Decomposition and Weak-Signal Gating # FactUEP at SemEval-2026 Task 4: Structured Narrative Similarity Scoring with Aspect Decomposition and Weak-Signal Gating **Authors:** Marcin Sawinski · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.147/) · [PDF](https://aclanthology.org/2026.semeval-1.147.pdf) · [DOI 10.18653/v1/2026.semeval-1.147](https://doi.org/10.18653/v1/2026.semeval-1.147) · [OpenAlex](https://openalex.org/W7166817144) **Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read ## Summary This paper describes a system for SemEval-2026 Task 4 Track A. Given an anchor story and two candidate stories, the system picks the candidate whose narrative is structurally closer to the anchor. Similarity is assessed by comparing the stories' Outcomes, their Abstract Theme, and their Course of Action. Human judgments here are highly subjective. Using Wikipedia-derived story triples (200 dev, 400 test), the author compared LLMs of several sizes and then built a pipeline around GPT-5.2. The pipeline has three steps: 1. JSON-schema prompts make the model break the comparison into per-aspect judgments and scores. 2. A "weak-signal gate" flags low-confidence predictions, using weighted aspect margins and the strength of the strongest single aspect. 3. Only the flagged cases are sent to an Answer Review prompt for reconsideration. Selective revision raised dev accuracy to 79.5% and reached 75.75% on test, which placed second on the leaderboard. Other findings: - Small 8B models performed close to chance. - Verbose reasoning did not improve single-pass accuracy. - Stripping surface details from the stories (decontextualization) hurt accuracy. The work shows that cheap, margin-based confidence estimates can direct extra LLM effort to the cases that need it. ## Key points - Baseline accuracy on dev with the Basic decomposition prompt: - Llama 3.1 8B and DeepSeek-R1 8B: 58.0% - GPT-OSS 20B: 65.5% - GPT-OSS 120B: 67.0% - GPT-5.2: 73.5% - The best single prompt was "No Monologue" (77.5% dev). It used detailed system instructions and allowed only a one-character answer. - Final gate settings were τS = 0.18 and τA = 0.25, with aspect weights of 0.4/0.3/0.3. The gate flagged 34.5% of dev items. On those flagged items, accuracy rose from 62.3% to 71.0%, while confident predictions were left untouched. - On test, Weak Gating beat Verbose Scoring by 1.5 points (75.75% vs. 74.25%). The gain was not statistically significant (McNemar p = 0.26; bootstrap 95% CI [−0.75, +3.75]). - Annotator agreement was low: Krippendorff's alpha was 0.33 overall and close to chance for individual aspects. The 1,900 LLM-generated synthetic training triples were of limited use. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Poznań University of Economics and Business](https://md.research.elamur.ai/free/en/universities/poznan-university-of-economics-and-business/) — University, Poznan, Poland · 3 works · 2 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works ## Cite Cite as: Marcin Sawinski. 2026. FactUEP at SemEval-2026 Task 4: Structured Narrative Similarity Scoring with Aspect Decomposition and Weak-Signal Gating. Proceedings of the 20th International Workshop on Semantic Evaluation (2026). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{sawinski2026factuep, title = {{FactUEP at SemEval-2026 Task 4: Structured Narrative Similarity Scoring with Aspect Decomposition and Weak-Signal Gating}}, author = {Marcin Sawinski}, booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)}, year = {2026}, doi = {10.18653/v1/2026.semeval-1.147}, url = {https://doi.org/10.18653/v1/2026.semeval-1.147} } ``` - [← previous EyeMulator: Improving Code Language Models by Mimicking Human Visual Attention](https://md.research.elamur.ai/free/en/papers/zhang-2026-eyemulator-improving-code-language-models/) - [next → FaithLens: Detecting and Explaining Faithfulness Hallucination](https://md.research.elamur.ai/free/en/papers/si-2026-faithlens-detecting-explaining-faithfulness-hallucination/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/12/) ## References 7 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Mohamed Nour Eljadiri and Diana Nurbakova. 2025. Team INSALyon2 at SemEval-2025 task 10: A zero-shot agentic approach to text classification. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 965–980, Vienna, Austria. Association for Computational Linguis… [find](https://scholar.google.com/scholar?q=Mohamed+Nour+Eljadiri+and+Diana+Nurbakova%2E+2025%2E+Team+INSALyon2+at+SemEval%2D2025+task+10%3A+A+zero%2Dshot+agentic+approach+to+text+classification%2E+In+Proceedings+of+the+19th+International+Workshop+on+Seman) 2. Hans Ole Hatzel, Ekaterina Artemova, Haimo Stiemer, Evelyn Gius, and Chris Biemann. 2026. SemEval-2025 task 4: Narrative similarity and narrative representation learning. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026), San Diego, CA, USA. [find](https://scholar.google.com/scholar?q=Hans+Ole+Hatzel%2C+Ekaterina+Artemova%2C+Haimo+Stiemer%2C+Evelyn+Gius%2C+and+Chris+Biemann%2E+2026%2E+SemEval%2D2025+task+4%3A+Narrative+similarity+and+narrative+representation+learning%2E+In+Proceedings+of+the+20th+In) 3. Hans Ole Hatzel and Chris Biemann. 2024. Story embeddings — narrative-focused representations of fictional stories. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 5931–5943, Miami, Florida, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Hans+Ole+Hatzel+and+Chris+Biemann%2E+2024%2E+Story+embeddings+%E2%80%94+narrative%2Dfocused+representations+of+fictional+stories%2E+In+Proceedings+of+the+2024+Conference+on+Empirical+Methods+in+Natural+Language+Proce) 4. Jakub Piskorski, Tarek Mahmoud, Nikolaos Nikolaidis, Ricardo Campos, Alipio Mario Jorge, Dimitar Dimitrov, Purificação Silvano, Roman Yangarber, Shivam Sharma, Tanmoy Chakraborty, Nuno Guimaraes, Elisa Sartori, Nicolas Stefanovitch, Zhuohan Xie, Preslav Nakov, and Giovanni Da San Martino. 2025. [find](https://scholar.google.com/scholar?q=Jakub+Piskorski%2C+Tarek+Mahmoud%2C+Nikolaos+Nikolaidis%2C+Ricardo+Campos%2C+Alipio+Mario+Jorge%2C+Dimitar+Dimitrov%2C+Purifica%C3%A7%C3%A3o+Silvano%2C+Roman+Yangarber%2C+Shivam+Sharma%2C+Tanmoy+Chakraborty%2C+Nuno+Guimaraes%2C+Elis) 5. SemEval 2025 task 10: Multilingual characterization and extraction of narratives from online news. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 2610– 2643, Vienna, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=SemEval+2025+task+10%3A+Multilingual+characterization+and+extraction+of+narratives+from+online+news%2E+In+Proceedings+of+the+19th+International+Workshop+on+Semantic+Evaluation+%28SemEval%2D2025%29%2C+pages+2610%E2%80%93+) 6. Iknoor Singh, Carolina Scarton, and Kalina Bontcheva. 2025. GateNLP at SemEval-2025 task 10: Hierarchical three-step prompting for multilingual narrative classification. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 148–154, Vienna, Austria. Associati… [find](https://scholar.google.com/scholar?q=Iknoor+Singh%2C+Carolina+Scarton%2C+and+Kalina+Bontcheva%2E+2025%2E+GateNLP+at+SemEval%2D2025+task+10%3A+Hierarchical+three%2Dstep+prompting+for+multilingual+narrative+classification%2E+In+Proceedings+of+the+19th+Int) 7. Ling Sun, Xue Wan, Yuyang Lin, Fengping Su, and Pengfei Chen. 2025. PATeam at SemEval-2025 task 10: Two-stage news analytical framework: Target-oriented semantic segmentation and sequence generation LLMs for cross-lingual entity and narrative analysis. In Proceedings of the 19th International Worksh… [find](https://scholar.google.com/scholar?q=Ling+Sun%2C+Xue+Wan%2C+Yuyang+Lin%2C+Fengping+Su%2C+and+Pengfei+Chen%2E+2025%2E+PATeam+at+SemEval%2D2025+task+10%3A+Two%2Dstage+news+analytical+framework%3A+Target%2Doriented+semantic+segmentation+and+sequence+generation+L) *Updated 31 October 2026*