--- title: "CICL26 at SemEval-2026 Task 4 — paper by Zhang and Yu (2026)" description: "A team from the University of Tübingen entered Track A of SemEval-2026 Task 4. 59% accuracy. 7 references, cited by 1 paper here." authors: ["Wanzhao Zhang", "Yue Yu"] year: 2026 venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)" source: "https://aclanthology.org/2026.semeval-1.310/" doi: "10.18653/v1/2026.semeval-1.310" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/zhang-2026-cicl26-semeval-2026-task-4/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2026-cicl26-semeval-2026-task-4/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/11/) / CICL26 at SemEval-2026 Task 4: Narrative Story Similarity and Narrative Representation Learning # CICL26 at SemEval-2026 Task 4: Narrative Story Similarity and Narrative Representation Learning **Authors:** Wanzhao Zhang and [Yue Yu](https://md.research.elamur.ai/free/en/authors/yue-yu/) · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.310/) · [PDF](https://aclanthology.org/2026.semeval-1.310.pdf) · [DOI 10.18653/v1/2026.semeval-1.310](https://doi.org/10.18653/v1/2026.semeval-1.310) · [OpenAlex](https://openalex.org/W7166884372) **Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read ## Summary A team from the University of Tübingen entered Track A of SemEval-2026 Task 4. In this track, a system receives one anchor story along with a pair of candidate stories and has to pick the candidate that is narratively closer in abstract theme, course of action and outcome. LLM judges can be unreliable in such pairwise choices, because random sampling and the order of candidates can change the answer. The team therefore built an inference-only cascade on top of GPT-4.1-mini, with no fine-tuning: - **Self-consistency voting:** three sampled calls are made in the original order and three with the candidates swapped, each settled by majority vote. - **Agreement check:** if the two orders agree, that answer is output. - **Disagreement rule:** otherwise, the side with the larger vote margin wins. Ties are settled by an extra voting round. The system reached 59% accuracy on the official CodaBench test set. The analysis shows that order sensitivity persists even under majority voting, which suggests that sampling-variance reduction and positional-bias mitigation are separate problems. ## Key points - **Data:** 39 sample, 200 development and 400 test triplets. The stories were human-written, human-written then LLM-edited, or sampled. The code is available on GitHub (cicl-iscl). - **Settings:** GPT-4.1-mini at temperature 0.3, N = 3 votes per order, and up to 5 API retries. The tie-break round is described as an extra forward vote, with temperature raised to 0.4 in the setup section. - **Result:** 59% test accuracy. The paper claims a gain over single-call inference but reports no single-call GPT-4.1-mini baseline on the test set. - **Ablation:** zero-shot GPT-3.5-turbo on the development set scored 52% (A–B order) and 50% (B–A order). Its judgments were inconsistent 46% of the time, and it tended to favor whichever story was shown second. - **Swap analysis:** even unanimous 3–0 votes often flipped after swapping, so internal agreement does not guarantee robustness to candidate order. - **Limitations:** dependence on a commercial API, higher inference cost from multiple calls, and only partial removal of order bias. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of Tübingen](https://md.research.elamur.ai/free/en/universities/university-of-tubingen/) — University, Tübingen, Germany · 75 works · 41,081 citations 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works ## Cite Cite as: Wanzhao Zhang and Yue Yu. 2026. CICL26 at SemEval-2026 Task 4: Narrative Story Similarity and Narrative Representation Learning. Proceedings of the 20th International Workshop on Semantic Evaluation (2026). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{zhang2026cicl26, title = {{CICL26 at SemEval-2026 Task 4: Narrative Story Similarity and Narrative Representation Learning}}, author = {Wanzhao Zhang and Yue Yu}, booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)}, year = {2026}, doi = {10.18653/v1/2026.semeval-1.310}, url = {https://doi.org/10.18653/v1/2026.semeval-1.310} } ``` - [← previous Children's English Reading Story Generation via Supervised Fine-Tuning of Compact LLMs with Controllable Difficulty and Safety](https://md.research.elamur.ai/free/en/papers/shen-2026-children-s-english-reading-story/) - [next → CiNet-Handai-Kyodai at SemEval-2026 Task 5: Combining LLM Prompting, Semantic Similarity, and Synthetic Gaze for Graded Sense Plausibility](https://md.research.elamur.ai/free/en/papers/pereira-2026-cinet-handai-kyodai-semeval-2026/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/11/) ## References 7 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, and 1 others. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in Neural Information Processing Systems, 33. 2. Hans Hatzel and Chris Biemann. 2024. Modeling narrative similarity across levels of abstraction. In Proceedings of *SEM. [find](https://scholar.google.com/scholar?q=Hans+Hatzel+and+Chris+Biemann%2E+2024%2E+Modeling+narrative+similarity+across+levels+of+abstraction%2E+In+Proceedings+of+%2ASEM%2E) 3. Hans Ole Hatzel, Ekaterina Artemova, Haimo Stiemer, Evelyn Gius, and Chris Biemann. 2026. SemEval-2026 Task 4: Narrative similarity and narrative representation learning. In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026), San Diego, CA, USA. Association for Comp… [find](https://scholar.google.com/scholar?q=Hans+Ole+Hatzel%2C+Ekaterina+Artemova%2C+Haimo+Stiemer%2C+Evelyn+Gius%2C+and+Chris+Biemann%2E+2026%2E+SemEval%2D2026+Task+4%3A+Narrative+similarity+and+narrative+representation+learning%2E+In+Proceedings+of+the+20th+In) 4. Aniket Keluskar and 1 others. 2024. Ambiguity and instability in large language model reasoning. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Aniket+Keluskar+and+1+others%2E+2024%2E+Ambiguity+and+instability+in+large+language+model+reasoning%2E+In+Proceedings+of+EMNLP%2E) 5. Andrew Piper. 2021. Enumerations: Data and Literary Study. University of Chicago Press. [find](https://scholar.google.com/scholar?q=Andrew+Piper%2E+2021%2E+Enumerations%3A+Data+and+Literary+Study%2E+University+of+Chicago+Press%2E) 6. Peiyi Wang, Lei Li, Liang Chen, Zefan Cai, Dawei Zhu, Binghuai Lin, Yunbo Cao, Lingpeng Kong, Qi Liu, Tianyu Liu, and Zhifang Sui. 2024. Large language models are not fair evaluators. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers),… [find](https://scholar.google.com/scholar?q=Peiyi+Wang%2C+Lei+Li%2C+Liang+Chen%2C+Zefan+Cai%2C+Dawei+Zhu%2C+Binghuai+Lin%2C+Yunbo+Cao%2C+Lingpeng+Kong%2C+Qi+Liu%2C+Tianyu+Liu%2C+and+Zhifang+Sui%2E+2024%2E+Large+language+models+are+not+fair+evaluators%2E+In+Proceedings+o) 7. Xuezhi Wang, Jason Wei, Dale Schuurmans, and 1 others. 2023. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). In International Conference on Learning Representations (ICLR). *Updated 31 October 2026*