--- title: "XiaoM at SemEval-2026 Task 7 — paper by Yao and Yang (2026)" description: "This system paper addresses SemEval-2026 Task 7, which tests how well language models handle everyday cultural… 8 references." authors: ["Xiao Yao", "Liang Yang"] year: 2026 venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)" source: "https://aclanthology.org/2026.semeval-1.213/" doi: "10.18653/v1/2026.semeval-1.213" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/yao-2026-xiaom-semeval-2026-task-7/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/yao-2026-xiaom-semeval-2026-task-7/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/57/) / XiaoM at SemEval-2026 Task 7: A Qwen-based System for Accurate Retrieval of Everyday Knowledge Across Diverse Languages and Cultures # XiaoM at SemEval-2026 Task 7: A Qwen-based System for Accurate Retrieval of Everyday Knowledge Across Diverse Languages and Cultures **Authors:** Xiao Yao and [Liang Yang](https://md.research.elamur.ai/free/en/authors/liang-yang/) · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.213/) · [PDF](https://aclanthology.org/2026.semeval-1.213.pdf) · [DOI 10.18653/v1/2026.semeval-1.213](https://doi.org/10.18653/v1/2026.semeval-1.213) · [OpenAlex](https://openalex.org/W7166849884) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary This system paper addresses SemEval-2026 Task 7, which tests how well language models handle everyday cultural knowledge across 26 languages and 30 countries/regions using an extended BLEnD benchmark. The task has two tracks: short-answer questions (SAQ) and multiple-choice questions (MCQ). The authors use Qwen2.5-7B-Instruct without fine-tuning. Their work focuses on engineering a reliable inference pipeline rather than improving the model's cultural knowledge. The pipeline has five stages: - input normalization; - prompts that make cultural, geographic and linguistic cues explicit; - fully deterministic decoding; - retries on failed inference and on failed file writes; - rule-based calibration that forces outputs into the required TSV formats. The paper's reported gains concern output consistency, failure rates and format compliance. It gives no official answer-accuracy figures. Its main value is as a practical template for producing valid, reproducible submissions under strict shared-task output constraints. ## Key points - **Model and benchmark:** Qwen2.5-7B-Instruct with sampling disabled, temperature 0 and a repetition penalty. BLEnD data was excluded from training and served for evaluation only. - **Consistency:** Over 5 repeated runs on 100 samples per track, consistency rose from 32% (SAQ) and 64% (MCQ) under default sampling (temperature 0.7) to 100% on both tracks. - **Retries:** On 1000 samples, the inference failure rate fell from 0.8% with no retry to 0.5% with one retry and 0.1% with two. - **Calibration:** SAQ answers are truncated to 1–3 words, with a fallback to "no answer". MCQ outputs are restricted to A–D, with invalid outputs defaulting to A, and are then converted to binary vectors. - **Format compliance:** Compliance rose from 79.35% with no calibration to 82.45% with basic cleaning and 89.55% with full calibration. However, the text elsewhere claims 100% compliance on the final submission, an inconsistency the paper does not explain. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Dalian University of Technology](https://md.research.elamur.ai/free/en/universities/dalian-university-of-technology/) — University, Dalian, China · 20 works · 46 citations 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works 🧭 Topics (1): [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 552 works ## Cite Cite as: Xiao Yao and Liang Yang. 2026. XiaoM at SemEval-2026 Task 7: A Qwen-based System for Accurate Retrieval of Everyday Knowledge Across Diverse Languages and Cultures. Proceedings of the 20th International Workshop on Semantic Evaluation (2026). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{yao2026xiaom, title = {{XiaoM at SemEval-2026 Task 7: A Qwen-based System for Accurate Retrieval of Everyday Knowledge Across Diverse Languages and Cultures}}, author = {Xiao Yao and Liang Yang}, booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)}, year = {2026}, doi = {10.18653/v1/2026.semeval-1.213}, url = {https://doi.org/10.18653/v1/2026.semeval-1.213} } ``` - [← previous WoVis: Interactive Visualization of Word Embeddings for Semantic Change in Historical and Dialectal Language Resources](https://md.research.elamur.ai/free/en/papers/miletic-2026-wovis-interactive-visualization-word-embeddings/) - [next → Xin1212 at NakbaVirality Shared Task: Frozen CLIP with Residual Adapter for Multimodal Virality Classification](https://md.research.elamur.ai/free/en/papers/zhang-2026-xin1212-nakbavirality-shared-task-frozen/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/57/) ## References 8 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. J. Anacleto and 1 others. 2006. Can common sense uncover cultural differences? In Artificial Intelligence in Theory and Practice. [find](https://scholar.google.com/scholar?q=J%2E+Anacleto+and+1+others%2E+2006%2E+Can+common+sense+uncover+cultural+differences%3F+In+Artificial+Intelligence+in+Theory+and+Practice%2E) 2. K. Arora and 1 others. 2022. Why exposure bias matters. In ACL Findings. [find](https://scholar.google.com/scholar?q=K%2E+Arora+and+1+others%2E+2022%2E+Why+exposure+bias+matters%2E+In+ACL+Findings%2E) 3. J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. 2018. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv. 4. D. Hershcovich and 1 others. 2022. Challenges in cross-cultural nlp. In ACL. [find](https://scholar.google.com/scholar?q=D%2E+Hershcovich+and+1+others%2E+2022%2E+Challenges+in+cross%2Dcultural+nlp%2E+In+ACL%2E) 5. We performed inference under three settings: no calibration, basic cleaning, full calibration. Statistical results are shown in Table 3. [find](https://scholar.google.com/scholar?q=We+performed+inference+under+three+settings%3A+no+calibration%2C+basic+cleaning%2C+full+calibration%2E+Statistical+results+are+shown+in+Table+3%2E) 6. J. Myung, N. Lee, and Y. Zhou. 2024. [Blend: A benchmark for llms on everyday knowledge in diverse cultures and languages](https://md.research.elamur.ai/free/en/papers/myung-2024-blend-benchmark-llms-everyday-knowledge/). In NeurIPS, volume 37. \#\#\# 5.5 Negative Results T.-P. Nguyen and 1 others. 2023. Extracting cultural commonsense knowledge at scale. In WWW. 7. Ineffective strategies include enabling gradients during inference, weak generic prompts, removing fallback rules, and lacking deterministic decoding. Stability, prompt design, and standardization are all essential. F. Petroni and 1 others. 2019. [Language models as knowledge bases?](https://md.research.elamur.ai/free/en/papers/petroni-2019-language-models-as-knowledge-bases/) In EMNLP. 8. D. Yin and 1 others. 2022. Geomlama: Geo-diverse commonsense probing. In EMNLP. [find](https://scholar.google.com/scholar?q=D%2E+Yin+and+1+others%2E+2022%2E+Geomlama%3A+Geo%2Ddiverse+commonsense+probing%2E+In+EMNLP%2E) *Updated 2 November 2026*