---
title: "CultRAG at SemEval-2026 Task 7: Hybrid Sparse-Dense"
description: "This paper presents CultRAG, a submission to SemEval-2026 Task 7 (BLEnD) Track 2. 77.55% accuracy. 12 references, cited by 1 paper here."
authors: ["Aditya Singh", "Rickarya Das"]
year: 2026
venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)"
source: "https://aclanthology.org/2026.semeval-1.393/"
doi: "10.18653/v1/2026.semeval-1.393"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/singh-2026-cultrag-semeval-2026-task-7/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/singh-2026-cultrag-semeval-2026-task-7/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/11/) / CultRAG at SemEval-2026 Task 7: Hybrid Sparse-Dense Retrieval with Entity-Centric Knowledge Bases for Cultural MCQ Answering

# CultRAG at SemEval-2026 Task 7: Hybrid Sparse-Dense Retrieval with Entity-Centric Knowledge Bases for Cultural MCQ Answering

**Authors:** Aditya Singh and Rickarya Das · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.393/) · [PDF](https://aclanthology.org/2026.semeval-1.393.pdf) · [DOI 10.18653/v1/2026.semeval-1.393](https://doi.org/10.18653/v1/2026.semeval-1.393) · [OpenAlex](https://openalex.org/W7166825378)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/singh-2026-cultrag-semeval-2026-task-7/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This paper presents CultRAG, a submission to SemEval-2026 Task 7 (BLEnD) Track 2. The track is a four-option multiple-choice benchmark of 47,014 English questions about everyday cultural knowledge across 30 country-language pairs. The authors asked whether retrieving from a curated cultural knowledge base could fill gaps in an LLM's knowledge of underrepresented cultures.

On top of Llama-3.1-8B-Instruct, they built a six-phase pipeline:
- spaCy entity-based query expansion
- hybrid BM25 + FAISS (all-MiniLM-L6-v2) retrieval combined via Reciprocal Rank Fusion
- country filtering and keyword-based intent detection
- tiered prompt routing and "anti-leak" prompting
- reranking by source trust tier

The result is negative. The model without retrieval scored 78.6%, and the full system reached 78.5%, a gap McNemar's test finds insignificant (p = 0.962). An oracle check showed the KB contains the answer for only 40.7% of questions, well below the model's own accuracy. Only the anti-leak instructions recovered the losses caused by retrieval. Country-level results suggest retrieval helps poorly represented cultures but hurts well-covered ones, which argues for selective, coverage-aware RAG.

## Key points
- **Knowledge base:** 1,262 chunks in 7 shards drawn from Wikipedia, travel guides, news and country portals. Each source gets a trust weight of 1.0 (high), 0.6 (mid) or 0.3 (low) for reranking.
- **Ablation results:**
  - Unfiltered RAG dropped accuracy to 77.9%.
  - Tiered routing (Phase 3) was the worst configuration at 77.6%.
  - Anti-leak filtering (Phase 4) brought it back to 78.5%.
  - Trust reranking and intent detection added nothing.
  - All effect sizes were negligible (Cohen's h ≤ 0.015).
- **KB coverage by country:** it varies widely, from Bulgaria (69%) and Ethiopia (65%) down to Great Britain (21%).
- **RAG gains and losses:**
  - Japan (+6.1pp) and Ethiopia (+4.1pp) gained most.
  - American Samoa (−6.2pp) and Great Britain (−3.0pp) lost most.
  - Overall, 14 countries improved and 14 got worse.
- **Official result:** 77.55% accuracy on 29 of 30 locales, ranking 12th of 18 systems. Per-locale scores ranged from es-MX (94.00%) to am-ET (63.05%).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Indian Institute of Technology Kharagpur](https://md.research.elamur.ai/free/en/universities/indian-institute-of-technology-kharagpur/) — University, Kharagpur, India · 25 works · 714 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

🧭 Topics (2): [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 552 works · [🕸️ Knowledge graphs & ontologies](https://md.research.elamur.ai/free/en/topics/knowledge-graphs/) · 211 works

## Cite

Cite as: Aditya Singh and Rickarya Das. 2026. CultRAG at SemEval-2026 Task 7: Hybrid Sparse-Dense Retrieval with Entity-Centric Knowledge Bases for Cultural MCQ Answering. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{singh2026cultrag,
  title = {{CultRAG at SemEval-2026 Task 7: Hybrid Sparse-Dense Retrieval with Entity-Centric Knowledge Bases for Cultural MCQ Answering}},
  author = {Aditya Singh and Rickarya Das},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.393},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.393}
}
```

- [← previous CUHKSZ Simultaneous Speech Translation System for IWSLT 2026](https://md.research.elamur.ai/free/en/papers/yang-2026-cuhksz-simultaneous-speech-translation-system/)
- [next → Cultural Benchmarking of LLMs in Standard and Dialectal Arabic Dialogues](https://md.research.elamur.ai/free/en/papers/kautsar-2026-cultural-benchmarking-llms-standard-dialectal/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/11/)

## References

12 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Gordon V. Cormack, Charles L. A. Clarke, and Stefan Buettcher. 2009. [Reciprocal rank fusion outperforms Condorcet and individual rank learning methods](https://md.research.elamur.ai/free/en/papers/cormack-2009-reciprocal-rank-fusion-outperforms/). In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 758–759.
2. Explosion. 2023. spACy: Industrial-strength natural language processing in Python. [https://spacy.io](https://spacy.io). [↗](https://spacy.io)
3. Debanjan Ghosh, Kai North, Ekaterina Kochmar, Mamoru Komachi, and Marcos Zampieri, editors. 2026. Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026). Association for Computational Linguistics, San Diego, United States. [find](https://scholar.google.com/scholar?q=Debanjan+Ghosh%2C+Kai+North%2C+Ekaterina+Kochmar%2C+Mamoru+Komachi%2C+and+Marcos+Zampieri%2C+editors%2E+2026%2E+Proceedings+of+the+20th+International+Workshop+on+Semantic+Evaluation+%28SemEval%2D2026%29%2E+Association+for+)
4. Nils Reimers and Iryna Gurevych. 2019. [Sentence-BERT: Sentence embeddings using siamese BERT-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992.
5. Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond. Foundations and Trends in Information Retrieval, 3(4):333–389. [find](https://scholar.google.com/scholar?q=Stephen+Robertson+and+Hugo+Zaragoza%2E+2009%2E+The+probabilistic+relevance+framework%3A+BM25+and+beyond%2E+Foundations+and+Trends+in+Information+Retrieval%2C+3%284%29%3A333%E2%80%93389%2E)
6. Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei. 2022. SimLM: Pre-training with representation bottleneck for document retrieval. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Pap… [find](https://scholar.google.com/scholar?q=Liang+Wang%2C+Nan+Yang%2C+Xiaolong+Huang%2C+Binxing+Jiao%2C+Linjun+Yang%2C+Daxin+Jiang%2C+Rangan+Majumder%2C+and+Furu+Wei%2E+2022%2E+SimLM%3A+Pre%2Dtraining+with+representation+bottleneck+for+document+retrieval%2E+In+Proceed)
7. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2021. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535–547. [find](https://scholar.google.com/scholar?q=Jeff+Johnson%2C+Matthijs+Douze%2C+and+Herve%CC%81+Je%CC%81gou%2E+2021%2E+Billion%2Dscale+similarity+search+with+GPUs%2E+IEEE+Transactions+on+Big+Data%2C+7%283%29%3A535%E2%80%93547%2E)
8. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. [Retrieval-augmented generation for knowledge-intensive NLP tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). In Advances in Neural Information Pr…
9. Alex Mallen, Akari Asai, Victor Zhong, Rajarshi Das, Hannaneh Hajishirzi, and Daniel Khashabi. 2023. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (V… [find](https://scholar.google.com/scholar?q=Alex+Mallen%2C+Akari+Asai%2C+Victor+Zhong%2C+Rajarshi+Das%2C+Hannaneh+Hajishirzi%2C+and+Daniel+Khashabi%2E+2023%2E+When+not+to+trust+language+models%3A+Investigating+effectiveness+of+parametric+and+non%2Dparametric+mem)
10. Meta AI. 2024. Llama 3 model card. [https://llama.meta.com/llama3](https://llama.meta.com/llama3). [↗](https://llama.meta.com/llama3)
11. Junho Myung, Nayeon Lee, Yi Zhou, Jiho Jin, Rifki Putri, Dimosthenis Antypas, Hsuvas Borkakoty, Eunsu Kim, Carla Perez-Almendros, Abinew Ali Ayele, and 1 others. 2024. [Blend: A benchmark for llms on everyday knowledge in diverse cultures and languages](https://md.research.elamur.ai/free/en/papers/myung-2024-blend-benchmark-llms-everyday-knowledge/). Advances in Neural Information Processing Syste…
12. Nedjma Ousidhoum, Junho Myung, Carla Perez-Almendros, Jiho Jin, Amr Keleg, Meriem Beloucif, Yi Zhou, Rodrigo Agerri, Vladimir Araujo, Naomi Baes, James Barry, Joanne Boisson, Nancy F. [find](https://scholar.google.com/scholar?q=Nedjma+Ousidhoum%2C+Junho+Myung%2C+Carla+Perez%2DAlmendros%2C+Jiho+Jin%2C+Amr+Keleg%2C+Meriem+Beloucif%2C+Yi+Zhou%2C+Rodrigo+Agerri%2C+Vladimir+Araujo%2C+Naomi+Baes%2C+James+Barry%2C+Joanne+Boisson%2C+Nancy+F%2E)

*Updated 31 October 2026*