--- title: "SlugRAG at SemEval-2026 Task 8: Domain-Specific Fine-Tuning" description: "The paper describes SlugRAG, an entry to SemEval-2026 Task 8 (MTRAGEval), Subtask A. The subtask… 7 references, cited by 1 paper here." authors: ["Pratibha Revankar", "Jihye Kim", "Umit Azirakhmet"] year: 2026 venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)" source: "https://aclanthology.org/2026.semeval-1.135/" doi: "10.18653/v1/2026.semeval-1.135" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/revankar-2026-slugrag-semeval-2026-task-8/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/revankar-2026-slugrag-semeval-2026-task-8/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/17/) / SlugRAG at SemEval-2026 Task 8: Domain-Specific Fine-Tuning and Model Scaling for Multi-Turn RAG Retrieval # SlugRAG at SemEval-2026 Task 8: Domain-Specific Fine-Tuning and Model Scaling for Multi-Turn RAG Retrieval **Authors:** Pratibha Revankar, Jihye Kim and Umit Azirakhmet · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.135/) · [PDF](https://aclanthology.org/2026.semeval-1.135.pdf) · [DOI 10.18653/v1/2026.semeval-1.135](https://doi.org/10.18653/v1/2026.semeval-1.135) · [OpenAlex](https://openalex.org/W7166859295) **Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/revankar-2026-slugrag-semeval-2026-task-8/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary The paper describes SlugRAG, an entry to SemEval-2026 Task 8 (MTRAGEval), Subtask A. The subtask asks systems to retrieve relevant passages for conversational queries whose meaning often depends on earlier turns. The authors fine-tune BGE dense retrievers (bge-base-en-v1.5 and bge-large-en-v1.5) with MultipleNegativesRankingLoss. Training uses roughly 2,900 query–passage pairs from four English domains: CLAPNQ, FIQA, GOVT and CLOUD. They test improvements in four stages: - basic training settings; - learning-rate tuning and data augmentation; - BM25 hybrid fusion and cross-encoder reranking; - per-domain models and hard negative mining. They also try LLM query rewriting with Qwen2.5-3B-Instruct. Augmentation and per-domain training gave the largest gains. The best offline configuration was a multi-domain fine-tuned BGE-large, reaching nDCG@10 0.5101 and Recall@10 0.6221, against 0.3000 and 0.3800 for the untuned model. Reranking and hard negatives did not help. The official submission ranked 25th of 38 teams. The study is a practical guide to which inexpensive retriever adaptations pay off in conversational RAG. ## Key points - **Domain-specific models:** average gains over pre-trained BGE are 44.3% in Recall@10 and 47.8% in nDCG@10. CLAPNQ is strongest (R@10 0.6016, nDCG@10 0.4981), and CLOUD benefits most relative to the multi-domain model (+15.1%). - **Data augmentation:** back-translation paraphrases, synonym swaps and context expansion raised Recall@10 by 34.2%. - **Hard negative mining:** the mined-negative variants collapsed to about 0.14 nDCG@10. The authors blame using CosineSimilarityLoss instead of a contrastive loss, plus unlabeled relevant passages acting as false negatives. - **Reranking:** cross-encoder reranking left nDCG@10 almost unchanged (+0.05%) but cut Recall@10 by 6.5%. - **Query rewriting:** only "minimal clarification" rewriting helped on the baseline, by a small margin. - **Leaderboard:** the official score was nDCG@5 0.3847, versus 0.5776 for the top system. Offline scores come from a different test split, so the two sets of numbers are not comparable. - **Error analysis:** 100 failures were examined. Errors are dominated by coreference ambiguity (41%), temporal dependencies (28%) and multi-hop reasoning (19%). *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of California, Santa Cruz](https://md.research.elamur.ai/free/en/universities/university-of-california-santa-cruz/) — University, Santa Cruz, United States · 38 works · 3,604 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works 🧭 Topics (1): [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 552 works ## Cite Cite as: Pratibha Revankar, Jihye Kim and Umit Azirakhmet. 2026. SlugRAG at SemEval-2026 Task 8: Domain-Specific Fine-Tuning and Model Scaling for Multi-Turn RAG Retrieval. Proceedings of the 20th International Workshop on Semantic Evaluation (2026). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{revankar2026slugrag, title = {{SlugRAG at SemEval-2026 Task 8: Domain-Specific Fine-Tuning and Model Scaling for Multi-Turn RAG Retrieval}}, author = {Pratibha Revankar, Jihye Kim and Umit Azirakhmet}, booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)}, year = {2026}, doi = {10.18653/v1/2026.semeval-1.135}, url = {https://doi.org/10.18653/v1/2026.semeval-1.135} } ``` - [← previous SkiLLens: Recognising and Mapping Novel Skills from Millions of Job Ads Across Europe Using Language Models](https://md.research.elamur.ai/free/en/papers/santo-2026-skillens-recognising-mapping-novel-skills/) - [next → SMART-Editor: A Multi-Agent Framework for Human-Like Design Editing with Structural Integrity](https://md.research.elamur.ai/free/en/papers/mondal-2026-smart-editor-multi-agent-framework/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/17/) ## References 7 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Beijing Academy of Artificial Intelligence. 2023. BAAI General Embedding (BGE) Model. https: Ac-//github.com/FlagOpen/FlagEmbedding. cessed: January 2025. [find](https://scholar.google.com/scholar?q=Beijing+Academy+of+Artificial+Intelligence%2E+2023%2E+BAAI+General+Embedding+%28BGE%29+Model%2E+https%3A+Ac%2D%2F%2Fgithub%2Ecom%2FFlagOpen%2FFlagEmbedding%2E+cessed%3A+January+2025%2E) 2. Vladimir Karpukhin, Barlas Oğuz, Sewon Min, Patrick Lewis, Ledell Wu, Sergey Edunov, Danqi Chen, and Wen-tau Yih. 2020. [Dense Passage Retrieval for Open-Domain Question Answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6769– 6781, On… 3. Yannis Katsis, Sara Rosenthal, Kshitij Fadnis, Chulaka Gunasekara, Young-Suk Lee, Lucian Popa, Vraj Shah, Huaiyu Zhu, Danish Contractor, and Marina Danilevsky. 2025. mtrag: A multi-turn conversational benchmark for evaluating retrieval-augmented generation systems. Transactions of the Association fo… [find](https://scholar.google.com/scholar?q=Yannis+Katsis%2C+Sara+Rosenthal%2C+Kshitij+Fadnis%2C+Chulaka+Gunasekara%2C+Young%2DSuk+Lee%2C+Lucian+Popa%2C+Vraj+Shah%2C+Huaiyu+Zhu%2C+Danish+Contractor%2C+and+Marina+Danilevsky%2E+2025%2E+mtrag%3A+A+multi%2Dturn+conversational) 4. Nils Reimers and Iryna Gurevych. 2019. [Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,… 5. Sara Rosenthal, Yannis Katsis, Vraj Shah, Lihong He, Lucian Popa, and Marina Danilevsky. 2026a. [Mtrag-un: A benchmark for open challenges in multi-turn rag conversations](https://md.research.elamur.ai/free/en/papers/rosenthal-2026-mtrag-un-benchmark-open-challenges/). 6. Sara Rosenthal, Vraj Shah, Yannis Katsis, and Marina Danilevsky. 2026b. [Semeval-2026 task 8: Mtrageval: Evaluating multi-turn rag conversations](https://md.research.elamur.ai/free/en/papers/rosenthal-2026-semeval-2026-task-8-mtrageval/). In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026), San Diego, California. Association for Computational Linguistics.… 7. In all cases, the user message was formatted as Question: \\. [find](https://scholar.google.com/scholar?q=In+all+cases%2C+the+user+message+was+formatted+as+Question%3A+%5C%3Cquery%5C%3E%2E) *Updated 31 October 2026*