---
title: "PFW Task 8 at SemEval-2026 Task 8: Lightweight Tri-Fusion"
description: "This SemEval-2026 Task 8 (MTRAGEval) submission addresses multi-turn retrieval-augmented… 12 references, cited by 1 paper here."
authors: ["Taleef Tamsal", "Jonathan Rusert"]
year: 2026
venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)"
source: "https://aclanthology.org/2026.semeval-1.198/"
doi: "10.18653/v1/2026.semeval-1.198"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/tamsal-2026-pfw-task-8-semeval-2026/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/tamsal-2026-pfw-task-8-semeval-2026/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/16/) / PFW Task 8 at SemEval-2026 Task 8: Lightweight Tri-Fusion Retrieval with Prompt-Engineered Faithful Generation for Multi-Turn RAG

# PFW Task 8 at SemEval-2026 Task 8: Lightweight Tri-Fusion Retrieval with Prompt-Engineered Faithful Generation for Multi-Turn RAG

**Authors:** [Taleef Tamsal](https://md.research.elamur.ai/free/en/authors/taleef-tamsal/) and [Jonathan Rusert](https://md.research.elamur.ai/free/en/authors/jonathan-rusert/) · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.198/) · [PDF](https://aclanthology.org/2026.semeval-1.198.pdf) · [DOI 10.18653/v1/2026.semeval-1.198](https://doi.org/10.18653/v1/2026.semeval-1.198) · [OpenAlex](https://openalex.org/W7166833339)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/tamsal-2026-pfw-task-8-semeval-2026/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This SemEval-2026 Task 8 (MTRAGEval) submission addresses multi-turn retrieval-augmented generation, where follow-up questions are often elliptical or ambiguous. Its evaluation spans four English document collections, namely Govt, FIQA, IBM Cloud and ClapNQ. The authors wanted a cheap, reproducible pipeline that needed no fine-tuning.

For retrieval, BM25, SPLADE-v3 and Jina Embeddings v4 are merged through weighted reciprocal rank fusion, and only the latest user turn serves as the query. For generation, zero-shot GPT-4o (Task B) and GPT-4o-mini (Task C) use a three-stage prompt that requires passage citations.

The system ranked 6th of 26 on Task B, beating the strongest baseline by 18.3%. It was weaker on retrieval (Task A) and near baseline end-to-end (Task C). Development analyses show three things:
- Explicit citation-format rules, not chain-of-thought, produce grounded answers.
- SPLADE-v3 carries most of the retrieval signal, and query rewriting helps.
- The system refuses unanswerable questions reliably but never asks for clarification.

The work gives a low-cost reference point and useful diagnostics for conversational RAG.

## Key points
- **Official results:** Task B H = 0.756 (6/26); Task C H = 0.533 (14/29); Task A nDCG@5 = 0.433 (20/38), below the ELSER-with-rewrite baseline (0.480). Generating the full test run cost about $5 via the OpenAI API.
- **Fusion setup:** the submitted fusion weights were 0.20/0.50/0.30 (BM25/SPLADE/dense) with k = 60.
- **Post-hoc retrieval rerun:** on the dev set at nDCG@5, SPLADE+Dense (0.376) and SPLADE-v3 alone (0.373) slightly beat the submitted tri-fusion (0.362). BM25 scored only 0.174.
- **Prompt ablation (balanced 100-example dev sample):** citation-format rules raised citation use from 4% to 93% (McNemar p ≈ 3×10⁻²⁷), while CoT alone had no effect. Under fixed context, GPT-4o improved Task C H from 0.463 (GPT-4o-mini) to 0.523.
- **Answerability breakdown:** UNANSWERABLE turns reached H = 0.990, but the 78 UNDERSPECIFIED turns scored only 0.039. On those turns the model answered directly 43 times and refused 35 times instead of asking clarifying questions.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Purdue University Fort Wayne](https://md.research.elamur.ai/free/en/universities/purdue-university-fort-wayne/) — University, Fort Wayne, United States · 2 works · 1 citation

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (1): [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 473 works

## Cite

Cite as: Taleef Tamsal and Jonathan Rusert. 2026. PFW Task 8 at SemEval-2026 Task 8: Lightweight Tri-Fusion Retrieval with Prompt-Engineered Faithful Generation for Multi-Turn RAG. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{tamsal2026pfw,
  title = {{PFW Task 8 at SemEval-2026 Task 8: Lightweight Tri-Fusion Retrieval with Prompt-Engineered Faithful Generation for Multi-Turn RAG}},
  author = {Taleef Tamsal and Jonathan Rusert},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.198},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.198}
}
```

- [← previous PEU Lab at SemEval-2026 Task 4: Pairwise Text Comparison using RoBERTa and Ranking Loss](https://md.research.elamur.ai/free/en/papers/ma-2026-peu-lab-semeval-2026-task/)
- [next → Phatthachdau at SemEval-2026 Task 9: A Multi-Stage Augment-Judge-Train Pipeline for Multilingual Online Polarization Detection](https://md.research.elamur.ai/free/en/papers/phat-2026-phatthachdau-semeval-2026-task-9/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/16/)

## References

12 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Gordon V. Cormack, Charles L. A. Clarke, and Stefan Büttcher. 2009. [Reciprocal rank fusion outperforms condorcet and individual rank learning methods](https://md.research.elamur.ai/free/en/papers/cormack-2009-reciprocal-rank-fusion-outperforms/). In Proceedings of the 32nd International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 758–759.
2. Shahul Es, Jithin James, Luis Espinosa Anke, and Steven Schockaert. 2024. [RAGAs: Automated evaluation of retrieval augmented generation](https://md.research.elamur.ai/free/en/papers/es-2023-ragas-automated-evaluation-retrieval-augmented/). In Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics: System Demonstrations, pages 150–158, St. Julians,…
3. Thibault Formal, Benjamin Piwowarski, and Stéphane Clinchant. 2021. [SPLADE: Sparse lexical and expansion model for first stage ranking](https://md.research.elamur.ai/free/en/papers/formal-2021-splade-sparse-lexical-expansion/). In Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, pages 2288–2292.
4. Luyu Gao, Xueguang Ma, Jimmy Lin, and Jamie Callan. 2023. [Precise zero-shot dense retrieval without relevance labels](https://md.research.elamur.ai/free/en/papers/gao-2022-precise-zero-shot-dense-retrieval/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1762–1777, Toronto, Canada. Association for Computational Li…
5. Yunah Jang, Kang-il Lee, Hyunkyung Bae, Hwanhee Lee, and Kyomin Jung. 2024. IterCQR: Iterative conversational query reformulation with retrieval guidance. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies… [find](https://scholar.google.com/scholar?q=Yunah+Jang%2C+Kang%2Dil+Lee%2C+Hyunkyung+Bae%2C+Hwanhee+Lee%2C+and+Kyomin+Jung%2E+2024%2E+IterCQR%3A+Iterative+conversational+query+reformulation+with+retrieval+guidance%2E+In+Proceedings+of+the+2024+Conference+of+the+)
6. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2021. Billion-scale similarity search with GPUs. IEEE Transactions on Big Data, 7(3):535–547. [find](https://scholar.google.com/scholar?q=Jeff+Johnson%2C+Matthijs+Douze%2C+and+Herv%C3%A9+J%C3%A9gou%2E+2021%2E+Billion%2Dscale+similarity+search+with+GPUs%2E+IEEE+Transactions+on+Big+Data%2C+7%283%29%3A535%E2%80%93547%2E)
7. Yannis Katsis, Sara Rosenthal, Kshitij Fadnis, Chulaka Gunasekara, Young-Suk Lee, Lucian Popa, Vraj Shah, Huaiyu Zhu, Danish Contractor, and Marina Danilevsky. 2025. mtRAG: A multi-turn conversational benchmark for evaluating Retrieval-Augmented Generation systems. Transactions of the Association fo… [find](https://scholar.google.com/scholar?q=Yannis+Katsis%2C+Sara+Rosenthal%2C+Kshitij+Fadnis%2C+Chulaka+Gunasekara%2C+Young%2DSuk+Lee%2C+Lucian+Popa%2C+Vraj+Shah%2C+Huaiyu+Zhu%2C+Danish+Contractor%2C+and+Marina+Danilevsky%2E+2025%2E+mtRAG%3A+A+multi%2Dturn+conversational)
8. Fengran Mo, Kelong Mao, Yutao Zhu, Yihong Wu, Kaiyu Huang, and Jian-Yun Nie. 2023. ConvGQR: Generative query reformulation for conversational search. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4998–5012, Toronto, Canada.… [find](https://scholar.google.com/scholar?q=Fengran+Mo%2C+Kelong+Mao%2C+Yutao+Zhu%2C+Yihong+Wu%2C+Kaiyu+Huang%2C+and+Jian%2DYun+Nie%2E+2023%2E+ConvGQR%3A+Generative+query+reformulation+for+conversational+search%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+A)
9. Sara Rosenthal, Yannis Katsis, Vraj Shah, Lihong He, Lucian Popa, and Marina Danilevsky. 2026a. [MTRAG-UN: A benchmark for open challenges in multi-turn RAG conversations](https://md.research.elamur.ai/free/en/papers/rosenthal-2026-mtrag-un-benchmark-open-challenges/). Preprint, arXiv:2602.23184.
10. Sara Rosenthal, Vraj Shah, Yannis Katsis, and Marina Danilevsky. 2026b. [SemEval-2026 task 8: MTRAGEval: Evaluating multi-turn RAG conversations](https://md.research.elamur.ai/free/en/papers/rosenthal-2026-semeval-2026-task-8-mtrageval/). In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026), San Diego, California. Association for Computational Linguistics.
11. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Brian Ichter, Fei Xia, Ed H. Chi, Quoc V. Le, and Denny Zhou. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In Advances in Neural Information Processing Systems, volume 35.
12. Fanghua Ye, Meng Fang, Shenghui Li, and Emine Yilmaz. 2023. Enhancing conversational search: Large language model-aided informative query rewriting. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 5985–6006, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Fanghua+Ye%2C+Meng+Fang%2C+Shenghui+Li%2C+and+Emine+Yilmaz%2E+2023%2E+Enhancing+conversational+search%3A+Large+language+model%2Daided+informative+query+rewriting%2E+In+Findings+of+the+Association+for+Computational+Li)

*Updated 1 November 2026*