---
title: "5ting at SemEval-2026 Task 8 — paper by T-Nguyen et al. (2026)"
description: "This paper describes 5ting, a team entry to SemEval-2026 Task 8 (MTRAGEval), which benchmarks… 15 references, cited by 1 paper here."
authors: ["Thien-Qua T-Nguyen", "Chi Hoang", "Nguyen Tran", "Tri Le", "Khanh Truong", "Chinh Nguyen"]
year: 2026
venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)"
source: "https://aclanthology.org/2026.semeval-1.254/"
doi: "10.18653/v1/2026.semeval-1.254"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/nguyen-2026-5ting-semeval-2026-task-8/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nguyen-2026-5ting-semeval-2026-task-8/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/9/) / 5ting at SemEval-2026 Task 8: Strong End-to-End Multi-Turn RAG via LLM-Based Reranking and Faithfulness Control

# 5ting at SemEval-2026 Task 8: Strong End-to-End Multi-Turn RAG via LLM-Based Reranking and Faithfulness Control

**Authors:** Thien-Qua T-Nguyen, Chi Hoang, [Nguyen Tran](https://md.research.elamur.ai/free/en/authors/nguyen-tran/), Tri Le, Khanh Truong and [Chinh Nguyen](https://md.research.elamur.ai/free/en/authors/chinh-nguyen/) · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.254/) · [PDF](https://aclanthology.org/2026.semeval-1.254.pdf) · [DOI 10.18653/v1/2026.semeval-1.254](https://doi.org/10.18653/v1/2026.semeval-1.254) · [OpenAlex](https://openalex.org/W7166847604)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/nguyen-2026-5ting-semeval-2026-task-8/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This paper describes 5ting, a team entry to SemEval-2026 Task 8 (MTRAGEval), which benchmarks retrieval-augmented generation in multi-turn conversations. In this setting, the final user question often depends on earlier turns, may be underspecified, and invites hallucination. The team took part in Subtask A (passage retrieval) and Subtask C (answer generation from the top-5 retrieved passages).

Their pipeline works as follows:
- GPT-4o-mini rewrites the last turn into a standalone query.
- Both the original and rewritten queries are embedded with BAAI/bge-m3 and searched in per-domain FAISS indices.
- The two result lists are merged, and GPT-4o-mini reranks the merged list in a single listwise pass.
- For answering, the prompt treats dialogue history only as a guide to user intent and treats the retrieved passages as the only source of facts.

The authors did not design a new retriever. They attribute most of their gains to reranking and prompt structure. The system placed 14th of 38 in Subtask A but 3rd of 29 in Subtask C. The authors read this gap as evidence that grounding on the generation side can offset moderate retrieval quality.

## Key points
- Data: MTRAG-based benchmark with 842 evaluation tasks over ClapNQ, FiQA, Cloud and Govt. The corpus has 78,170 documents split into 366,479 passages.
- Subtask A, test set: nDCG@5 = 0.4719, close to the ELSER baseline (0.48). FiQA was the hardest domain on dev (nDCG@5 = 0.3872); ClapNQ was the easiest (0.5112).
- Ablation on the dev set: LLM reranking raised nDCG@5 by 17.7% on rewritten queries. The full pipeline improved nDCG@5 by 27.2% relative to plain BGE-M3, from 0.3454 to 0.4393.
- Subtask C, test set: harmonic score 0.5597, with RL F = 0.7692, RB llm = 0.6784 and RB alg = 0.3867. The weakest component, RB alg, measures alignment with reference answers.
- Error analysis of 30 dev retrieval failures: 40% were unresolved coreference. Stated limitations are reliance on commercial APIs and about 1–2 seconds of extra reranking latency per query.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Vietnam National University Ho Chi Minh City](https://md.research.elamur.ai/free/en/universities/vietnam-national-university-ho-chi-minh-city/) — University, Ho Chi Minh City, Vietnam · 21 works · 14 citations
- [University of Information Technology](https://md.research.elamur.ai/free/en/universities/university-of-information-technology/) — University, Ho Chi Minh City, Vietnam · 13 works · 7 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works

🧭 Topics (1): [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 473 works

## Cite

Cite as: Thien-Qua T-Nguyen, Chi Hoang, Nguyen Tran, Tri Le, Khanh Truong and Chinh Nguyen. 2026. 5ting at SemEval-2026 Task 8: Strong End-to-End Multi-Turn RAG via LLM-Based Reranking and Faithfulness Control. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{tnguyen20265ting,
  title = {{5ting at SemEval-2026 Task 8: Strong End-to-End Multi-Turn RAG via LLM-Based Reranking and Faithfulness Control}},
  author = {Thien-Qua T-Nguyen, Chi Hoang, Nguyen Tran, Tri Le, Khanh Truong and Chinh Nguyen},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.254},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.254}
}
```

- [← previous XOXO: Stealthy Cross-Origin Context Poisoning Attacks against AI Coding Assistants](https://md.research.elamur.ai/free/en/papers/storek-2026-xoxo-stealthy-cross-origin-context/)
- [next → 6fanle Submission to the AmericasNLP 2026 Shared Task on Wixarika Image Captioning](https://md.research.elamur.ai/free/en/papers/wang-2026-6fanle-submission-americasnlp-2026-shared/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/9/)

## References

15 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jianlv Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. BGE [M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation](https://md.research.elamur.ai/free/en/papers/chen-2024-m3-embedding-multi-linguality/). In Findings of the Association for Computational Linguistics: ACL 2024.
2. Yunfan Gao, Yun Xiong, Xinyu Gao, Kangxiang Jia, Jinliu Pan, Yuxi Bi, Yi Dai, Jiawei Sun, Meng Wang, and Haofen Wang. 2024. [Retrieval-augmented generation for large language models: A survey](https://md.research.elamur.ai/free/en/papers/gao-2023-retrieval-augmented-generation-large-language/). Preprint, arXiv:2312.10997.
3. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019. [Billion-scale similarity search with GPUs](https://md.research.elamur.ai/free/en/papers/johnson-2017-billion-scale-similarity-search-gpus/). IEEE Transactions on Big Data, 7(3):535–547.
4. Yannis Katsis, Sara Rosenthal, Kshitij Fadnis, Chulaka Gunasekara, Young-Suk Lee, Lucian Popa, Vraj Shah, Huaiyu Zhu, Danish Contractor, and Marina Danilevsky. 2025. mtrag: A multi-turn conversational benchmark for evaluating retrieval-augmented generation systems. Transactions of the Association fo… [find](https://scholar.google.com/scholar?q=Yannis+Katsis%2C+Sara+Rosenthal%2C+Kshitij+Fadnis%2C+Chulaka+Gunasekara%2C+Young%2DSuk+Lee%2C+Lucian+Popa%2C+Vraj+Shah%2C+Huaiyu+Zhu%2C+Danish+Contractor%2C+and+Marina+Danilevsky%2E+2025%2E+mtrag%3A+A+multi%2Dturn+conversational)
5. Zihan Liu, Wei Ping, Rajarshi Roy, Peng Xu, Chankyu Lee, Mohammad Shoeybi, and Bryan Catanzaro. 2024. [ChatQA: Surpassing GPT-4 on conversational QA and RAG](https://md.research.elamur.ai/free/en/papers/liu-2024-chatqa-surpassing-gpt-4-conversational/). In Advances in Neural Information Processing Systems (NeurIPS), volume 37.
6. Xueguang Ma, Xinyu Zhang, Ronak Pradeep, and Jimmy Lin. 2023. Zero-shot listwise document reranking with a large language model. Preprint, arXiv:2305.02156. [find](https://scholar.google.com/scholar?q=Xueguang+Ma%2C+Xinyu+Zhang%2C+Ronak+Pradeep%2C+and+Jimmy+Lin%2E+2023%2E+Zero%2Dshot+listwise+document+reranking+with+a+large+language+model%2E+Preprint%2C+arXiv%3A2305%2E02156%2E)
7. Paul Owoicho, Ivan Sekulić, Mohammad Aliannejadi, Jeffrey Dalton, and Fabio Crestani. 2023. Exploiting simulated user feedback for conversational search: Ranking, rewriting, and beyond. In Proceedings of the 46th International ACM SIGIR Conference on Research and Development in Information Retrieva… [find](https://scholar.google.com/scholar?q=Paul+Owoicho%2C+Ivan+Sekulic%CC%81%2C+Mohammad+Aliannejadi%2C+Jeffrey+Dalton%2C+and+Fabio+Crestani%2E+2023%2E+Exploiting+simulated+user+feedback+for+conversational+search%3A+Ranking%2C+rewriting%2C+and+beyond%2E+In+Proceeding)
8. Ronak Pradeep, Sahel Sharifymoghaddam, and Jimmy Lin. 2023. Rankzephyr: Effective and robust zero-shot listwise reranking is a breeze! Preprint, arXiv:2312.02724. — [arXiv:2312.02724](https://arxiv.org/abs/2312.02724) [find](https://scholar.google.com/scholar?q=Ronak+Pradeep%2C+Sahel+Sharifymoghaddam%2C+and+Jimmy+Lin%2E+2023%2E+Rankzephyr%3A+Effective+and+robust+zero%2Dshot+listwise+reranking+is+a+breeze%21+Preprint%2C+arXiv%3A2312%2E02724%2E+%E2%80%94+%5BarXiv%3A2312%2E02724%5D%28https%3A%2F%2Farxiv%2Eor)
9. Sara Rosenthal, Yannis Katsis, Vraj Shah, Lihong He, Lucian Popa, and Marina Danilevsky. 2026a. [MTRAG-UN: A benchmark for open challenges in multi-turn RAG conversations](https://md.research.elamur.ai/free/en/papers/rosenthal-2026-mtrag-un-benchmark-open-challenges/). Preprint, arXiv:2602.23184.
10. Sara Rosenthal, Vraj Shah, Yannis Katsis, and Marina Danilevsky. 2026b. [SemEval-2026 task 8: MTRAGEval: Evaluating multi-turn RAG conversations](https://md.research.elamur.ai/free/en/papers/rosenthal-2026-semeval-2026-task-8-mtrageval/). In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026), San Diego, California. Association for Computational Linguistics.
11. Weiwei Sun, Lingyong Yan, Xinyu Ma, Shuaiqiang Wang, Pengjie Ren, Zhumin Chen, Dawei Yin, and Zhaochun Ren. 2023. [Is ChatGPT good at search? investigating large language models as re-ranking agents](https://md.research.elamur.ai/free/en/papers/sun-2023-chatgpt-good-search-investigating-large/). In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing (EMNLP). As…
12. [ROLE 1 — CONVERSATION HISTORY: For context resolution ONLY] The conversation history is provided exclusively to help you: • Identify what topic the conversation concerns. • Resolve coreferences and pronouns (e.g., “it”, “that option”, “the previous method”). [find](https://scholar.google.com/scholar?q=%5BROLE+1+%E2%80%94+CONVERSATION+HISTORY%3A+For+context+resolution+ONLY%5D+The+conversation+history+is+provided+exclusively+to+help+you%3A+%E2%80%A2+Identify+what+topic+the+conversation+concerns%2E+%E2%80%A2+Resolve+coreferences+and+p)
13. Svitlana Vakulenko, Shayne Longpre, Zhucheng Tu, and Raviteja Anantha. 2021. Question rewriting for conversational question answering. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining (WSDM), pages 355–363. ACM. [find](https://scholar.google.com/scholar?q=Svitlana+Vakulenko%2C+Shayne+Longpre%2C+Zhucheng+Tu%2C+and+Raviteja+Anantha%2E+2021%2E+Question+rewriting+for+conversational+question+answering%2E+In+Proceedings+of+the+14th+ACM+International+Conference+on+Web+Se)
14. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2024. [Improving text embeddings with large language models](https://md.research.elamur.ai/free/en/papers/wang-2024-improving-text-embeddings-large-language/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 11897–11916, Bangkok, Thailand. Asso…
15. Shengyao Zhuang, Honglei Zhuang, Bevan Koopman, and Guido Zuccon. 2024. A setwise approach for effective and highly efficient zero-shot ranking with large language models. In Proceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval. ACM. [find](https://scholar.google.com/scholar?q=Shengyao+Zhuang%2C+Honglei+Zhuang%2C+Bevan+Koopman%2C+and+Guido+Zuccon%2E+2024%2E+A+setwise+approach+for+effective+and+highly+efficient+zero%2Dshot+ranking+with+large+language+models%2E+In+Proceedings+of+the+47th+I)

*Updated 1 November 2026*