---
title: "Howard University-AI4PC at SemEval-2026 Task 8: Query"
description: "The paper tackles multi-turn retrieval-augmented generation. 26 references, cited by 1 paper here."
authors: ["Sijan Shrestha", "Saurav Aryal"]
year: 2026
venue: "Proceedings of the 20th International Workshop on Semantic Evaluation (2026)"
source: "https://aclanthology.org/2026.semeval-1.320/"
doi: "10.18653/v1/2026.semeval-1.320"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shrestha-2026-howard-university-ai4pc-semeval-2026/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shrestha-2026-howard-university-ai4pc-semeval-2026/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/13/) / Howard University-AI4PC at SemEval-2026 Task 8: Query Reformulation and Dense-Lexical Retrieval Fusion for Multi-Turn Retrieval-Augmented Generation

# Howard University-AI4PC at SemEval-2026 Task 8: Query Reformulation and Dense-Lexical Retrieval Fusion for Multi-Turn Retrieval-Augmented Generation

**Authors:** Sijan Shrestha and [Saurav Aryal](https://md.research.elamur.ai/free/en/authors/saurav-aryal/) · **Year:** 2026 · **Venue:** [Proceedings of the 20th International Workshop on Semantic Evaluation (2026)](https://md.research.elamur.ai/free/en/venues/semeval/) ([SemEval 2026](https://md.research.elamur.ai/free/en/venues/semeval/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.semeval-1.320/) · [PDF](https://aclanthology.org/2026.semeval-1.320.pdf) · [DOI 10.18653/v1/2026.semeval-1.320](https://doi.org/10.18653/v1/2026.semeval-1.320) · [OpenAlex](https://openalex.org/W7166809938)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/shrestha-2026-howard-university-ai4pc-semeval-2026/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper tackles multi-turn retrieval-augmented generation. In this setting, follow-up questions depend on earlier dialogue, some cannot be answered from the collection, and the relevant passages change as the conversation moves on. The authors took part in every subtask of the SemEval-2026 Task 8 (MTRAGEval) shared task: retrieval (A), answering from given gold passages (B) and full end-to-end RAG (C). They used the four MTRAG corpora (ClapNQ, Cloud, FiQA, Govt), with test data drawn from MTRAG-UN.

Their pipeline requires no training. Llama-3.3-70B-Instruct turns the latest turn into a standalone query, splits it into sub-queries and writes a HyDE pseudo-answer. Candidates from BGE-base-en-v1.5 dense search and BM25 are merged with Reciprocal Rank Fusion and reordered by BGE-reranker-large. The same LLM then produces short extractive answers and declines when evidence is lacking.

Retrieval alone placed mid-table, but the end-to-end system finished 2nd of 29, close to the winner. This suggests that combining several query signals with faithful, abstention-aware generation can make up for mediocre retrieval, using only open models.

## Key points
- Official results: Task A nDCG@5 0.4098 (22nd/38; best was 0.5776). Task B harmonic mean 0.7462 (9th/26). Task C 0.5796 (2nd/29), within 1.1% of the top score of 0.5861.
- Task C retrieval combines several channels:
  - dense search, top 100;
  - BM25 (k1=1.5, b=0.75), top 100;
  - HyDE, top 50;
  - 2–3 sub-queries, top 33 each.
  
  The lists are fused with RRF (k=60), and the top 75 are reranked down to 5 passages for generation.
- Development-set runs on 842 queries (intermediate configurations, not controlled single-variable ablations) show nDCG@5 rising from 0.262 (dense only) to 0.334 (adding BM25 via RRF) and 0.381 (full pipeline).
- Faithfulness (RL F) reached 0.854 on Task B and 0.751 on Task C. The system abstained on 29.4% (B) and 27.6% (C) of 507 test queries, versus roughly 19% truly unanswerable, which implies some false refusals.
- FiQA was the hardest domain and Cloud the easiest. Code is released at github.com/shrsijan/mt-rag-benchmark.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Howard University](https://md.research.elamur.ai/free/en/universities/howard-university/) — University, Washington, United States · 2 works · 1 citation

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

🧭 Topics (1): [🔍 Information retrieval & search](https://md.research.elamur.ai/free/en/topics/information-retrieval/) · 552 works

## Cite

Cite as: Sijan Shrestha and Saurav Aryal. 2026. Howard University-AI4PC at SemEval-2026 Task 8: Query Reformulation and Dense-Lexical Retrieval Fusion for Multi-Turn Retrieval-Augmented Generation. Proceedings of the 20th International Workshop on Semantic Evaluation (2026).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{shrestha2026howard,
  title = {{Howard University-AI4PC at SemEval-2026 Task 8: Query Reformulation and Dense-Lexical Retrieval Fusion for Multi-Turn Retrieval-Augmented Generation}},
  author = {Sijan Shrestha and Saurav Aryal},
  booktitle = {Proceedings of the 20th International Workshop on Semantic Evaluation (2026)},
  year = {2026},
  doi = {10.18653/v1/2026.semeval-1.320},
  url = {https://doi.org/10.18653/v1/2026.semeval-1.320}
}
```

- [← previous How to Set the Learning Rate for Large-Scale Pre-training?](https://md.research.elamur.ai/free/en/papers/zhou-2026-how-set-learning-rate-large/)
- [next → HPLT 3.0: Very Large-Scale Multilingual Resources for LLMs and MT. Mono- and Bi-lingual Data, Multilingual Evaluation, and Pre-Trained Models](https://md.research.elamur.ai/free/en/papers/oepen-2026-hplt-3-0-very-large/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/13/)

## References

26 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Vaibhav Adlakha, Parishad BehnamGhader, Xing Han Lu, Nicholas Meade, and Siva Reddy. 2024. [Evaluating correctness and faithfulness of instruction-following models for question answering](https://md.research.elamur.ai/free/en/papers/adlakha-2024-evaluating-correctness-faithfulness-instruction-following/). Transactions of the Association for Computational Linguistics, 12:681–699.
2. Saurav Aryal and Mildness Akomoize. 2025. Howard university-ai4pc at semeval-2025 task 3: Logit-based supervised token classification for multilingual hallucination span identification using xgbod. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 1790–17… [find](https://scholar.google.com/scholar?q=Saurav+Aryal+and+Mildness+Akomoize%2E+2025%2E+Howard+university%2Dai4pc+at+semeval%2D2025+task+3%3A+Logit%2Dbased+supervised+token+classification+for+multilingual+hallucination+span+identification+using+xgbod%2E+In)
3. Saurav Aryal and Kritika Pant. 2025. Howard university-ai4pc at semeval-2025 task 9: Using open-weight bart-mnli for zero shot classification of food recall documents. In Proceedings of the 19th International Workshop on Semantic Evaluation (SemEval-2025), pages 1919–1923. [find](https://scholar.google.com/scholar?q=Saurav+Aryal+and+Kritika+Pant%2E+2025%2E+Howard+university%2Dai4pc+at+semeval%2D2025+task+9%3A+Using+open%2Dweight+bart%2Dmnli+for+zero+shot+classification+of+food+recall+documents%2E+In+Proceedings+of+the+19th+Inter)
4. Saurav Aryal and Howard Prioleau. 2023. Howard university computer science at semeval-2023 task 12: A 2-step system design for multilingual sentiment classification with language identification. In Proceedings of the 17th international workshop on semantic evaluation (SemEval-2023), pages 2153–2159. [find](https://scholar.google.com/scholar?q=Saurav+Aryal+and+Howard+Prioleau%2E+2023%2E+Howard+university+computer+science+at+semeval%2D2023+task+12%3A+A+2%2Dstep+system+design+for+multilingual+sentiment+classification+with+language+identification%2E+In+Pr)
5. Saurav K Aryal and Howard Prioleau. 2024. Ad-hoc ensemble approach for detecting adverse drug events in electronic health records. Journal of Computing Sciences in Colleges, 40(3):238–249. [find](https://scholar.google.com/scholar?q=Saurav+K+Aryal+and+Howard+Prioleau%2E+2024%2E+Ad%2Dhoc+ensemble+approach+for+detecting+adverse+drug+events+in+electronic+health+records%2E+Journal+of+Computing+Sciences+in+Colleges%2C+40%283%29%3A238%E2%80%93249%2E)
6. Saurav K Aryal, Ujjawal Shah, Howard Prioleau, and Legand Burge. 2023. Ensembling and modeling approaches for enhancing alzheimer’s disease scoring and severity assessment. In 2023 International Conference on Computational Science and Computational Intelligence (CSCI), pages 1364–1370. IEEE. [find](https://scholar.google.com/scholar?q=Saurav+K+Aryal%2C+Ujjawal+Shah%2C+Howard+Prioleau%2C+and+Legand+Burge%2E+2023%2E+Ensembling+and+modeling+approaches+for+enhancing+alzheimer%E2%80%99s+disease+scoring+and+severity+assessment%2E+In+2023+International+Confe)
7. Saurav Keshari Aryal and Gaurav Adhikari. 2023. Evaluating impact of emoticons and pre-processing on sentiment classification of translated african tweets. [find](https://scholar.google.com/scholar?q=Saurav+Keshari+Aryal+and+Gaurav+Adhikari%2E+2023%2E+Evaluating+impact+of+emoticons+and+pre%2Dprocessing+on+sentiment+classification+of+translated+african+tweets%2E)
8. Ho Bae. 2025. A study on enhancing zero-shot dense retrieval using query and hypothetical document embedding combination. The Transactions of the Korea Information Processing Society, 14(3):161–171. [find](https://scholar.google.com/scholar?q=Ho+Bae%2E+2025%2E+A+study+on+enhancing+zero%2Dshot+dense+retrieval+using+query+and+hypothetical+document+embedding+combination%2E+The+Transactions+of+the+Korea+Information+Processing+Society%2C+14%283%29%3A161%E2%80%93171%2E)
9. Eunsol Choi, He He, Mohit Iyyer, Mark Yatskar, Wen-tau Yih, Yejin Choi, Percy Liang, and Luke Zettlemoyer. 2018. [Quac: Question answering in context](https://md.research.elamur.ai/free/en/papers/choi-2018-quac-question-answering-context/). In Proceedings of the 2018 conference on empirical methods in natural language processing, pages 2174–2184.
10. Gordon V Cormack, Charles LA Clarke, and Stefan Buettcher. 2009. [Reciprocal rank fusion outperforms condorcet and individual rank learning methods](https://md.research.elamur.ai/free/en/papers/cormack-2009-reciprocal-rank-fusion-outperforms/). In Proceedings of the 32nd international ACM SIGIR conference on Research and development in information retrieval, pages 758–759.
11. Nouha Dziri, Ehsan Kamalloo, Sivan Milton, Osmar R Zaïane, Mo Yu, Edoardo M Ponti, and Siva Reddy. 2022. [Faithdial: A faithful benchmark for information-seeking dialogue](https://md.research.elamur.ai/free/en/papers/dziri-2022-faithdial-faithful-benchmark-information-seeking/). Transactions of the Association for Computational Linguistics, 10:1473– 1490.
12. Shahul Es, Jithin James, Luis Espinosa Anke, and Steven Schockaert. 2024. [Ragas: Automated evaluation of retrieval augmented generation](https://md.research.elamur.ai/free/en/papers/es-2023-ragas-automated-evaluation-retrieval-augmented/). In Proceedings of the 18th conference of the european chapter of the association for computational linguistics: system demonstrations, pages 150–158.
13. Luyu Gao, Xueguang Ma, Jimmy Lin, and Jamie Callan. 2023. [Precise zero-shot dense retrieval without relevance labels](https://md.research.elamur.ai/free/en/papers/gao-2022-precise-zero-shot-dense-retrieval/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1762–1777.
14. Desta Haileselassie Hagos, Saurav Keshari Aryal, Patrick Ymele-Leki, and Legand L Burge. 2025. Ai-driven multimodal colorimetric analytics for biomedical and behavioral health diagnostics. Computational and structural biotechnology journal, 27:2219–2232. [find](https://scholar.google.com/scholar?q=Desta+Haileselassie+Hagos%2C+Saurav+Keshari+Aryal%2C+Patrick+Ymele%2DLeki%2C+and+Legand+L+Burge%2E+2025%2E+Ai%2Ddriven+multimodal+colorimetric+analytics+for+biomedical+and+behavioral+health+diagnostics%2E+Computation)
15. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019. [Billion-scale similarity search with gpus](https://md.research.elamur.ai/free/en/papers/johnson-2017-billion-scale-similarity-search-gpus/). IEEE transactions on big data, 7(3):535–547.
16. Yannis Katsis, Sara Rosenthal, Kshitij Fadnis, Chulaka Gunasekara, Young-Suk Lee, Lucian Popa, Vraj Shah, Huaiyu Zhu, Danish Contractor, and Marina Danilevsky. 2025. mt RAG: A multi-turn conversational benchmark for evaluating retrieval-augmented generation systems. Transactions of the Association f… [find](https://scholar.google.com/scholar?q=Yannis+Katsis%2C+Sara+Rosenthal%2C+Kshitij+Fadnis%2C+Chulaka+Gunasekara%2C+Young%2DSuk+Lee%2C+Lucian+Popa%2C+Vraj+Shah%2C+Huaiyu+Zhu%2C+Danish+Contractor%2C+and+Marina+Danilevsky%2E+2025%2E+mt+RAG%3A+A+multi%2Dturn+conversationa)
17. Tzu-Lin Kuo, FengTing Liao, Mu-Wei Hsieh, Fu-Chieh Chang, Po-Chun Hsu, and Da-Shan Shiu. 2025. Rad-bench: Evaluating large language models’ capabilities in retrieval augmented dialogues. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational… [find](https://scholar.google.com/scholar?q=Tzu%2DLin+Kuo%2C+FengTing+Liao%2C+Mu%2DWei+Hsieh%2C+Fu%2DChieh+Chang%2C+Po%2DChun+Hsu%2C+and+Da%2DShan+Shiu%2E+2025%2E+Rad%2Dbench%3A+Evaluating+large+language+models%E2%80%99+capabilities+in+retrieval+augmented+dialogues%2E+In+Proceeding)
18. Mikel K Ngueajio, Saurav Aryal, Marcellin Atemkeng, Gloria Washington, and Danda Rawat. 2025. Decoding fake news and hate speech: A survey of explainable ai techniques. ACM Computing Surveys, 57(7):1–37. [find](https://scholar.google.com/scholar?q=Mikel+K+Ngueajio%2C+Saurav+Aryal%2C+Marcellin+Atemkeng%2C+Gloria+Washington%2C+and+Danda+Rawat%2E+2025%2E+Decoding+fake+news+and+hate+speech%3A+A+survey+of+explainable+ai+techniques%2E+ACM+Computing+Surveys%2C+57%287%29%3A1%E2%80%93)
19. Ronak Pradeep, Nandan Thakur, Sahel Sharifymoghaddam, Eric Zhang, Ryan Nguyen, Daniel Campos, Nick Craswell, and Jimmy Lin. 2025. Ragnarök: A reusable rag framework and baselines for trec 2024 retrieval-augmented generation track. In European Conference on Information Retrieval, pages 132–148. Sprin… [find](https://scholar.google.com/scholar?q=Ronak+Pradeep%2C+Nandan+Thakur%2C+Sahel+Sharifymoghaddam%2C+Eric+Zhang%2C+Ryan+Nguyen%2C+Daniel+Campos%2C+Nick+Craswell%2C+and+Jimmy+Lin%2E+2025%2E+Ragnar%C3%B6k%3A+A+reusable+rag+framework+and+baselines+for+trec+2024+retriev)
20. Howard Prioleau, Saurav K Aryal, and Jeremy Blackstone. 2025. Leveraging large language models for adverse drug event detection: A comparative study of token and span-based named entity recognition. In Biocomputing 2026: Proceedings of the Pacific Symposium, pages 205–218. [find](https://scholar.google.com/scholar?q=Howard+Prioleau%2C+Saurav+K+Aryal%2C+and+Jeremy+Blackstone%2E+2025%2E+Leveraging+large+language+models+for+adverse+drug+event+detection%3A+A+comparative+study+of+token+and+span%2Dbased+named+entity+recognition%2E+I)
21. Siva Reddy, Danqi Chen, and Christopher D Manning. 2019. [Coqa: A conversational question answering challenge](https://md.research.elamur.ai/free/en/papers/reddy-2019-coqa-conversational-question-answering-challenge/). Transactions of the Association for Computational Linguistics, 7:249–266.
22. Stephen Robertson and Hugo Zaragoza. 2009. The probabilistic relevance framework: BM25 and beyond, volume 4. Now Publishers Inc. [find](https://scholar.google.com/scholar?q=Stephen+Robertson+and+Hugo+Zaragoza%2E+2009%2E+The+probabilistic+relevance+framework%3A+BM25+and+beyond%2C+volume+4%2E+Now+Publishers+Inc%2E)
23. Sara Rosenthal, Yannis Katsis, Vraj Shah, Lihong He, Lucian Popa, and Marina Danilevsky. 2026a. [Mtrag-un: A benchmark for open challenges in multi-turn rag conversations](https://md.research.elamur.ai/free/en/papers/rosenthal-2026-mtrag-un-benchmark-open-challenges/). Preprint, arXiv:2602.23184.
24. Sara Rosenthal, Vraj Shah, Yannis Katsis, and Marina Danilevsky. 2026b. [Semeval-2026 task 8: Mtrageval: Evaluating multi-turn rag conversations](https://md.research.elamur.ai/free/en/papers/rosenthal-2026-semeval-2026-task-8-mtrageval/). In Proceedings of the 20th International Workshop on Semantic Evaluation (SemEval-2026), San Diego, California. Association for Computational Linguistics.
25. Saharsha Tiwari, Saurav K Aryal, and Legand Burge. 2025. Enhancing geospatial reasoning in large language models: An optimized retriever approach using r-tree-based point-in-polygon and nearest neighbor search. In International Conference on Information and Communication Technology for Intelligent S… [find](https://scholar.google.com/scholar?q=Saharsha+Tiwari%2C+Saurav+K+Aryal%2C+and+Legand+Burge%2E+2025%2E+Enhancing+geospatial+reasoning+in+large+language+models%3A+An+optimized+retriever+approach+using+r%2Dtree%2Dbased+point%2Din%2Dpolygon+and+nearest+neighb)
26. Shitao Xiao, Zheng Liu, Peitian Zhang, Niklas Muennighoff, Defu Lian, and Jian-Yun Nie. 2024. [C-pack: Packed resources for general chinese embeddings](https://md.research.elamur.ai/free/en/papers/xiao-2023-c-pack-packed-resources/). In Proceedings of the 47th international ACM SIGIR conference on research and development in information retrieval, pages 641–649.

*Updated 1 November 2026*