---
title: "DIRECT: Directional Relevance in Conversational Trajectories"
description: "Users of conversational assistants frequently struggle with opening a dialogue or deciding which question to pose next. 15 references."
authors: ["Anshuman Mourya", "Rajdeep Mukherjee", "Prerna Jolly", "Vinayak S Puranik", "Sivaramakrishnan R Kaveri"]
year: 2026
venue: "Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)"
source: "https://aclanthology.org/2026.eacl-industry.71/"
doi: "10.18653/v1/2026.eacl-industry.71"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/mourya-2026-direct-directional-relevance-conversational-trajectories/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/mourya-2026-direct-directional-relevance-conversational-trajectories/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/27/) / DIRECT: Directional Relevance in Conversational Trajectories

# DIRECT: Directional Relevance in Conversational Trajectories

**Authors:** Anshuman Mourya, Rajdeep Mukherjee, Prerna Jolly, [Vinayak S Puranik](https://md.research.elamur.ai/free/en/authors/vinayak-s-puranik/) and Sivaramakrishnan R Kaveri · **Year:** 2026 · **Venue:** [Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2026](https://md.research.elamur.ai/free/en/venues/eacl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.eacl-industry.71/) · [PDF](https://aclanthology.org/2026.eacl-industry.71.pdf) · [DOI 10.18653/v1/2026.eacl-industry.71](https://doi.org/10.18653/v1/2026.eacl-industry.71) · [OpenAlex](https://openalex.org/W7140113899)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Users of conversational assistants frequently struggle with opening a dialogue or deciding which question to pose next. This Amazon industry paper treats Next Question Suggestion (NQS) as a fast retrieval task rather than LLM generation. Generation is costly, slow, and can produce unsafe or unanswerable questions, so DIRECT instead selects candidates from a pre-built Question Bank.

The key difficulty is directionality. A good suggestion should be related to the query but not a paraphrase of it, and it should move the dialogue forward rather than ask about background information (a "prior"). DIRECT addresses this in three steps:
- domain-adaptive MLM pre-training,
- a dual-encoder retriever fine-tuned hierarchically so that relevant pairs fall between similar and irrelevant ones in similarity,
- a cross-encoder re-ranker trained to tell follow-ups from priors.

The system is evaluated on a proprietary dataset and on FQ-Bank. An LLM judge estimates click-through rate. DIRECT beats an off-the-shelf BGE-Large retriever while staying well within a 1-second latency budget. This makes it a practical recipe for low-latency suggestion features in commercial assistants.

## Key points
- Query–suggestion pairs fall into four categories: similar, follow-up, prior, and irrelevant. Only follow-ups should be shown to users, and priors serve as hard negatives for the re-ranker.
- Pre-training uses NER-guided masking: function words are filtered out, and 25% of the remaining domain-specific tokens are masked.
- Hierarchical fine-tuning improves Precision@Recall on the 3-way task by up to 0.08 on the proprietary data and up to 0.16 on FQ-Bank. On follow-up vs. prior detection, the gains are up to 0.12 and 0.1.
- In the Claude Sonnet 4.5 LLM-as-a-judge evaluation, predicted CTR rises 4% without the re-ranker and 6% with it, relative to vanilla BGE-Large. p90 latency is roughly 1.3 ms and 27 ms respectively; this pairing is inferred from Table 5, whose layout was garbled in extraction.
- Limitations: the system has not yet been deployed to production, depends on annotated data, and was tested on English only.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Indian Institute of Technology Hyderabad](https://md.research.elamur.ai/free/en/universities/indian-institute-of-technology-hyderabad/) — University, Sangāreddi, India · 4 works · 108 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 624 works

## Cite

Cite as: Anshuman Mourya, Rajdeep Mukherjee, Prerna Jolly, Vinayak S Puranik and Sivaramakrishnan R Kaveri. 2026. DIRECT: Directional Relevance in Conversational Trajectories. Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{mourya2026direct,
  title = {{DIRECT: Directional Relevance in Conversational Trajectories}},
  author = {Anshuman Mourya, Rajdeep Mukherjee, Prerna Jolly, Vinayak S Puranik and Sivaramakrishnan R Kaveri},
  booktitle = {Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 5: Industry Track)},
  year = {2026},
  doi = {10.18653/v1/2026.eacl-industry.71},
  url = {https://doi.org/10.18653/v1/2026.eacl-industry.71}
}
```

- [← previous DiPS: Dialogue Policy Selection for High-Stakes Persuasion Agents](https://md.research.elamur.ai/free/en/papers/zhang-2026-dips-dialogue-policy-selection-high/)
- [next → Disagreement-Driven Joint Refinement of Retrieval and Decision Rules for Imbalanced Counseling Risk Classification](https://md.research.elamur.ai/free/en/papers/shao-2026-disagreement-driven-joint-refinement-retrieval/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/27/)

## References

15 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jianlyu Chen, Shitao Xiao, Peitian Zhang, Kun Luo, Defu Lian, and Zheng Liu. 2024. [M3-embedding: Multi-linguality, multi-functionality, multi-granularity text embeddings through self-knowledge distillation](https://md.research.elamur.ai/free/en/papers/chen-2024-m3-embedding-multi-linguality/). In Findings of the Association for Computational Linguistics: ACL 2024, pages 2318–2335, Bang…
2. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Yubin Ge, Ziang Xiao, Jana Diesner, Heng Ji, Karrie Karahalios, and Hari Sundaram. 2023. What should I ask: A knowledge-driven approach for follow-up questions generation in conversational surveys. In Proce… [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+Pre%2Dtraining+of+Yubin+Ge%2C+Ziang+Xiao%2C+Jana+Diesner%2C+Heng+Ji%2C+Karrie+Karahalios%2C+and+Hari+Sundaram%2E+2023%2E+What+should+I+ask)
3. Ehsan Kamalloo, Xinyu Zhang, Odunayo Ogundepo, Nandan Thakur, David Alfonso-Hermelo, Mehdi Rezagholizadeh, and Jimmy Lin. 2023. Evaluating embedding apis for information retrieval. Preprint, arXiv:2305.06300. [find](https://scholar.google.com/scholar?q=Ehsan+Kamalloo%2C+Xinyu+Zhang%2C+Odunayo+Ogundepo%2C+Nandan+Thakur%2C+David+Alfonso%2DHermelo%2C+Mehdi+Rezagholizadeh%2C+and+Jimmy+Lin%2E+2023%2E+Evaluating+embedding+apis+for+information+retrieval%2E+Preprint%2C+arXiv%3A230)
4. Chankyu Lee, Rajarshi Roy, Mengyao Xu, Jonathan Raiman, Mohammad Shoeybi, Bryan Catanzaro, and Wei Ping. 2025. [Nv-embed: Improved techniques for training llms as generalist embedding models](https://md.research.elamur.ai/free/en/papers/lee-2024-nv-embed-improved-techniques-training/). Preprint, arXiv:2405.17428.
5. Chaofan Li, MingHao Qin, Shitao Xiao, Jianlyu Chen, Kun Luo, Yingxia Shao, Defu Lian, and Zheng Liu. 2024a. Making text embedders few-shot learners. Preprint, arXiv:2409.15700. [find](https://scholar.google.com/scholar?q=Chaofan+Li%2C+MingHao+Qin%2C+Shitao+Xiao%2C+Jianlyu+Chen%2C+Kun+Luo%2C+Yingxia+Shao%2C+Defu+Lian%2C+and+Zheng+Liu%2E+2024a%2E+Making+text+embedders+few%2Dshot+learners%2E+Preprint%2C+arXiv%3A2409%2E15700%2E)
6. Haitao Li, Qian Dong, Junjie Chen, Huixue Su, Yujia Zhou, Qingyao Ai, Ziyi Ye, and Yiqun Liu. 2024b. [Llms-as-judges: A comprehensive survey on llm-based evaluation methods](https://md.research.elamur.ai/free/en/papers/li-2024-llms-judges-comprehensive-survey-llm/). Preprint, arXiv:2412.05579.
7. Jianyu Liu, Yi Huang, Sheng Bi, Junlan Feng, and Guilin Qi. 2025. From superficial to deep: Integrating external knowledge for follow-up question generation using knowledge graph and LLM. In Proceedings of the 31st International Conference on Computational Linguistics, pages 828–840, Abu Dhabi, UAE.… [find](https://scholar.google.com/scholar?q=Jianyu+Liu%2C+Yi+Huang%2C+Sheng+Bi%2C+Junlan+Feng%2C+and+Guilin+Qi%2E+2025%2E+From+superficial+to+deep%3A+Integrating+external+knowledge+for+follow%2Dup+question+generation+using+knowledge+graph+and+LLM%2E+In+Proceedin)
8. Yan Meng, Liangming Pan, Yixin Cao, and Min-Yen Kan. 2023. FollowupQG: Towards information-seeking follow-up question generation. In Proceedings of the 13th International Joint Conference on Natural Language Processing and the 3rd Conference of the Asia-Pacific Chapter of the Association for Computa… [find](https://scholar.google.com/scholar?q=Yan+Meng%2C+Liangming+Pan%2C+Yixin+Cao%2C+and+Min%2DYen+Kan%2E+2023%2E+FollowupQG%3A+Towards+information%2Dseeking+follow%2Dup+question+generation%2E+In+Proceedings+of+the+13th+International+Joint+Conference+on+Natural+L)
9. Luke Merrick, Danmei Xu, Gaurav Nuti, and Daniel Campos. 2024. Arctic-embed: Scalable, efficient, and accurate text embedding models. arXiv preprint arXiv:2405.05374. — [arXiv:2405.05374](https://arxiv.org/abs/2405.05374) [find](https://scholar.google.com/scholar?q=Luke+Merrick%2C+Danmei+Xu%2C+Gaurav+Nuti%2C+and+Daniel+Campos%2E+2024%2E+Arctic%2Dembed%3A+Scalable%2C+efficient%2C+and+accurate+text+embedding+models%2E+arXiv+preprint+arXiv%3A2405%2E05374%2E+%E2%80%94+%5BarXiv%3A2405%2E05374%5D%28https%3A%2F%2Farxi)
10. Jianmo Ni, Gustavo Hernandez Abrego, Noah Constant, Ji Ma, Keith Hall, Daniel Cer, and Yinfei Yang. 2022. [Sentence-t5: Scalable sentence encoders from pre-trained text-to-text models](https://md.research.elamur.ai/free/en/papers/ni-2021-sentence-t5-scalable-sentence-encoders/). In Findings of the Association for Computational Linguistics: ACL 2022, pages 1864–1874, Dublin, Ireland. Associatio…
11. Nils Reimers and Iryna Gurevych. 2019. [Sentence-BERT: Sentence embeddings using Siamese BERT-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992,…
12. Christopher Richardson, Sudipta Kar, Anjishnu Kumar, Anand Ramachandran, Zeynab Raeesy, Omar Khan, and Abhinav Sethy. 2023. Learning to retrieve engaging follow-up queries. In Findings of the Association for Computational Linguistics: EACL 2023, pages 2009–2016, Dubrovnik, Croatia. Association for C… [find](https://scholar.google.com/scholar?q=Christopher+Richardson%2C+Sudipta+Kar%2C+Anjishnu+Kumar%2C+Anand+Ramachandran%2C+Zeynab+Raeesy%2C+Omar+Khan%2C+and+Abhinav+Sethy%2E+2023%2E+Learning+to+retrieve+engaging+follow%2Dup+queries%2E+In+Findings+of+the+Associat)
13. Aivin V. Solatorio. 2024. Gistembed: Guided in-sample selection of training negatives for text embedding fine-tuning. Preprint, arXiv:2402.16829. — [arXiv:2402.16829](https://arxiv.org/abs/2402.16829) [find](https://scholar.google.com/scholar?q=Aivin+V%2E+Solatorio%2E+2024%2E+Gistembed%3A+Guided+in%2Dsample+selection+of+training+negatives+for+text+embedding+fine%2Dtuning%2E+Preprint%2C+arXiv%3A2402%2E16829%2E+%E2%80%94+%5BarXiv%3A2402%2E16829%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2402%2E16829%29)
14. Hongjin Su, Weijia Shi, Jungo Kasai, Yizhong Wang, Yushi Hu, Mari Ostendorf, Wen-tau Yih, Noah A. Smith, Luke Zettlemoyer, and Tao Yu. 2023. [One embedder, any task: Instruction-finetuned text embeddings](https://md.research.elamur.ai/free/en/papers/su-2022-one-embedder-any-task-instruction/). In Findings of the Association for Computational Linguistics: ACL 2023, pages 1102–1121, Toronto…
15. Zhichao Xu, Fengran Mo, Zhiqi Huang, Crystina Zhang, Puxuan Yu, Bei Wang, Jimmy Lin, and Vivek Srikumar. 2025. A survey of model architectures in information retrieval. Preprint, arXiv:2502.14822. — [arXiv:2502.14822](https://arxiv.org/abs/2502.14822) [find](https://scholar.google.com/scholar?q=Zhichao+Xu%2C+Fengran+Mo%2C+Zhiqi+Huang%2C+Crystina+Zhang%2C+Puxuan+Yu%2C+Bei+Wang%2C+Jimmy+Lin%2C+and+Vivek+Srikumar%2E+2025%2E+A+survey+of+model+architectures+in+information+retrieval%2E+Preprint%2C+arXiv%3A2502%2E14822%2E+%E2%80%94+%5B)

*Updated 1 November 2026*