--- title: "Retrieve and Refine — Weston 2018" description: "Neural generators for open-domain chat often give bland, short replies built from common words." authors: ["Jason Weston", "Emily Dinan", "Alexander Miller"] year: 2018 venue: "Proceedings of the 2018 EMNLP Workshop SCAI: The 2nd International Workshop on Search-Oriented Conversational AI" source: "https://aclanthology.org/W18-5713/" doi: "10.18653/v1/w18-5713" arxiv: "1808.04776" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/weston-2018-retrieve-refine-improved-sequence-generation/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/weston-2018-retrieve-refine-improved-sequence-generation/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/) / Retrieve and Refine: Improved Sequence Generation Models For Dialogue # Retrieve and Refine: Improved Sequence Generation Models For Dialogue **Authors:** [Jason Weston](https://md.research.elamur.ai/free/en/authors/jason-weston/), [Emily Dinan](https://md.research.elamur.ai/free/en/authors/emily-dinan/), [Alexander Miller](https://md.research.elamur.ai/free/en/authors/alexander-miller/) · **Year:** 2018 · **Venue:** Proceedings of the 2018 EMNLP Workshop SCAI: The 2nd International Workshop on Search-Oriented Conversational AI · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/W18-5713/) · [PDF](https://aclanthology.org/W18-5713.pdf) · [DOI 10.18653/v1/w18-5713](https://doi.org/10.18653/v1/w18-5713) · [arXiv 1808.04776](https://arxiv.org/abs/1808.04776) · [OpenAlex](https://openalex.org/W2885421725) **Citations:** 164 citations (openalex-snapshot · 2026-10-14) · [90](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary Neural generators for open-domain chat often give bland, short replies built from common words. Retrieval systems can return livelier responses, but they can only choose from a fixed pool, so their answers may not fit the context. The authors combine the two in RetrieveNRefine (RetNRef). A Key-Value Memory Network retriever picks a candidate utterance, which is appended to the dialogue history. A 2-layer attentive LSTM Seq2Seq model then reads this input and produces the final reply. Two heuristics strengthen the approach. RetNRef+ drops the persona sentences from the generator input, which pushes the generator to rely more on the retrieved candidate. RetNRef++ adds a rule that outputs the retrieved text verbatim whenever the generated reply largely overlaps with it. On the ConvAI2 persona-chat task, perplexity gives little useful signal. Human judges, however, rated RetNRef++ as more engaging than both the Seq2Seq and Memory Network baselines. Its outputs also resembled human utterances more closely in length and rare-word use. The work shows that retrieval can help generative dialogue models produce less generic replies. ## Key points - **Data:** ConvAI2, a modified and larger version of PersonaChat, with about 160,000 utterances in about 11,000 dialogues. Personas come from a pool of 1155. Implementation uses ParlAI. - **Training:** For each training turn, the top 100 retrieved candidates are reranked by embedding similarity to the gold reply. The copy rule in RetNRef++ triggers at more than 60% word overlap. - **Perplexity results:** - No retrieval: 31.4. - Memory Network retrieval: 31.8. - Random retrieval: 32.0. - Gold reply's nearest neighbour as the retrieval: 25.9. - Gold reply itself: 9.2. - The history-truncating variant: 48.4, despite its better human scores. - **Human evaluation (600 dialogues):** - Engagingness was 3.80 for RetNRef++, 3.66 for the Memory Network and 2.70 for Seq2Seq. - In A/B tests, RetNRef++ won about 54% of decisive comparisons against each baseline (ties excluded), which was statistically significant. - RetNRef++ was weaker at persona use, scoring 0.65 vs 0.90 for Seq2Seq. - **Output statistics:** - Average reply length was 12.7 words for RetNRef++, 11.7 for Seq2Seq and 13.0 for humans. - Words with frequency under 1k made up 10.9% of RetNRef++ output, against 11.5% for humans and 5.8% for Seq2Seq. - About 53% of RetNRef++ outputs overlap the retrieved text by more than 80%, so it mixes copying with novel generation. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Cited by Cited by 90 papers of our library cite it (of the 140884 whose bibliographies we read); 9 of them have a page here, 81 not yet. - [Controllable Paraphrase Generation with a Syntactic Exemplar](https://md.research.elamur.ai/free/en/papers/chen-2019-controllable-paraphrase-generation-syntactic-exemplar/) - [The Second Conversational Intelligence Challenge (ConvAI2)](https://md.research.elamur.ai/free/en/papers/dinan-2019-second-conversational-intelligence-challenge-convai2/) - [ConveRT: Efficient and Accurate Conversational Representations from Transformers](https://md.research.elamur.ai/free/en/papers/henderson-2020-convert-efficient-accurate-conversational-representations/) - [MoEL: Mixture of Empathetic Listeners](https://md.research.elamur.ai/free/en/papers/lin-2019-moel-mixture-empathetic-listeners/) - [What Makes Good In-Context Examples for GPT-3?](https://md.research.elamur.ai/free/en/papers/liu-2022-what-makes-good-context-examples/) - [Justifying Recommendations using Distantly-Labeled Reviews and Fine-Grained Aspects](https://md.research.elamur.ai/free/en/papers/ni-2019-justifying-recommendations-distantly-labeled-reviews/) - [Recipes for Building an Open-Domain Chatbot](https://md.research.elamur.ai/free/en/papers/roller-2021-recipes-building-open-domain-chatbot/) - [“I'm sorry to hear that”: Finding New Biases in Language Models with a Holistic Descriptor Dataset](https://md.research.elamur.ai/free/en/papers/smith-2022-i-m-sorry-hear-that/) - [Neural Text Generation with Unlikelihood Training](https://md.research.elamur.ai/free/en/papers/welleck-2019-neural-text-generation-unlikelihood-training/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{weston2018retrieve, title = {{Retrieve and Refine: Improved Sequence Generation Models For Dialogue}}, author = {Weston, Jason and Dinan, Emily and Miller, Alexander}, booktitle = {Proceedings of the 2018 EMNLP Workshop SCAI: The 2nd International Workshop on Search-Oriented Conversational AI}, year = {2018}, doi = {10.18653/v1/w18-5713}, eprint = {1808.04776}, archivePrefix = {arXiv}, url = {https://doi.org/10.18653/v1/w18-5713} } ``` - [← previous Retrieval of the Best Counterargument without Prior Topic Knowledge](https://md.research.elamur.ai/free/en/papers/wachsmuth-2018-retrieval-best-counterargument-without-prior/) - [next → Rhotic Phonemes in Semitic Languages – The Case of Classical Arabic and Biblical Hebrew](https://md.research.elamur.ai/free/en/papers/hbo-2018-pietrzak-rhotic-phonemes-semitic-languages-case/) - [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/3/) ## References 13 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Ziqiang Cao, Wenjie Li, Sujian Li, and Furu Wei. 2018. Retrieve, rerank and rewrite: Soft template based neural summarization. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), volume 1, pages 152–161. [find](https://scholar.google.com/scholar?q=Ziqiang+Cao%2C+Wenjie+Li%2C+Sujian+Li%2C+and+Furu+Wei%2E+2018%2E+Retrieve%2C+rerank+and+rewrite%3A+Soft+template+based+neural+summarization%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computat) 2. Iulian Vlad Serban, Ryan Lowe, Laurent Charlin, and Joelle Pineau. 2016. Generative deep neural networks for dialogue: A short review. arXiv preprint arXiv:1611.06216. — [arXiv:1611.06216](https://arxiv.org/abs/1611.06216) [find](https://scholar.google.com/scholar?q=Iulian+Vlad+Serban%2C+Ryan+Lowe%2C+Laurent+Charlin%2C+and+Joelle+Pineau%2E+2016%2E+Generative+deep+neural+networks+for+dialogue%3A+A+short+review%2E+arXiv+preprint+arXiv%3A1611%2E06216%2E+%E2%80%94+%5BarXiv%3A1611%2E06216%5D%28https%3A%2F%2Farx) 3. Ilya Sutskever, Oriol Vinyals, and Quoc V Le. 2014. Sequence to sequence learning with neural networks. In Advances in neural information processing systems, pages 3104–3112. [find](https://scholar.google.com/scholar?q=Ilya+Sutskever%2C+Oriol+Vinyals%2C+and+Quoc+V+Le%2E+2014%2E+Sequence+to+sequence+learning+with+neural+networks%2E+In+Advances+in+neural+information+processing+systems%2C+pages+3104%E2%80%933112%2E) 4. Angela Fan, David Grangier, and Michael Auli. 2017. Controllable abstractive summarization. arXiv preprint arXiv:1711.05217. [find](https://scholar.google.com/scholar?q=Angela+Fan%2C+David+Grangier%2C+and+Michael+Auli%2E+2017%2E+Controllable+abstractive+summarization%2E+arXiv+preprint+arXiv%3A1711%2E05217%2E) [our card](https://md.research.elamur.ai/free/en/papers/fan-2018-controllable-abstractive-summarization/) 5. David Grangier and Michael Auli. 2017. Quickedit: Editing text & translations via simple delete actions. arXiv preprint arXiv:1711.04805. — [arXiv:1711.04805](https://arxiv.org/abs/1711.04805) [find](https://scholar.google.com/scholar?q=David+Grangier+and+Michael+Auli%2E+2017%2E+Quickedit%3A+Editing+text+%26+translations+via+simple+delete+actions%2E+arXiv+preprint+arXiv%3A1711%2E04805%2E+%E2%80%94+%5BarXiv%3A1711%2E04805%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1711%2E04805%29) 6. Yingce Xia, Fei Tian, Lijun Wu, Jianxin Lin, Tao Qin, Nenghai Yu, and Tie-Yan Liu. 2017. Deliberation networks: Sequence generation beyond one-pass decoding. In Advances in Neural Information Processing Systems, pages 1782–1792. [find](https://scholar.google.com/scholar?q=Yingce+Xia%2C+Fei+Tian%2C+Lijun+Wu%2C+Jianxin+Lin%2C+Tao+Qin%2C+Nenghai+Yu%2C+and+Tie%2DYan+Liu%2E+2017%2E+Deliberation+networks%3A+Sequence+generation+beyond+one%2Dpass+decoding%2E+In+Advances+in+Neural+Information+Processi) 7. Jiatao Gu, Yong Wang, Kyunghyun Cho, and Victor OK Li. 2017. Search engine guided non-parametric neural machine translation. arXiv preprint arXiv:1705.07267. — [arXiv:1705.07267](https://arxiv.org/abs/1705.07267) [find](https://scholar.google.com/scholar?q=Jiatao+Gu%2C+Yong+Wang%2C+Kyunghyun+Cho%2C+and+Victor+OK+Li%2E+2017%2E+Search+engine+guided+non%2Dparametric+neural+machine+translation%2E+arXiv+preprint+arXiv%3A1705%2E07267%2E+%E2%80%94+%5BarXiv%3A1705%2E07267%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs) 8. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. Personalizing dialogue agents: I have a dog, do you have pets too? arXiv preprint arXiv:1801.07243. [find](https://scholar.google.com/scholar?q=Saizheng+Zhang%2C+Emily+Dinan%2C+Jack+Urbanek%2C+Arthur+Szlam%2C+Douwe+Kiela%2C+and+Jason+Weston%2E+2018%2E+Personalizing+dialogue+agents%3A+I+have+a+dog%2C+do+you+have+pets+too%3F+arXiv+preprint+arXiv%3A1801%2E07243%2E) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2018-personalizing-dialogue-agents-i-have/) 9. Kelvin Guu, Tatsunori B Hashimoto, Yonatan Oren, and Percy Liang. 2017. Generating sentences by editing prototypes. arXiv preprint arXiv:1709.08878. [find](https://scholar.google.com/scholar?q=Kelvin+Guu%2C+Tatsunori+B+Hashimoto%2C+Yonatan+Oren%2C+and+Percy+Liang%2E+2017%2E+Generating+sentences+by+editing+prototypes%2E+arXiv+preprint+arXiv%3A1709%2E08878%2E) [our card](https://md.research.elamur.ai/free/en/papers/guu-2017-generating-sentences-editing-prototypes/) 10. Zongcheng Ji, Zhengdong Lu, and Hang Li. 2014. An information retrieval approach to short text conversation. arXiv preprint arXiv:1408.6988. — [arXiv:1408.6988](https://arxiv.org/abs/1408.6988) [find](https://scholar.google.com/scholar?q=Zongcheng+Ji%2C+Zhengdong+Lu%2C+and+Hang+Li%2E+2014%2E+An+information+retrieval+approach+to+short+text+conversation%2E+arXiv+preprint+arXiv%3A1408%2E6988%2E+%E2%80%94+%5BarXiv%3A1408%2E6988%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1408%2E6988%29) 11. Marcin Junczys-Dowmunt and Roman Grundkiewicz. 2017. An exploration of neural sequence-to-sequence architectures for automatic post-editing. arXiv preprint arXiv:1706.04138. [find](https://scholar.google.com/scholar?q=Marcin+Junczys%2DDowmunt+and+Roman+Grundkiewicz%2E+2017%2E+An+exploration+of+neural+sequence%2Dto%2Dsequence+architectures+for+automatic+post%2Dediting%2E+arXiv+preprint+arXiv%3A1706%2E04138%2E) 12. Chia-Wei Liu, Ryan Lowe, Iulian V Serban, Michael Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How not to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. arXiv preprint arXiv:1603.08023. [find](https://scholar.google.com/scholar?q=Chia%2DWei+Liu%2C+Ryan+Lowe%2C+Iulian+V+Serban%2C+Michael+Noseworthy%2C+Laurent+Charlin%2C+and+Joelle+Pineau%2E+2016%2E+How+not+to+evaluate+your+dialogue+system%3A+An+empirical+study+of+unsupervised+evaluation+metrics+) 13. Alexander Miller, Adam Fisch, Jesse Dodge, Amir-Hossein Karimi, Antoine Bordes, and Jason Weston. 2016. Key-value memory networks for directly reading documents. arXiv preprint arXiv:1606.03126. [find](https://scholar.google.com/scholar?q=Alexander+Miller%2C+Adam+Fisch%2C+Jesse+Dodge%2C+Amir%2DHossein+Karimi%2C+Antoine+Bordes%2C+and+Jason+Weston%2E+2016%2E+Key%2Dvalue+memory+networks+for+directly+reading+documents%2E+arXiv+preprint+arXiv%3A1606%2E03126%2E)