---
title: "Augmenting Transformers with KNN-Based Composite Memory"
description: "Fan, Gardent, Braud and Bordes propose KNN-based Information Fetching (KIF) modules for… 45 references, cited by 30 papers here."
authors: ["Angela Fan", "Claire Gardent", "Chloé Braud", "Antoine Bordes"]
year: 2021
venue: "Transactions of the Association for Computational Linguistics"
source: "https://aclanthology.org/2021.tacl-1.6/"
doi: "10.1162/tacl_a_00356"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/fan-2021-augmenting-transformers-knn-based-composite/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/fan-2021-augmenting-transformers-knn-based-composite/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/5/) / Augmenting Transformers with KNN-Based Composite Memory for Dialog

# Augmenting Transformers with KNN-Based Composite Memory for Dialog

**Authors:** [Angela Fan](https://md.research.elamur.ai/free/en/authors/angela-fan/), [Claire Gardent](https://md.research.elamur.ai/free/en/authors/claire-gardent/), [Chloé Braud](https://md.research.elamur.ai/free/en/authors/chloe-braud/) and [Antoine Bordes](https://md.research.elamur.ai/free/en/authors/antoine-bordes/) · **Year:** 2021 · **Venue:** [Transactions of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/tacl/) ([Transactions of the Association for Computational Linguistics (TACL) 2021](https://md.research.elamur.ai/free/en/venues/tacl/2021/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.tacl-1.6/) · [PDF](https://aclanthology.org/2021.tacl-1.6.pdf) · [DOI 10.1162/tacl\_a\_00356](https://doi.org/10.1162/tacl_a_00356) · [OpenAlex](https://openalex.org/W3134891661)

**Citations:** 38 citations (OpenAlex, 2026-10-14) · [30](https://md.research.elamur.ai/free/en/papers/fan-2021-augmenting-transformers-knn-based-composite/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Dialog systems that generate their own replies often need outside information, such as encyclopedic facts or images, to stay on topic and say something substantive. Earlier memory-augmented networks mostly attend over every memory slot, which becomes expensive and less effective as memories grow. Fan, Gardent, Braud and Bordes propose KNN-based Information Fetching (KIF) modules for generative Transformers.

Each external source is encoded once and then frozen. The model learns only a mapping that turns the dialog context into a query. Nearest-neighbour search, implemented with faiss, uses that query to pick the most relevant items. The selected items are re-encoded, merged through a gated weighted sum, and concatenated with the encoder output.

KIF is evaluated on two benchmarks, Wizard of Wikipedia and Engaging ImageChat, using memories built from Wikipedia sentences, related images paired with their dialogs, and utterances from the training set. Among generative models, KIF reaches the best reported results. Human judges found no statistically significant gap in interestingness or preference against top retrieval systems. Because each retrieved item is readable text, the model's choices are also easy to inspect.

## Key points
- The memories are pre-computed and never updated by backpropagation. This lets KIF search hundreds of thousands of candidates. In contrast, the Generative Transformer MemNet baseline handles larger knowledge pools poorly.
- Combining sources helps on Wizard of Wikipedia. F1 is 18.1 with training utterances alone, 23.9 with Wikipedia knowledge alone, and 25.9 with both. The corresponding Engaging ImageChat scores are 13.9, 13.8 and 14.4.
- Richer search queries raise validation F1 on Wizard of Wikipedia. Scores go from 24.6 with the previous utterance only, to 26.4 with full dialog context, to 27.4 with turn embeddings. Engaging ImageChat gains in a similar way, rising from 13.3 to about 15.1.
- The gating mechanism can suppress an unrelated source. When PersonaChat turns are added as noise, gating recovers nearly the performance of a model without that source.
- Multi-hop retrieval brought no gain on Wizard of Wikipedia; 2–3 fetches scored 25.9–26.9 F1. Retrieving too many neighbours (large k) also hurts performance.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Meta](https://md.research.elamur.ai/free/en/companies/meta/) — Company, Menlo Park, United States · 183 works · 228,558 citations
- [Centre National de la Recherche Scientifique](https://md.research.elamur.ai/free/en/labs/centre-national-de-la-recherche-scientifique/) — Research institute, Paris, France · 168 works · 62,369 citations
- [Laboratoire Lorrain de Recherche en Informatique et ses Applications](https://md.research.elamur.ai/free/en/labs/laboratoire-lorrain-de-recherche-en-informatique-et-ses-applications/) — Research institute, Nancy, France · 30 works · 3,899 citations
- [Institut de Recherche en Informatique de Toulouse](https://md.research.elamur.ai/free/en/labs/institut-de-recherche-en-informatique-de-toulouse/) — Research institute, Toulouse, France · 35 works · 1,872 citations
- [Université de Lorraine](https://md.research.elamur.ai/free/en/universities/universite-de-lorraine/) — University, Nancy, France · 20 works · 945 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works

🧭 Topics (1): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 624 works

## Cite

Cite as: Angela Fan, Claire Gardent, Chloé Braud and Antoine Bordes. 2021. Augmenting Transformers with KNN-Based Composite Memory for Dialog. Transactions of the Association for Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{fan2021augmenting,
  title = {{Augmenting Transformers with KNN-Based Composite Memory for Dialog}},
  author = {Angela Fan, Claire Gardent, Chloé Braud and Antoine Bordes},
  journal = {Transactions of the Association for Computational Linguistics},
  year = {2021},
  doi = {10.1162/tacl\_a\_00356},
  url = {https://doi.org/10.1162/tacl_a_00356}
}
```

- [← previous It's All in the Heads: Using Attention Heads as a Baseline for Cross-Lingual Transfer in Commonsense Reasoning](https://md.research.elamur.ai/free/en/papers/tikhonov-2021-it-s-all-heads-attention/)
- [next → NADI 2021: The Second Nuanced Arabic Dialect Identification Shared Task](https://md.research.elamur.ai/free/en/papers/mageed-2021-nadi-2021-second-nuanced-arabic/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/5/)

## References

45 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 4177–4187. [find](https://scholar.google.com/scholar?q=in+Natural+Language+Processing+and+the+9th+International+Joint+Conference+on+Natural+Language+Processing+%28EMNLP%2DIJCNLP%29%2C+pages+4177%E2%80%934187%2E)
2. Antoine Bordes, Y-Lan Boureau, and Jason Weston. 2017. Learning end-to-end goal-oriented dialog. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. [find](https://scholar.google.com/scholar?q=Antoine+Bordes%2C+Y%2DLan+Boureau%2C+and+Jason+Weston%2E+2017%2E+Learning+end%2Dto%2Dend+goal%2Doriented+dialog%2E+In+5th+International+Conference+on+Learning+Representations%2C+ICLR+2017%2C+Toulon%2C+France%2C+April+24%2D26%2C+20)
3. Angela Fan, David Grangier, and Michael Auli. 2018a. [Controllable abstractive summarization](https://md.research.elamur.ai/free/en/papers/fan-2018-controllable-abstractive-summarization/). In Proceedings of the 2nd Workshop on Neural Machine Translation and Generation, pages 45–54.
4. Deng Cai, Yan Wang, Wei Bi, Zhaopeng Tu, Xiao-jiang Liu, and Shuming Shi. 2019. Retrieval-guided dialogue response generation via a matching-to-generation framework. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference o… [find](https://scholar.google.com/scholar?q=Deng+Cai%2C+Yan+Wang%2C+Wei+Bi%2C+Zhaopeng+Tu%2C+Xiao%2Djiang+Liu%2C+and+Shuming+Shi%2E+2019%2E+Retrieval%2Dguided+dialogue+response+generation+via+a+matching%2Dto%2Dgeneration+framework%2E+In+Proceedings+of+the+2019+Confere)
5. Angela Fan, Mike Lewis, and Yann Dauphin. 2018b. [Hierarchical neural story generation](https://md.research.elamur.ai/free/en/papers/fan-2018-hierarchical-neural-story-generation/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 889–898.
6. Edouard Grave, Armand Joulin, Moustapha Cissé, David Grangier, and Hervé Jégou. 2017a. [Efficient softmax approximation for GPUs](https://md.research.elamur.ai/free/en/papers/grave-2016-efficient-softmax-approximation-gpus/). In Proceedings of the 34th International Conference on Machine Learning-Volume 70, pages 1302–1310.
7. Sarath Chandar, Sungjin Ahn, Hugo Larochelle, Pascal Vincent, Gerald Tesauro, and Yoshua Bengio. 2016. Hierarchical memory networks. CoRR, abs/1605.07427. — [arXiv:1605.07427](https://arxiv.org/abs/1605.07427) [find](https://scholar.google.com/scholar?q=Sarath+Chandar%2C+Sungjin+Ahn%2C+Hugo+Larochelle%2C+Pascal+Vincent%2C+Gerald+Tesauro%2C+and+Yoshua+Bengio%2E+2016%2E+Hierarchical+memory+networks%2E+CoRR%2C+abs%2F1605%2E07427%2E+%E2%80%94+%5BarXiv%3A1605%2E07427%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F16)
8. Edouard Grave, Armand Joulin, and Nicolas Usunier. 2017b. Improving neural language models with a continuous cache. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. [find](https://scholar.google.com/scholar?q=Edouard+Grave%2C+Armand+Joulin%2C+and+Nicolas+Usunier%2E+2017b%2E+Improving+neural+language+models+with+a+continuous+cache%2E+In+5th+International+Conference+on+Learning+Representations%2C+ICLR+2017%2C+Toulon%2C+Fran)
9. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. [Reading Wikipedia to answer open-domain questions](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1870–1879. DOI: https://doi.org/10.18653/v1/P17-1171, PMCID: P… [↗](https://doi.org/10.18653/v1/P17-1171)
10. Alex Graves, Greg Wayne, and Ivo Danihelka. 2014. Neural Turing machines. arXiv preprint arXiv:1410.5401. [find](https://scholar.google.com/scholar?q=Alex+Graves%2C+Greg+Wayne%2C+and+Ivo+Danihelka%2E+2014%2E+Neural+Turing+machines%2E+arXiv+preprint+arXiv%3A1410%2E5401%2E)
11. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. [Retrieval augmented language model pre-training](https://md.research.elamur.ai/free/en/papers/guu-2020-realm-retrieval-augmented-language-model/). In Proceedings of the International Conference on Machine Learning, pages 5695–5704.
12. Wenlin Chen, David Grangier, and Michael Auli. 2016. Strategies for training large vocabulary neural language models. In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1975–1985. DOI: https://doi.org/10.18653/v1/P16 -1186 [↗](https://doi.org/10.18653/v1/P16)
13. Samuel Humeau, Kurt Shuster, Marie-Anne Lachaux, and Jason Weston. 2019. Poly-encoders: Architectures and pre-training strategies for fast and accurate multi-sentence scoring. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Samuel+Humeau%2C+Kurt+Shuster%2C+Marie%2DAnne+Lachaux%2C+and+Jason+Weston%2E+2019%2E+Poly%2Dencoders%3A+Architectures+and+pre%2Dtraining+strategies+for+fast+and+accurate+multi%2Dsentence+scoring%2E+In+International+Confere)
14. Emily Dinan, Stephen Roller, Kurt Shuster, Angela Fan, Michael Auli, and Jason Weston. 2018. [Wizard of Wikipedia: Knowledge-powered conversational agents](https://md.research.elamur.ai/free/en/papers/dinan-2018-wizard-wikipedia-knowledge-powered-conversational/). In International Conference on Learning Representations.
15. Jeff Johnson, Matthijs Douze, and Hervé Jégou. 2019. [Billion-scale similarity search with GPUs](https://md.research.elamur.ai/free/en/papers/johnson-2017-billion-scale-similarity-search-gpus/). IEEE Transactions on Big Data. DOI: https://doi.org/10.1109/TBDATA .2019.2921572 [↗](https://doi.org/10.1109/TBDATA)
16. Angela Fan, Claire Gardent, Chloé Braud, and Antoine Bordes. 2019. [Using local knowledge graph construction to scale seq2seq models to multi-document inputs](https://md.research.elamur.ai/free/en/papers/miller-2017-parlai-dialog-research-software-platform/). In Proceedings of the 2019 Conference on Empirical Methods Armand Joulin and Tomas Mikolov. 2015. Inferring algorithmic patterns with stack-a…
17. Urvashi Khandelwal, Omer Levy, Dan Jurafsky, Luke Zettlemoyer, and Mike Lewis. 2019. Generalization through memorization: Nearest neighbor language models. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Urvashi+Khandelwal%2C+Omer+Levy%2C+Dan+Jurafsky%2C+Luke+Zettlemoyer%2C+and+Mike+Lewis%2E+2019%2E+Generalization+through+memorization%3A+Nearest+neighbor+language+models%2E+In+International+Conference+on+Learning+Repr)
18. Andriy Mnih and Geoffrey Hinton. 2009. A scalable hierarchical distributed language model. In Advances in Neural Information Processing Systems, pages 1081–1088. [find](https://scholar.google.com/scholar?q=Andriy+Mnih+and+Geoffrey+Hinton%2E+2009%2E+A+scalable+hierarchical+distributed+language+model%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+1081%E2%80%931088%2E)
19. Diederik P. Kingma and Jimmy Ba. [Adam: A Method for Stochastic Optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings.
20. Fabio Petroni, Tim Rocktäschel, Sebastian Riedel, Patrick Lewis, Anton Bakhtin, Yuxiang Wu, and Alexander Miller. 2019. [Language models as knowledge bases?](https://md.research.elamur.ai/free/en/papers/petroni-2019-language-models-as-knowledge-bases/) In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural…
21. Guillaume Lample, Alexandre Sablayrolles, Marc’Aurelio Ranzato, Ludovic Denoyer, and Hervé Jégou. 2019. Large memory layers with product keys. In Advances in Neural Information Processing Systems, pages 8548–8559. [find](https://scholar.google.com/scholar?q=Guillaume+Lample%2C+Alexandre+Sablayrolles%2C+Marc%E2%80%99Aurelio+Ranzato%2C+Ludovic+Denoyer%2C+and+Herve%CC%81+Je%CC%81gou%2E+2019%2E+Large+memory+layers+with+product+keys%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+p)
22. Tobias Plötz and Stefan Roth. 2018. Neural nearest neighbors networks. In Advances in Neural Information Processing Systems, pages 1087–1098. [find](https://scholar.google.com/scholar?q=Tobias+Plo%CC%88tz+and+Stefan+Roth%2E+2018%2E+Neural+nearest+neighbors+networks%2E+In+Advances+in+Neural+Information+Processing+Systems%2C+pages+1087%E2%80%931098%2E)
23. Margaret Li, Jason Weston, and Stephen Roller. 2019. ACUTE-EVAL: Improved dialogue evaluation with optimized questions and multi-turn comparisons. arXiv preprint arXiv:1909.03087. — [arXiv:1909.03087](https://arxiv.org/abs/1909.03087) [find](https://scholar.google.com/scholar?q=Margaret+Li%2C+Jason+Weston%2C+and+Stephen+Roller%2E+2019%2E+ACUTE%2DEVAL%3A+Improved+dialogue+evaluation+with+optimized+questions+and+multi%2Dturn+comparisons%2E+arXiv+preprint+arXiv%3A1909%2E03087%2E+%E2%80%94+%5BarXiv%3A1909%2E03087%5D)
24. Lianhui Qin, Michel Galley, Chris Brockett, Xiaodong Liu, Xiang Gao, Bill Dolan, Yejin Choi, and Jianfeng Gao. 2019. Conversing by reading: Contentful neural conversation with on-demand machine reading. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages… [find](https://scholar.google.com/scholar?q=Lianhui+Qin%2C+Michel+Galley%2C+Chris+Brockett%2C+Xiaodong+Liu%2C+Xiang+Gao%2C+Bill+Dolan%2C+Yejin+Choi%2C+and+Jianfeng+Gao%2E+2019%2E+Conversing+by+reading%3A+Contentful+neural+conversation+with+on%2Ddemand+machine+readin)
25. Rongzhong Lian, Min Xie, Fan Wang, Jinhua Peng, and Hua Wu. 2019. Learning to select knowledge for response generation in dialog systems. In Proceedings of the 28th International Joint Conference on Artificial Intelligence, pages 5081–5087. AAAI Press. [find](https://scholar.google.com/scholar?q=Rongzhong+Lian%2C+Min+Xie%2C+Fan+Wang%2C+Jinhua+Peng%2C+and+Hua+Wu%2E+2019%2E+Learning+to+select+knowledge+for+response+generation+in+dialog+systems%2E+In+Proceedings+of+the+28th+International+Joint+Conference+on+A)
26. Jack Rae, Jonathan J. Hunt, Ivo Danihelka, Timothy Harley, Andrew W. Senior, Gregory Wayne, Alex Graves, and Timothy Lillicrap. 2016. Scaling memory-augmented neural networks with sparse reads and writes. In Advances in Neural Information Processing Systems, pages 3621–3629. [find](https://scholar.google.com/scholar?q=Jack+Rae%2C+Jonathan+J%2E+Hunt%2C+Ivo+Danihelka%2C+Timothy+Harley%2C+Andrew+W%2E+Senior%2C+Gregory+Wayne%2C+Alex+Graves%2C+and+Timothy+Lillicrap%2E+2016%2E+Scaling+memory%2Daugmented+neural+networks+with+sparse+reads+and+wri)
27. Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, and Laurens van der Maaten. 2018. Exploring the limits of weakly supervised pretraining. In Proceedings of the European Conference on Computer Vision (ECCV), pages 181–196. DOI: https://doi.org/… [find](https://scholar.google.com/scholar?q=Dhruv+Mahajan%2C+Ross+Girshick%2C+Vignesh+Ramanathan%2C+Kaiming+He%2C+Manohar+Paluri%2C+Yixuan+Li%2C+Ashwin+Bharambe%2C+and+Laurens+van+der+Maaten%2E+2018%2E+Exploring+the+limits+of+weakly+supervised+pretraining%2E+In+Pr)
28. Bryan McCann, James Bradbury, Caiming Xiong, and Richard Socher. 2017. [Learned in translation: Contextualized word vectors](https://md.research.elamur.ai/free/en/papers/mccann-2017-learned-translation-contextualized-word-vectors/). In Advances in Neural Information Processing Systems, pages 6294–6305.
29. Rico Sennrich, Barry Haddow, and Alexandra Birch. 2016. [Neural machine translation of rare words with subword units](https://md.research.elamur.ai/free/en/papers/sennrich-2016-neural-machine-translation-rare-words/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1715–1725. DOI: https://doi.org/10.18653/v1/P16 -1162 [↗](https://doi.org/10.18653/v1/P16)
30. Alexander Miller, Will Feng, Dhruv Batra, Antoine Bordes, Adam Fisch, Jiasen Lu, Devi Parikh, and Jason Weston. 2017. [parl.ai: A Minjoon Seo, Jinhyuk Lee, Tom Kwiatkowski, Ankur Parikh, Ali Farhadi, and Hannaneh Hajishirzi](https://md.research.elamur.ai/free/en/papers/seo-2019-real-time-open-domain-question/). 2019. Real-time open-domain question answering with dense-sparse phrase inde…
31. Advances in Neural Information Processing Systems, pages 2440–2448. [find](https://scholar.google.com/scholar?q=Advances+in+Neural+Information+Processing+Systems%2C+pages+2440%E2%80%932448%2E)
32. Bart Thomee, David A Shamma, Gerald Friedland, Benjamin Elizalde, Karl Ni, Douglas Poland, Damian Borth, and Li-Jia Li. 2016. YFCC100M: The new data in multimedia research. Communications of the ACM, 59(2):64–73. DOI: https://doi.org/10 .1145/2812802 [find](https://scholar.google.com/scholar?q=Bart+Thomee%2C+David+A+Shamma%2C+Gerald+Friedland%2C+Benjamin+Elizalde%2C+Karl+Ni%2C+Douglas+Poland%2C+Damian+Borth%2C+and+Li%2DJia+Li%2E+2016%2E+YFCC100M%3A+The+new+data+in+multimedia+research%2E+Communications+of+the+ACM%2C+)
33. Iulian V. Serban, Ryan Lowe, Laurent Charlin, and Joelle Pineau. 2016a. Generative deep neural networks for dialogue: A short review. arXiv preprint arXiv:1611.06216. — [arXiv:1611.06216](https://arxiv.org/abs/1611.06216) [find](https://scholar.google.com/scholar?q=Iulian+V%2E+Serban%2C+Ryan+Lowe%2C+Laurent+Charlin%2C+and+Joelle+Pineau%2E+2016a%2E+Generative+deep+neural+networks+for+dialogue%3A+A+short+review%2E+arXiv+preprint+arXiv%3A1611%2E06216%2E+%E2%80%94+%5BarXiv%3A1611%2E06216%5D%28https%3A%2F%2Farxi)
34. Iulian V. Serban, Alessandro Sordoni, Yoshua Bengio, Aaron Courville, and Joelle Pineau. 2016b. [Building end-to-end dialogue systems using generative hierarchical neural network models](https://md.research.elamur.ai/free/en/papers/serban-2015-building-end-end-dialogue-systems/). In Thirtieth AAAI Conference on Artificial Intelligence.
35. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in Neural Information Processing Systems, pages 5998–6008.
36. Kurt Shuster, Samuel Humeau, Antoine Bordes, and Jason Weston. 2020. Image-chat: Engaging grounded conversations. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2414–2429. DOI: https://doi.org/10.18653/v1/2020.acl-main.219 [↗](https://doi.org/10.18653/v1/2020.acl-main.219)
37. Jason Weston, Sumit Chopra, and Antoine Bordes. 2015. Memory networks. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. [find](https://scholar.google.com/scholar?q=Jason+Weston%2C+Sumit+Chopra%2C+and+Antoine+Bordes%2E+2015%2E+Memory+networks%2E+In+3rd+International+Conference+on+Learning+Representations%2C+ICLR+2015%2C+San+Diego%2C+CA%2C+USA%2C+May+7%2D9%2C+2015%2C+Conference+Track+Proce)
38. Jason Weston, Emily Dinan, and Alexander Miller. 2018. [Retrieve and refine: Improved sequence generation models for dialogue](https://md.research.elamur.ai/free/en/papers/weston-2018-retrieve-refine-improved-sequence-generation/). In Proceedings of the 2018 EMNLP Workshop SCAI: The 2nd International Workshop on Search-Oriented Conversational AI, pages 87–92. DOI: https://doi.org/10 .18653/v1/W18-5713
39. Kurt Shuster, Samuel Humeau, Hexiang Hu, Antoine Bordes, and Jason Weston. 2019. Engaging image captioning via personality. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 12516–12526. DOI: https://doi.org/10.1109/CVPR.2019.01280 — [doi:10.1109/cvpr.2019.01280](https://doi.org/10.1109/cvpr.2019.01280) [↗](https://doi.org/10.1109/CVPR.2019.01280)
40. Saining Xie, Ross Girshick, Piotr Dollár, Zhuowen Tu, and Kaiming He. 2017. Aggregated residual transformations for deep neural networks. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 1492–1500. [find](https://scholar.google.com/scholar?q=Saining+Xie%2C+Ross+Girshick%2C+Piotr+Dolla%CC%81r%2C+Zhuowen+Tu%2C+and+Kaiming+He%2E+2017%2E+Aggregated+residual+transformations+for+deep+neural+networks%2E+In+Proceedings+of+the+IEEE+conference+on+computer+vision+and+)
41. Haoyu Song, Yan Wang, Wei-Nan Zhang, Xiaojiang Liu, and Ting Liu. 2020. Generate, delete and rewrite: A three-stage framework for improving persona consistency of dialogue generation. arXiv preprint arXiv:2004.07672. DOI: https://doi.org/10.18653/v1/2020.acl-main.516, PMID: 32249355 [↗](https://doi.org/10.18653/v1/2020.acl-main.516)
42. Saizheng Zhang, Emily Dinan, Jack Urbanek, Arthur Szlam, Douwe Kiela, and Jason Weston. 2018. [Personalizing dialogue agents: I have a dog, do you have pets too?](https://md.research.elamur.ai/free/en/papers/zhang-2018-personalizing-dialogue-agents-i-have/) In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2204–2213. DOI: h…
43. Yiping Song, Rui Yan, Xiang Li, Dongyan Zhao, and Ming Zhang. 2016. Two are better than one: An ensemble of retrieval-and generation-based dialog systems. arXiv preprint arXiv:1610.07149. — [arXiv:1610.07149](https://arxiv.org/abs/1610.07149) [find](https://scholar.google.com/scholar?q=Yiping+Song%2C+Rui+Yan%2C+Xiang+Li%2C+Dongyan+Zhao%2C+and+Ming+Zhang%2E+2016%2E+Two+are+better+than+one%3A+An+ensemble+of+retrieval%2Dand+generation%2Dbased+dialog+systems%2E+arXiv+preprint+arXiv%3A1610%2E07149%2E+%E2%80%94+%5BarXiv%3A161)
44. Sainbayar Sukhbaatar, Edouard Grave, Guillaume Lample, Herve Jegou, and Armand Joulin. 2019. Augmenting self-attention with persistent memory. https://arxiv.org/abs/1907 .01470 [↗](https://arxiv.org/abs/1907)
45. Yutao Zhu, Zhicheng Dou, Jian-Yun Nie, and Ji-Rong Wen. 2020. ReBoost: A retrieval-boosted sequence-to-sequence model for neural response generation. Information Retrieval Journal, 23(1):27–48. DOI: https://doi.org/10.1007/s10791-019-09364-x Sainbayar Sukhbaatar, Jason Weston, Rob Fergus, et al. 20… — [doi:10.1007/s10791-019-09364-x](https://doi.org/10.1007/s10791-019-09364-x) [↗](https://doi.org/10.1007/s10791-019-09364-x)

*Updated 29 October 2026*