--- title: "PLATO-2 — paper by Bao et al. (2021) — summary and ideas" description: "PLATO-2 addresses the difficulty of scaling an open-domain chatbot that explicitly models the… 6 references, cited by 51 papers here." authors: ["Siqi Bao", "Huang He", "Fan Wang", "Hua Wu", "Haifeng Wang", "Wenquan Wu", "Zhen Guo", "Zhibin Liu", "Xinchao Xu"] year: 2021 venue: "Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021" source: "https://aclanthology.org/2021.findings-acl.222/" doi: "10.18653/v1/2021.findings-acl.222" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/bao-2021-plato-2-building-open-domain/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bao-2021-plato-2-building-open-domain/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/3/) / PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning # PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning **Authors:** [Siqi Bao](https://md.research.elamur.ai/free/en/authors/siqi-bao/), [Huang He](https://md.research.elamur.ai/free/en/authors/huang-he/), [Fan Wang](https://md.research.elamur.ai/free/en/authors/fan-wang/), [Hua Wu](https://md.research.elamur.ai/free/en/authors/hua-wu/), [Haifeng Wang](https://md.research.elamur.ai/free/en/authors/haifeng-wang/), [Wenquan Wu](https://md.research.elamur.ai/free/en/authors/wenquan-wu/), [Zhen Guo](https://md.research.elamur.ai/free/en/authors/zhen-guo/), Zhibin Liu and Xinchao Xu · **Year:** 2021 · **Venue:** [Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2021](https://md.research.elamur.ai/free/en/venues/findings/2021/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2021.findings-acl.222/) · [PDF](https://aclanthology.org/2021.findings-acl.222.pdf) · [DOI 10.18653/v1/2021.findings-acl.222](https://doi.org/10.18653/v1/2021.findings-acl.222) · [OpenAlex](https://openalex.org/W3174696767) **Citations:** 94 citations (OpenAlex, 2026-10-14) · [51](https://md.research.elamur.ai/free/en/papers/bao-2021-plato-2-building-open-domain/cited-by/) in our library, of the 141,840 papers whose bibliographies we read ## Summary PLATO-2 addresses the difficulty of scaling an open-domain chatbot that explicitly models the one-to-many nature of dialogue, where a single context can have many valid replies. Enlarging the original PLATO directly led to unstable and inefficient training. The authors therefore split learning into a two-stage curriculum. First, a coarse model is trained on a simpler one-to-one mapping to learn general response generation. Second, this model warm-starts two separate networks. One is a fine-grained generator that uses discrete latent variables to produce varied candidate replies. The other is an evaluation model that scores how well each candidate fits the context and picks the best one. All models use a unified transformer with pre-normalization and flexible attention instead of a separate encoder and decoder. English and Chinese versions were trained, the Chinese ones on large social media conversation data. In human evaluations, PLATO-2 beat Meena, Blender and other strong systems. The authors argue that curriculum learning makes billion-parameter latent-variable dialogue models trainable and yields a reusable pre-training framework for several kinds of conversational tasks. ## Key points - Model sizes trained: 1.6B, 314M and 93M parameters. The predecessor PLATO had 132M parameters and was trained on 8M samples. - Stage 2 generator details: - It uses a K-way categorical latent "speech act" with Gumbel-Softmax approximation. - It adds a bag-of-words (BOW) loss on top of NLL. - The evaluation model is trained with response coherence estimation (RCE) plus MLM (15% masking). - RCE uses randomly sampled responses as negatives. - It replaces likelihood- and MMI-based ranking, which tend to favor dull or repetitive replies. - PLATO-2 took first place in three DSTC9 tasks: - Track3-task2: interactive open-domain evaluation. - Track3-task1: knowledge grounded dialogue, using the stage 2 models. - Track2-task1: end-to-end task-oriented dialogue, using the stage 1 model. - English models and code are released via the PaddlePaddle Knover GitHub repository. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Baidu](https://md.research.elamur.ai/free/en/companies/baidu/) — Company, Beijing, China · 40 works · 14,627 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works 🧭 Topics (1): [🗨️ Dialogue systems](https://md.research.elamur.ai/free/en/topics/dialogue-systems/) · 401 works ## Cite Cite as: Siqi Bao, Huang He, Fan Wang, Hua Wu, Haifeng Wang, Wenquan Wu, Zhen Guo, Zhibin Liu and Xinchao Xu. 2021. PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning. Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{bao2021plato, title = {{PLATO-2: Towards Building an Open-Domain Chatbot via Curriculum Learning}}, author = {Siqi Bao, Huang He, Fan Wang, Hua Wu, Haifeng Wang, Wenquan Wu, Zhen Guo, Zhibin Liu and Xinchao Xu}, booktitle = {Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021}, year = {2021}, doi = {10.18653/v1/2021.findings-acl.222}, url = {https://doi.org/10.18653/v1/2021.findings-acl.222} } ``` - [← previous Out of Order: How important is the sequential order of words in a sentence in Natural Language Understanding tasks?](https://md.research.elamur.ai/free/en/papers/pham-2021-out-order-how-important-is/) - [next → Zero-shot Neural Passage Retrieval via Domain-targeted Synthetic Question Generation](https://md.research.elamur.ai/free/en/papers/ma-2021-zero-shot-neural-passage-retrieval/) - [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/3/) ## References 6 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, and Quoc V. Le. 2020. [Towards a human-like open-domain chatbot](https://md.research.elamur.ai/free/en/papers/adiwardana-2020-towards-human-like-open-domain/). arXiv preprint arXiv:2001.09977. 2. Siqi Bao, Bingjin Chen, Huang He, Xin Tian, Han Zhou, Fan Wang, Hua Wu, Haifeng Wang, Wenquan Wu, and Yingzhan Lin. 2021. A unified pre-training framework for conversational ai. arXiv preprint arXiv:2105.02482. — [arXiv:2105.02482](https://arxiv.org/abs/2105.02482) [find](https://scholar.google.com/scholar?q=Siqi+Bao%2C+Bingjin+Chen%2C+Huang+He%2C+Xin+Tian%2C+Han+Zhou%2C+Fan+Wang%2C+Hua+Wu%2C+Haifeng+Wang%2C+Wenquan+Wu%2C+and+Yingzhan+Lin%2E+2021%2E+A+unified+pre%2Dtraining+framework+for+conversational+ai%2E+arXiv+preprint+arXiv%3A2) 3. Siqi Bao, Huang He, Fan Wang, Rongzhong Lian, and Hua Wu. 2019. Know more about each other: Evolving dialogue strategy via compound assessment. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5382– 5391. [find](https://scholar.google.com/scholar?q=Siqi+Bao%2C+Huang+He%2C+Fan+Wang%2C+Rongzhong+Lian%2C+and+Hua+Wu%2E+2019%2E+Know+more+about+each+other%3A+Evolving+dialogue+strategy+via+compound+assessment%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Associa) 4. Siqi Bao, Huang He, Fan Wang, Hua Wu, and Haifeng Wang. 2020. [Plato: Pre-trained dialogue generation model with discrete latent variable](https://md.research.elamur.ai/free/en/papers/bao-2020-plato-pre-trained-dialogue-generation/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 85–96. 5. Jason Baumgartner, Savvas Zannettou, Brian Keegan, Megan Squire, and Jeremy Blackburn. 2020. The pushshift reddit dataset. In Proceedings of the International AAAI Conference on Web and Social Media, volume 14, pages 830–839. [find](https://scholar.google.com/scholar?q=Jason+Baumgartner%2C+Savvas+Zannettou%2C+Brian+Keegan%2C+Megan+Squire%2C+and+Jeremy+Blackburn%2E+2020%2E+The+pushshift+reddit+dataset%2E+In+Proceedings+of+the+International+AAAI+Conference+on+Web+and+Social+Media%2C+) 6. Yoshua Bengio, Jérôme Louradour, Ronan Collobert, and Jason Weston. 2009. [Curriculum learning](https://md.research.elamur.ai/free/en/papers/bengio-2009-curriculum-learning/). In Proceedings of the 26th Annual International Conference on Machine Learning, pages 41–48. *Updated 24 October 2026*