--- title: "Kaitao Song — author profile and papers" description: "Kaitao Song, Fudan University: 3 papers on Elamur Research (2019–2023), h-index 18 — affiliations, topics, co-authors and who cites them in our library." canonical: "https://research.elamur.ai/free/en/authors/kaitao-song/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/authors/kaitao-song/) [home](https://md.research.elamur.ai/free/en/) / [authors](https://md.research.elamur.ai/free/en/authors/) / Kaitao Song # Kaitao Song Kaitao Song works on pre-training methods for neural language models and on systems built around large language models. Song's papers here ask how a model trained on unlabeled text can be made to serve both understanding and generation tasks, and later how a language model can direct other specialised models. The methods centre on Transformer architectures, masking and ordering schemes for pre-training objectives, and agent-style pipelines. The work falls under [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) and [Text generation](https://md.research.elamur.ai/free/en/topics/text-generation/). The most cited paper here is [MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://md.research.elamur.ai/free/en/papers/song-2019-mass-masked-sequence-sequence-pre/) (2019). Encoder-only methods such as BERT suit generation poorly, so MASS trains an encoder and a decoder jointly. A contiguous stretch of a sentence is masked at the encoder, and the decoder must rebuild just that stretch. The approach was tested on multilingual WMT monolingual text and fine-tuned for several generation tasks. A year later, [MPNet](https://md.research.elamur.ai/free/en/papers/song-2020-mpnet-masked-permuted-pre-training/) (2020) placed the masked objective of BERT and the permuted objective of XLNet in one shared framework. It then combined their strengths, predicting tokens in permuted order while keeping position information so the model sees a full-length sentence, as it does during fine-tuning. The newest paper here, [HuggingGPT](https://md.research.elamur.ai/free/en/papers/shen-2023-hugginggpt-solving-ai-tasks-chatgpt/) (2023), moves from training objectives to coordination. ChatGPT acts as a controller that plans a multi-step job and calls expert models from the Hugging Face hub, using their natural-language descriptions as a common interface. This extends a text-only model to tasks involving images and speech. Song's longest-running affiliation is [Microsoft Research Asia](https://md.research.elamur.ai/free/en/companies/microsoft-research-asia/) in Beijing, with earlier bylines at [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) in Redmond and at [Nanjing University of Science and Technology](https://md.research.elamur.ai/free/en/universities/nanjing-university-of-science-and-technology/). More recent papers also list [Fudan University](https://md.research.elamur.ai/free/en/universities/fudan-university/) and [Apple](https://md.research.elamur.ai/free/en/companies/apple/). Frequent co-authors include [Xu Tan](https://md.research.elamur.ai/free/en/authors/xu-tan/), [Tao Qin](https://md.research.elamur.ai/free/en/authors/tao-qin/), [Tie-Yan Liu](https://md.research.elamur.ai/free/en/authors/tie-yan-liu/) and [Jianfeng Lu](https://md.research.elamur.ai/free/en/authors/jianfeng-lu/). Kaitao Song has 3 papers in our library, in 2019–2023, most of them on [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/), [Text generation](https://md.research.elamur.ai/free/en/topics/text-generation/); often with [Xu Tan](https://md.research.elamur.ai/free/en/authors/xu-tan/), [Jianfeng Lu](https://md.research.elamur.ai/free/en/authors/jianfeng-lu/), [Tao Qin](https://md.research.elamur.ai/free/en/authors/tao-qin/). 3 papers in our library · 2019–2023 current affiliation [Fudan University](https://md.research.elamur.ai/free/en/universities/fudan-university/), Shanghai, China; [ShanghaiTech University](https://md.research.elamur.ai/free/en/universities/shanghaitech-university/), Shanghai, China; [Microsoft Research Asia (China)](https://md.research.elamur.ai/free/en/companies/microsoft-research-asia/), Beijing, China; [Apple (United States)](https://md.research.elamur.ai/free/en/companies/apple/), Cupertino, United States [OpenAlex](https://openalex.org/A5028035527) · [ORCID](https://orcid.org/0000-0002-4046-8594) · [**h-index 18**](https://openalex.org/authors/A5028035527) · [77 works](https://openalex.org/works?filter=author.id:A5028035527) · [9 672 citations](https://openalex.org/works?filter=author.id:A5028035527&sort=cited_by_count:desc) OpenAlex, 2026-10-14 [3 works in our library](https://md.research.elamur.ai/free/en/authors/kaitao-song/#papers) · [1 237 citations in our library](https://md.research.elamur.ai/free/en/authors/kaitao-song/cited-by/) · 4 more works in our library · [⭐ Paper Star rank #911](https://md.research.elamur.ai/free/en/paper-stars/page/10/#star-911) The profile is the library's match of the name; two people with one name are not always told apart. Elsewhere: [Google Scholar](https://scholar.google.com/citations?view_op=search_authors&mauthors=Kaitao+Song) · [Semantic Scholar](https://www.semanticscholar.org/search?q=Kaitao+Song) · [dblp](https://dblp.org/search?q=Kaitao+Song) · [ACL Anthology](https://aclanthology.org/search/?q=Kaitao+Song) ## Affiliations - [Microsoft Research Asia (China)](https://md.research.elamur.ai/free/en/companies/microsoft-research-asia/) · Beijing, China —current · 2021–2026 on 17 papers - [Apple (United States)](https://md.research.elamur.ai/free/en/companies/apple/) · Cupertino, United States —current · 2025–2026 on 6 papers - [Fudan University](https://md.research.elamur.ai/free/en/universities/fudan-university/) · Shanghai, China —current · 2025–2026 on 4 papers - [ShanghaiTech University](https://md.research.elamur.ai/free/en/universities/shanghaitech-university/) · Shanghai, China —current · 2026 on 1 paper - [Microsoft (United States)](https://md.research.elamur.ai/free/en/companies/microsoft/) · Redmond, United States —2018–2025 on 12 papers - [Nanjing University of Science and Technology](https://md.research.elamur.ai/free/en/universities/nanjing-university-of-science-and-technology/) · Nanjing, China —2018–2024 on 17 papers - [Zhejiang University](https://md.research.elamur.ai/free/en/universities/zhejiang-university/) · Hangzhou, China —2023–2024 on 2 papers From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine. 2 more institutions on single bylines are not listed. ## Topics [Language models & pretraining 2](https://md.research.elamur.ai/free/en/topics/language-models/) · [Text generation 1](https://md.research.elamur.ai/free/en/topics/text-generation/) ## Works by year ### 2020s [**2023** 1 paper](https://md.research.elamur.ai/free/en/papers/shen-2023-hugginggpt-solving-ai-tasks-chatgpt/) · [**2020** 1 paper](https://md.research.elamur.ai/free/en/papers/song-2020-mpnet-masked-permuted-pre-training/) ### 2010s [**2019** 1 paper](https://md.research.elamur.ai/free/en/papers/song-2019-mass-masked-sequence-sequence-pre/) ## Papers The most cited first (OpenAlex and our library). 1. **[MASS: Masked Sequence to Sequence Pre-training for Language Generation](https://md.research.elamur.ai/free/en/papers/song-2019-mass-masked-sequence-sequence-pre/)** *(2019 · 1 109 citations)* Pre-training methods like BERT work well for language understanding, but they train only an encoder. That makes them a poor fit for generation tasks, which use a full encoder-decoder. MASS addresses this with one pre-training objective that trains both halves together on unlabeled text. A contiguous span of a… 2. **[MPNet: Masked and Permuted Pre-training for Language Understanding](https://md.research.elamur.ai/free/en/papers/song-2020-mpnet-masked-permuted-pre-training/)** *(2020 · 944 citations)* BERT and XLNet each use a pre-training objective with its own weakness. BERT's masked language modeling (MLM) predicts each masked token independently of the others. XLNet's permuted language modeling (PLM) captures dependencies between predicted tokens, but it hides the positions of tokens that come later. As a… 3. **[HuggingGPT: Solving AI Tasks with ChatGPT and its Friends in Hugging Face](https://md.research.elamur.ai/free/en/papers/shen-2023-hugginggpt-solving-ai-tasks-chatgpt/)** *(2023 · 737 citations)* Large language models handle text well, but they cannot directly see images, hear speech, or coordinate several specialised models to finish a multi-step job. ## Also in our library 4 more works of theirs are in our library without a page here yet; they will appear as we publish them. ## Co-authors here 6 co-authors with a page, the most papers together first. - [Xu Tan](https://md.research.elamur.ai/free/en/authors/xu-tan/) ×3 - [Jianfeng Lu](https://md.research.elamur.ai/free/en/authors/jianfeng-lu/) ×2 - [Tao Qin](https://md.research.elamur.ai/free/en/authors/tao-qin/) ×2 - [Tie-Yan Liu](https://md.research.elamur.ai/free/en/authors/tie-yan-liu/) ×2 - [Dongsheng Li](https://md.research.elamur.ai/free/en/authors/dongsheng-li/) - [Yueting Zhuang](https://md.research.elamur.ai/free/en/authors/yueting-zhuang/) ## Cited by Papers of our library cite their works 1 237 times (of the 140 884 papers whose bibliographies we read); [95 citing papers with a page here →](https://md.research.elamur.ai/free/en/authors/kaitao-song/cited-by/).