--- title: "Kaiyue Wen — author profile and papers — Elamur Research" description: "Kaiyue Wen, Shanghai University of Finance and Economics: 2 papers on Elamur Research (2022–2025), h-index 3 — affiliations, topics…" canonical: "https://research.elamur.ai/free/en/authors/kaiyue-wen/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/authors/kaiyue-wen/) [home](https://md.research.elamur.ai/free/en/) / [authors](https://md.research.elamur.ai/free/en/authors/) / Kaiyue Wen # Kaiyue Wen Kaiyue Wen has 2 papers in our library, in 2022–2025, most of them on [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/), [Interpretability](https://md.research.elamur.ai/free/en/topics/interpretability/); often with [Bo Zheng](https://md.research.elamur.ai/free/en/authors/bo-zheng/), [Dayiheng Liu](https://md.research.elamur.ai/free/en/authors/dayiheng-liu/), [Ivan Titov](https://md.research.elamur.ai/free/en/authors/ivan-titov/). 2 papers in our library · 2022–2025 [OpenAlex](https://openalex.org/A5008378675) · [ORCID](https://orcid.org/0009-0002-9168-1851) · [**h-index 3**](https://openalex.org/authors/A5008378675) · [31 works](https://openalex.org/works?filter=author.id:A5008378675) · [109 citations](https://openalex.org/works?filter=author.id:A5008378675&sort=cited_by_count:desc) OpenAlex, 2026-10-14 [📄 2 works in our library](https://md.research.elamur.ai/free/en/authors/kaiyue-wen/#papers) · [🔗 Cited 183 times in our library](https://md.research.elamur.ai/free/en/authors/kaiyue-wen/cited-by/) · [8 more in our library without a page yet](#also) · [⭐ Paper Stars ranking: #7812](https://md.research.elamur.ai/free/en/paper-stars/page/79/#star-7812) The profile is the library's match of the name; two people with one name are not always told apart. Elsewhere: [Google Scholar](https://scholar.google.com/citations?view_op=search_authors&mauthors=Kaiyue+Wen) · [Semantic Scholar](https://www.semanticscholar.org/search?q=Kaiyue+Wen) · [dblp](https://dblp.org/search?q=Kaiyue+Wen) · [ACL Anthology](https://aclanthology.org/search/?q=Kaiyue+Wen) ## Affiliations [🎓**Stanford University** Stanford · United States 2024–2026 · 17 papers*current*](https://md.research.elamur.ai/free/en/universities/stanford-university/) · [🏢**Huawei Technologies** Shenzhen · China 2026 · 1 paper*current*](https://md.research.elamur.ai/free/en/companies/huawei-technologies/) · [🎓**Shanghai University of Finance and Economics** Shanghai · China 2026 · 1 paper*current*](https://md.research.elamur.ai/free/en/universities/shanghai-university-of-finance-and-economics/) · [🎓**Tsinghua University** Beijing · China 2022–2024 · 4 papers](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine. ## Fields, topics and concepts Of its works in our library, with the number of works on each. 📚 Shelves 4 The field of science this work belongs to (OpenAlex fields).[Artificial Intelligence · 2 works](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · [Arts and Humanities · 2 works](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · [Computer Science · 2 works](https://md.research.elamur.ai/free/en/shelves/computer-science/) · [Language and Linguistics · 2 works](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) 🧭 Topics 2 Research subjects and languages the work is about.[Interpretability · 1 work](https://md.research.elamur.ai/free/en/topics/interpretability/) · [Language models & pretraining · 1 work](https://md.research.elamur.ai/free/en/topics/language-models/) ## Works by year ### 2020s [**2025** 1 paper](https://md.research.elamur.ai/free/en/authors/kaiyue-wen/year/2025/) · [**2022** 1 paper](https://md.research.elamur.ai/free/en/authors/kaiyue-wen/year/2022/) ## Papers The most cited first (OpenAlex and our library). 1. **[Finding Skill Neurons in Pre-trained Transformer-based Language Models](https://md.research.elamur.ai/free/en/papers/wang-2022-finding-skill-neurons-pre-trained/)** *(2022 · 85 citations)* Pre-trained Transformers perform well across many NLP tasks, but it is unclear where task-relevant abilities sit among their parameters. 2. **[Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models](https://md.research.elamur.ai/free/en/papers/qiu-2025-demons-detail-implementing-load-balancing/)** *(2025 · 21 citations)* Mixture-of-Experts LLMs rely on a Load-balancing Loss (LBL) so that the router spreads tokens across experts. ## All works 10 works of theirs in our library, the most cited first; 2 with a page here, the others link their original until we summarise them. 1. **[On Transferability of Prompt Tuning for Natural Language Processing ↗](https://aclanthology.org/2022.naacl-main.290/) Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies · no summary yet**Yusheng Su et al.*2022 · 107 citations* 2. **[Finding Skill Neurons in Pre-trained Transformer-based Language Models](https://md.research.elamur.ai/free/en/papers/wang-2022-finding-skill-neurons-pre-trained/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Xiaozhi Wang, Kaiyue Wen, Zhengyan Zhang, et al.*2022 · 85 citations* 3. **[Demons in the Detail: On Implementing Load Balancing Loss for Training Specialized Mixture-of-Expert Models](https://md.research.elamur.ai/free/en/papers/qiu-2025-demons-detail-implementing-load-balancing/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Zihan Qiu, Zeyu Huang, Bo Zheng, et al.*2025 · 21 citations* 4. **[Understanding Warmup-Stable-Decay Learning Rates: A River Valley Loss Landscape Perspective ↗](https://arxiv.org/abs/2410.05192) arXiv (Cornell University) · no summary yet**Kaiyue Wen et al.*2024 · 18 citations* 5. **[RNNs are not Transformers (Yet): The Key Bottleneck on In-context Retrieval ↗](https://arxiv.org/abs/2402.18510) arXiv:2402.18510 · no summary yet**Kaiyue Wen, Xingyu Dang, Kaifeng Lyu*2024 · 17 citations* 6. **[Fantastic Pretraining Optimizers and Where to Find Them ↗](https://arxiv.org/abs/2509.02046) arXiv (Cornell University) · no summary yet**Kaiyue Wen et al.*2025 · 11 citations* 7. **[Weight Ensembling Improves Reasoning in Language Models ↗](https://arxiv.org/abs/2504.10478) arXiv:2504.10478 · COLM 2025 · no summary yet**Xingyu Dang et al.*2025 · 10 citations* 8. **[Overtrained Language Models Are Harder to Fine-Tune ↗](https://arxiv.org/abs/2503.19206) arXiv (Cornell University) · no summary yet**Jacob M. Springer et al.*2025 · 8 citations* 9. **[Fantastic Pretraining Optimizers and Where to Find Them II: Hyperball Optimization ↗](https://arxiv.org/abs/2606.16899) arXiv (Cornell University) · no summary yet**Kaiyue Wen et al.*2026 · 2 citations* 10. **[Priced Guidance: Can Language Models Generate Future Research Ideas? ↗](https://arxiv.org/abs/2610.04976) arXiv (Cornell University) · no summary yet**Kaiyue Wen, Tengyu Ma, Percy Liang*2026* ## Co-authors here 12 of 13 co-authors with a page, the most papers together first. - [Bo Zheng](https://md.research.elamur.ai/free/en/authors/bo-zheng/) - [Dayiheng Liu](https://md.research.elamur.ai/free/en/authors/dayiheng-liu/) - [Ivan Titov](https://md.research.elamur.ai/free/en/authors/ivan-titov/) - [Jingren Zhou](https://md.research.elamur.ai/free/en/authors/jingren-zhou/) - [Juanzi Li](https://md.research.elamur.ai/free/en/authors/juanzi-li/) - [Junyang Lin](https://md.research.elamur.ai/free/en/authors/junyang-lin/) - [Lei Hou](https://md.research.elamur.ai/free/en/authors/lei-hou/) - [Rui Men](https://md.research.elamur.ai/free/en/authors/rui-men/) - [Xiaozhi Wang](https://md.research.elamur.ai/free/en/authors/xiaozhi-wang/) - [Zekun Wang](https://md.research.elamur.ai/free/en/authors/zekun-wang/) - [Zeyu Huang](https://md.research.elamur.ai/free/en/authors/zeyu-huang/) - [Zhengyan Zhang](https://md.research.elamur.ai/free/en/authors/zhengyan-zhang/) ## Venues Where their works here were published, the most works first, with the editions of those that have a page. - [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/): [ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/) - [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/): [EMNLP 2022](https://md.research.elamur.ai/free/en/venues/emnlp/2022/) - Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies ## Cited by Papers of our library cite their works 183 times (of the 149,439 papers whose bibliographies we read); [26 citing papers with a page here →](https://md.research.elamur.ai/free/en/authors/kaiyue-wen/cited-by/). Updated 30 October 2026