--- title: "Bo Wen — author profile and papers" description: "Bo Wen: 2 papers on Elamur Research (2021–2024) — affiliations, topics, co-authors and who cites them in our library." canonical: "https://research.elamur.ai/free/en/authors/bo-wen/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/authors/bo-wen/) [home](https://md.research.elamur.ai/free/en/) / [authors](https://md.research.elamur.ai/free/en/authors/) / Bo Wen # Bo Wen Bo Wen's papers here deal with transformer-based [language models and pretraining](https://md.research.elamur.ai/free/en/topics/language-models/), from two angles. One is architectural: how a model whose self-attention ignores word order can be given a sense of position. The other concerns data and coverage, specifically building encoders for the spoken varieties of Arabic, which links the work to research on [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/). The most cited of these works is [RoFormer: Enhanced Transformer with Rotary Position Embedding](https://md.research.elamur.ai/free/en/papers/su-2021-roformer-enhanced-transformer-rotary-position/) (2021). It introduces Rotary Position Embedding (RoPE). Instead of adding a position vector to each token, it rotates the query and key vectors through an angle tied to the token's absolute place in the sequence. Attention scores then end up depending only on the distance between tokens. The paper motivates this with complex numbers in two dimensions and extends it to higher dimensions by treating coordinates in pairs. Because of this design, the method works with sequences of different lengths and fits alongside linear self-attention, which additive position schemes handle poorly. The more recent paper, [AlcLaM: Arabic Dialectal Language Model](https://md.research.elamur.ai/free/en/papers/ahmed-2024-alclam-arabic-dialectal-language-model/) (2024), turns to the gap between Modern Standard Arabic and the regional dialects people actually write online. The authors gathered comments and posts from Arabic social media pages and used a classifier to filter out standard-Arabic passages such as religious quotations. They added subtitle text, built a vocabulary from dialectal material, and trained a BERT-base encoder from scratch. Its training corpus is far smaller than those behind CAMeL, MARBERT and ArBERT. The evaluation also groups a fine-grained dialect label set into coarser regional and binary tasks, which shows at what level of detail models break down. [Yunfeng Liu](https://md.research.elamur.ai/free/en/authors/yunfeng-liu/) co-authored both papers, and [Jianlin Su](https://md.research.elamur.ai/free/en/authors/jianlin-su/) collaborated on RoFormer. Bo Wen has 2 papers in our library, in 2021–2024, most of them on [Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/), [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/); often with [Yunfeng Liu](https://md.research.elamur.ai/free/en/authors/yunfeng-liu/), [Jianlin Su](https://md.research.elamur.ai/free/en/authors/jianlin-su/). 2 papers in our library · 2021–2024 No OpenAlex profile matched yet. [📄 2 works in our library](https://md.research.elamur.ai/free/en/authors/bo-wen/#papers) · [🔗 1,073 citations in our library](https://md.research.elamur.ai/free/en/authors/bo-wen/cited-by/) · [⭐ Paper Star rank #1898](https://md.research.elamur.ai/free/en/paper-stars/page/19/#star-1898) The profile is the library's match of the name; two people with one name are not always told apart. Elsewhere: [Google Scholar](https://scholar.google.com/citations?view_op=search_authors&mauthors=Bo+Wen) · [Semantic Scholar](https://www.semanticscholar.org/search?q=Bo+Wen) · [dblp](https://dblp.org/search?q=Bo+Wen) · [ACL Anthology](https://aclanthology.org/search/?q=Bo+Wen) ## Topics [Language models & pretraining (1)](https://md.research.elamur.ai/free/en/topics/language-models/) · [Semitic languages (1)](https://md.research.elamur.ai/free/en/topics/semitic-languages/) ## Works by year ### 2020s [**2024** 1 paper](https://md.research.elamur.ai/free/en/papers/ahmed-2024-alclam-arabic-dialectal-language-model/) · [**2021** 1 paper](https://md.research.elamur.ai/free/en/papers/su-2021-roformer-enhanced-transformer-rotary-position/) ## Papers The most cited first (OpenAlex and our library). 1. **[RoFormer: Enhanced Transformer with Rotary Position Embedding](https://md.research.elamur.ai/free/en/papers/su-2021-roformer-enhanced-transformer-rotary-position/)** *(2021 · 1,072 citations)* Self-attention in transformers is inherently blind to token order, so position has to be encoded separately. 2. **[AlcLaM: Arabic Dialectal Language Model](https://md.research.elamur.ai/free/en/papers/ahmed-2024-alclam-arabic-dialectal-language-model/)** *(2024 · 1 citation)* Most Arabic pre-trained language models learn mainly from Modern Standard Arabic (MSA). Everyday Arabic online is written in regional dialects, so these models often miss dialect words and give too much weight to MSA tokens. The authors scraped posts and comments from Arabic YouTube channels and Facebook Pages and… ## Co-authors here 2 co-authors with a page, the most papers together first. - [Yunfeng Liu](https://md.research.elamur.ai/free/en/authors/yunfeng-liu/) ×2 - [Jianlin Su](https://md.research.elamur.ai/free/en/authors/jianlin-su/) ## Cited by Papers of our library cite their works 1,073 times (of the 140,884 papers whose bibliographies we read); [85 citing papers with a page here →](https://md.research.elamur.ai/free/en/authors/bo-wen/cited-by/).