---
title: "Who cites Peter Hawkins — 531 papers of our library"
description: "531 papers of the Elamur Research library cite the works of Peter Hawkins, found in their bibliographies; the most cited first."
canonical: "https://research.elamur.ai/free/en/authors/peter-hawkins/cited-by/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/authors/peter-hawkins/cited-by/)

[home](https://md.research.elamur.ai/free/en/) / [authors](https://md.research.elamur.ai/free/en/authors/) / [Peter Hawkins](https://md.research.elamur.ai/free/en/authors/peter-hawkins/) / cited by

# Who cites Peter Hawkins

540 citations of their works from 531 papers of our library (of the 149,439 whose bibliographies we read) · 87 with a page here, the most cited first · [← Peter Hawkins](https://md.research.elamur.ai/free/en/authors/peter-hawkins/)

1. **[Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://md.research.elamur.ai/free/en/papers/liu-2021-swin-transformer-hierarchical-vision-transformer/) [IEEE/CVF International Conference on Computer Vision (ICCV)](https://md.research.elamur.ai/free/en/venues/ieee-cvf-international-conference-on-computer-vision-iccv/)**Ze Liu, Yutong Lin, Yue Cao, et al.*2021 · 33,968 citations*
2. [**Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer arXiv:1910.10683**Colin Raffel, Noam Shazeer, Adam Roberts, et al.*2019 · 11,325 citations*](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/)
3. [**PaLM: Scaling Language Modeling with Pathways arXiv:2204.02311**Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, et al.*2022 · 4,753 citations*](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
4. [**Scaling Laws for Neural Language Models arXiv:2001.08361**Jared Kaplan, Sam McCandlish, Tom Henighan, et al.*2020 · 3,434 citations*](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/)
5. [**On the Opportunities and Risks of Foundation Models arXiv:2108.07258**Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, et al.*2021 · 3,160 citations*](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/)
6. [**RoFormer: Enhanced Transformer with Rotary Position Embedding arXiv:2104.09864**Jianlin Su, Yu Lu, Shengfeng Pan, et al.*2021 · 3,028 citations*](https://md.research.elamur.ai/free/en/papers/su-2021-roformer-enhanced-transformer-rotary-position/)
7. [**ALBERT: A Lite BERT for Self-supervised Learning of Language Representations arXiv:1909.11942**Zhenzhong Lan, Mingda Chen, Sebastian Goodman, et al.*2019 · 2,806 citations*](https://md.research.elamur.ai/free/en/papers/lan-2019-albert-lite-bert-self-supervised/)
8. **[Efficient Transformers: A Survey](https://md.research.elamur.ai/free/en/papers/tay-2020-efficient-transformers-survey/) [ACM Computing Surveys](https://md.research.elamur.ai/free/en/venues/acm-computing-surveys/)**Yi Tay, Mostafa Dehghani, Dara Bahri, et al.*2020 · 1,539 citations*
9. [**FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning arXiv:2307.08691**Tri Dao*2023 · 1,240 citations*](https://md.research.elamur.ai/free/en/papers/dao-2023-flashattention-2-faster-attention-better/)
10. [**Reformer: The Efficient Transformer arXiv:2001.04451**Nikita Kitaev, Łukasz Kaiser, Anselm Levskaya*2020 · 829 citations*](https://md.research.elamur.ai/free/en/papers/kitaev-2020-reformer-efficient-transformer/)
11. **[FNet: Mixing Tokens with Fourier Transforms](https://md.research.elamur.ai/free/en/papers/thorp-2022-fnet-mixing-tokens-fourier-transforms/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, et al.*2022 · 597 citations*
12. **[AudioLM: a Language Modeling Approach to Audio Generation](https://md.research.elamur.ai/free/en/papers/borsos-2022-audiolm-language-modeling-approach-audio/) [IEEE/ACM Transactions on Audio Speech and Language Processing](https://md.research.elamur.ai/free/en/venues/ieee-acm-transactions-on-audio-speech-and-language-processing/)**Zalan Borsos, Raphael Marinier, Damien Vincent, et al.*2022 · 581 citations*
13. [**Massively Multilingual Neural Machine Translation in the Wild: Findings and Challenges arXiv:1907.05019**Naveen Arivazhagan, Ankur Bapna, Orhan Firat, et al.*2019 · 564 citations*](https://md.research.elamur.ai/free/en/papers/arivazhagan-2019-massively-multilingual-neural-machine-translation/)
14. **[Simple, Scalable Adaptation for Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/bapna-2019-simple-scalable-adaptation-neural-machine/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Ankur Bapna, Orhan Firat*2019 · 530 citations*
15. [**Vision Mamba: Efficient Visual Representation Learning with Bidirectional State Space Model arXiv:2401.09417**Lianghui Zhu, Bencheng Liao, Qian Zhang, et al.*2024 · 500 citations*](https://md.research.elamur.ai/free/en/papers/zhu-2024-vision-mamba-efficient-visual-representation/)
16. [**GLaM: Efficient Scaling of Language Models with Mixture-of-Experts arXiv:2112.06905**Nan Du, Yanping Huang, Andrew M. Dai, et al.*2021 · 496 citations*](https://md.research.elamur.ai/free/en/papers/du-2021-glam-efficient-scaling-language-models/)
17. [**Orca: A Distributed Serving System for Transformer-Based Generative Models OSDI 2022**Gyeong-In Yu, Joo Seong Jeong, Geon-Woo Kim, et al.*2022 · 478 citations*](https://md.research.elamur.ai/free/en/papers/yu-2022-orca-distributed-serving-system-transformer/)
18. [**The Falcon Series of Open Language Models arXiv:2311.16867**Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, et al.*2023 · 443 citations*](https://md.research.elamur.ai/free/en/papers/almazrouei-2023-falcon-series-open-language-models/)
19. [**Long Range Arena: A Benchmark for Efficient Transformers arXiv:2011.04006**Yi Tay, Mostafa Dehghani, Samira Abnar, et al.*2020 · 414 citations*](https://md.research.elamur.ai/free/en/papers/tay-2020-long-range-arena-benchmark-efficient/)
20. **[FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision](https://md.research.elamur.ai/free/en/papers/shah-2024-flashattention-3-fast-accurate-attention/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Jay Shah, Ganesh Bikshandi, Ying Zhang, et al.*2024 · 400 citations*
21. **[LongT5: Efficient Text-To-Text Transformer for Long Sequences](https://md.research.elamur.ai/free/en/papers/guo-2022-longt5-efficient-text-text-transformer/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Mandy Guo, Joshua Ainslie, David Uthus, et al.*2022 · 399 citations*
22. [**PyTorch Distributed: Experiences on Accelerating Data Parallel Training arXiv:2006.15704**Shen Li, Yanli Zhao, Rohan Varma, et al.*2020 · 348 citations*](https://md.research.elamur.ai/free/en/papers/li-2020-pytorch-distributed-experiences-accelerating-data/)
23. **[Efficient Attentions for Long Document Summarization](https://md.research.elamur.ai/free/en/papers/huang-2021-efficient-attentions-long-document-summarization/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Luyang Huang, Shuyang Cao, Nikolaus Parulian, et al.*2021 · 336 citations*
24. [**ZeRO-Offload: Democratizing Billion-Scale Model Training arXiv:2101.06840**Jie Ren, Samyam Rajbhandari, Reza Yazdani Aminabadi, et al.*2021 · 309 citations*](https://md.research.elamur.ai/free/en/papers/ren-2021-zero-offload-democratizing-billion-scale/)
25. [**Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language arXiv:2204.00598**Andy Zeng, Maria Attarian, Brian Ichter, et al.*2022 · 302 citations*](https://md.research.elamur.ai/free/en/papers/zeng-2022-socratic-models-composing-zero-shot/)
26. [**Challenges and Applications of Large Language Models arXiv:2307.10169**Jean Kaddour, Joshua Harris, Maximilian Mozes, et al.*2023 · 301 citations*](https://md.research.elamur.ai/free/en/papers/kaddour-2023-challenges-applications-large-language-models/)
27. **[Canine: Pre-training an Efficient Tokenization-Free Encoder for Language Representation](https://md.research.elamur.ai/free/en/papers/clark-2022-canine-pre-training-efficient-tokenization/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Jonathan H. Clark, Dan Garrette, Iulia Turc, et al.*2022 · 301 citations*
28. **[xLSTM: Extended Long Short-Term Memory](https://md.research.elamur.ai/free/en/papers/beck-2024-xlstm-extended-long-short-term/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Maximilian Beck, Korbinian Pöppel, Markus Spanring, et al.*2024 · 296 citations*
29. [**Alpa: Automating Inter- and Intra-Operator Parallelism for Distributed Deep Learning OSDI 2022**Lianmin Zheng, Zhuohan Li, Hao Zhang, et al.*2022 · 279 citations*](https://md.research.elamur.ai/free/en/papers/zheng-2022-alpa-automating-inter-intra-operator/)
30. [**Reducing Activation Recomputation in Large Transformer Models arXiv:2205.05198**Vijay Korthikanti, Jared Casper, Sangkug Lym, et al.*2022 · 273 citations*](https://md.research.elamur.ai/free/en/papers/korthikanti-2022-reducing-activation-recomputation-large-transformer/)
31. [**Perceiver: General Perception with Iterative Attention arXiv:2103.03206**Andrew Jaegle, Felix Gimeno, Andrew Brock, et al.*2021 · 269 citations*](https://md.research.elamur.ai/free/en/papers/jaegle-2021-perceiver-general-perception-iterative-attention/)
32. [**ZeRO-Infinity: Breaking the GPU Memory Wall for Extreme Scale Deep Learning SC 2021**Samyam Rajbhandari, Olatunji Ruwase, Jeff Rasley, et al.*2021 · 254 citations*](https://md.research.elamur.ai/free/en/papers/rajbhandari-2021-zero-infinity-breaking-gpu-memory/)
33. [**Transformers learn in-context by gradient descent arXiv:2212.07677**Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, et al.*2022 · 247 citations*](https://md.research.elamur.ai/free/en/papers/oswald-2022-transformers-learn-context-gradient-descent/)
34. **[A New Generation of Perspective API: Efficient Multilingual Character-level Transformers](https://md.research.elamur.ai/free/en/papers/lees-2022-new-generation-perspective-api/) [ACM SIGKDD Conference on Knowledge Discovery and Data Mining (KDD)](https://md.research.elamur.ai/free/en/venues/acm-sigkdd-conference-on-knowledge-discovery-and-data-mining-kdd/)**Alyssa Lees, Vinh Q. Tran, Yi Tay, et al.*2022 · 238 citations*
35. [**Charformer: Fast Character Transformers via Gradient-based Subword Tokenization arXiv:2106.12672**Yi Tay, Vinh Q. Tran, Sebastian Ruder, et al.*2021 · 186 citations*](https://md.research.elamur.ai/free/en/papers/tay-2021-charformer-fast-character-transformers-gradient/)
36. **[Adapting Language Models to Compress Contexts](https://md.research.elamur.ai/free/en/papers/chevalier-2023-adapting-language-models-compress-contexts/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Alexis Chevalier, Alexander Wettig, Anirudh Ajith, et al.*2023 · 182 citations*
37. [**AlpaServe: Statistical Multiplexing with Model Parallelism for Deep Learning Serving OSDI 2023**Zhuohan Li, Lianmin Zheng, Yinmin Zhong, et al.*2023 · 170 citations*](https://md.research.elamur.ai/free/en/papers/li-2023-alpaserve-statistical-multiplexing-model-parallelism/)
38. **[SCROLLS: Standardized CompaRison Over Long Language Sequences](https://md.research.elamur.ai/free/en/papers/shaham-2022-scrolls-standardized-comparison-over-long/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Uri Shaham, Elad Segal, Maor Ivgi, et al.*2022 · 165 citations*
39. [**PanGu-α: Large-scale Autoregressive Pretrained Chinese Language Models with Auto-parallel Computation arXiv:2104.12369**Wei Zeng, Xiaozhe Ren, Teng Su, et al.*2021 · 161 citations*](https://md.research.elamur.ai/free/en/papers/zeng-2021-pangu-alpha-large-scale/)
40. **[Efficient Methods for Natural Language Processing: A Survey](https://md.research.elamur.ai/free/en/papers/treviso-2023-efficient-methods-natural-language-processing/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Marcos Treviso, Ji-Ung Lee, Tianchu Ji, et al.*2023 · 145 citations*
41. [**Rethinking embedding coupling in pre-trained language models arXiv:2010.12821**Hyung Won Chung, Thibault Févry, Henry Tsai, et al.*2020 · 142 citations*](https://md.research.elamur.ai/free/en/papers/chung-2020-rethinking-embedding-coupling-pre-trained/)
42. [**Scaling Up Models and Data with t5x and seqio arXiv:2203.17189**Adam Roberts, Hyung Won Chung, Anselm Levskaya, et al.*2022 · 130 citations*](https://md.research.elamur.ai/free/en/papers/roberts-2022-scaling-up-models-data-texttt/)
43. [**H2O: Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models arXiv:2306.14048**Zhenyu Zhang, Ying Sheng, Tianyi Zhou, et al.*2023 · 124 citations*](https://md.research.elamur.ai/free/en/papers/zhang-2023-h-2-o-heavy-hitter/)
44. [**LongNet: Scaling Transformers to 1,000,000,000 Tokens arXiv:2307.02486**Jiayu Ding, Shuming Ma, Li Dong, et al.*2023 · 123 citations*](https://md.research.elamur.ai/free/en/papers/ding-2023-longnet-scaling-transformers-1-000/)
45. **[A Length-Extrapolatable Transformer](https://md.research.elamur.ai/free/en/papers/sun-2023-length-extrapolatable-transformer/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Yutao Sun, Li Dong, Barun Patra, et al.*2023 · 118 citations*
46. [**In-context Autoencoder for Context Compression in a Large Language Model arXiv:2307.06945**Tao Ge, Jing Hu, Lei Wang, et al.*2023 · 114 citations*](https://md.research.elamur.ai/free/en/papers/ge-2023-context-autoencoder-context-compression-large/)
47. [**ExT5: Towards Extreme Multi-Task Scaling for Transfer Learning arXiv:2111.10952**Vamsi Aribandi, Yi Tay, Tal Schuster, et al.*2021 · 114 citations*](https://md.research.elamur.ai/free/en/papers/aribandi-2021-ext5-towards-extreme-multi-task/)
48. [**A Survey on Efficient Inference for Large Language Models arXiv:2404.14294**Zixuan Zhou, Xuefei Ning, Ke Hong, et al.*2024 · 113 citations*](https://md.research.elamur.ai/free/en/papers/zhou-2024-survey-efficient-inference-large-language/)
49. **[Learning from Task Descriptions](https://md.research.elamur.ai/free/en/papers/weller-2020-learning-task-descriptions/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Orion Weller, Nicholas Lourie, Matt Gardner, et al.*2020 · 112 citations*
50. **[Unlimiformer: Long-Range Transformers with Unlimited Length Input](https://md.research.elamur.ai/free/en/papers/bertsch-2023-unlimiformer-long-range-transformers-unlimited/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Amanda Bertsch, Uri Alon, Graham Neubig, et al.*2023 · 111 citations*
51. [**Linear Transformers Are Secretly Fast Weight Programmers arXiv:2102.11174**Imanol Schlag, Kazuki Irie, Jurgen Schmidhuber*2021 · 111 citations*](https://md.research.elamur.ai/free/en/papers/schlag-2021-linear-transformers-secretly-fast-weight/)
52. [**Efficient Large Language Models: A Survey arXiv:2312.03863**Zhongwei Wan, Xin Wang, Che Liu, et al.*2023 · 110 citations*](https://md.research.elamur.ai/free/en/papers/wan-2023-efficient-large-language-models-survey/)
53. **[Do Long-Range Language Models Actually Use Long-Range Context?](https://md.research.elamur.ai/free/en/papers/sun-2021-do-long-range-language-models/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Simeng Sun, Kalpesh Krishna, Andrew Mattarella-Micke, et al.*2021 · 101 citations*
54. [**Transformer Quality in Linear Time arXiv:2202.10447**Weizhe Hua, Zihang Dai, Hanxiao Liu, et al.*2022 · 99 citations*](https://md.research.elamur.ai/free/en/papers/hua-2022-transformer-quality-linear-time/)
55. [**Accelerating Sparse Deep Neural Networks arXiv 2021 (NVIDIA)**Asit Mishra, Jorge Albericio Latorre, Jeff Pool, et al.*2021 · 97 citations*](https://md.research.elamur.ai/free/en/papers/mishra-2021-accelerating-sparse-deep-neural-networks/)
56. [**Accelerating Distributed MoE Training and Inference with Lina arXiv:2210.17223**Jiamin Li, Yimin Jiang, Yibo Zhu, et al.*2022 · 92 citations*](https://md.research.elamur.ai/free/en/papers/li-2022-accelerating-distributed-moe-training-inference/)
57. **[How to Train Long-Context Language Models (Effectively)](https://md.research.elamur.ai/free/en/papers/gao-2025-how-train-long-context-language/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Tianyu Gao, Alexander Wettig, Howard Yen, et al.*2025 · 89 citations*
58. [**Landmark Attention: Random-Access Infinite Context Length for Transformers arXiv:2305.16300**Amirkeivan Mohtashami, Martin Jaggi*2023 · 88 citations*](https://md.research.elamur.ai/free/en/papers/mohtashami-2023-landmark-attention-random-access-infinite/)
59. **[Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?](https://md.research.elamur.ai/free/en/papers/tay-2023-scaling-laws-vs-model-architectures/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Yi Tay, Mostafa Dehghani, Samira Abnar, et al.*2023 · 86 citations*
60. **[MoBA: Mixture of Block Attention for Long-Context LLMs](https://md.research.elamur.ai/free/en/papers/lu-2025-moba-mixture-block-attention/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Enzhe Lu, Zhejun Jiang, Jingyuan Liu, et al.*2025 · 80 citations*
61. **[Domain Adaptation with Pre-trained Transformers for Query-Focused Abstractive Text Summarization](https://md.research.elamur.ai/free/en/papers/laskar-2022-domain-adaptation-pre-trained-transformers/) [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/)**Md Tahmid Rahman Laskar, Enamul Hoque, Jimmy Xiangji Huang*2022 · 76 citations*
62. **[SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration](https://md.research.elamur.ai/free/en/papers/zhang-2024-sageattention-accurate-8-bit-attention/) [International Conference on Learning Representations (ICLR)](https://md.research.elamur.ai/free/en/venues/international-conference-on-learning-representations-iclr/)**Jintao Zhang, Jia Wei, Haofeng Huang, et al.*2024 · 74 citations*
63. [**Pushing Mixture of Experts to the Limit: Extremely Parameter Efficient MoE for Instruction Tuning arXiv:2309.05444**Ted Zadouri, Ahmet Ustun, Arash Ahmadian, et al.*2023 · 69 citations*](https://md.research.elamur.ai/free/en/papers/zadouri-2023-pushing-mixture-experts-limit-extremely/)
64. **[The Devil in Linear Transformer](https://md.research.elamur.ai/free/en/papers/qin-2022-devil-linear-transformer/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Zhen Qin, Xiaodong Han, Weixuan Sun, et al.*2022 · 67 citations*
65. [**MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention arXiv:2506.13585**MiniMax, Aili Chen, Aonian Li, et al.*2025 · 66 citations*](https://md.research.elamur.ai/free/en/papers/minimax-2025-minimax-m1-scaling-test/)
66. [**A Comprehensive Survey on Long Context Language Modeling arXiv:2503.17407**Jiaheng Liu, Dawei Zhu, Zhiqi Bai, et al.*2025 · 64 citations*](https://md.research.elamur.ai/free/en/papers/liu-2025-comprehensive-survey-long-context-language/)
67. [**ERNIE 3.0 Titan: Exploring Larger-scale Knowledge Enhanced Pre-training for Language Understanding and Generation arXiv:2112.12731**Shuohuan Wang, Yu Sun, Yang Xiang, et al.*2021 · 63 citations*](https://md.research.elamur.ai/free/en/papers/wang-2021-ernie-3-0-titan/)
68. [**Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models arXiv:2402.02244**Xindi Wang, Mahsa Salmani, Parsa Omidi, et al.*2024 · 60 citations*](https://md.research.elamur.ai/free/en/papers/wang-2024-limits-survey-techniques-extend-context/)
69. [**Simple linear attention language models balance the recall-throughput tradeoff arXiv:2402.18668**Simran Arora, Sabri Eyuboglu, Michael Zhang, et al.*2024 · 59 citations*](https://md.research.elamur.ai/free/en/papers/arora-2024-simple-linear-attention-language-models/)
70. [**Scaling Laws of RoPE-based Extrapolation arXiv:2310.05209**Xiaoran Liu, Hang Yan, Shuo Zhang, et al.*2023 · 54 citations*](https://md.research.elamur.ai/free/en/papers/liu-2023-scaling-laws-rope-based/)
71. **[Long-Context Language Modeling with Parallel Context Encoding](https://md.research.elamur.ai/free/en/papers/yen-2024-long-context-language-modeling-parallel/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Howard Yen, Tianyu Gao, Danqi Chen*2024 · 44 citations*
72. [**TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer arXiv:2307.14995**Zhen Qin, Dong Li, Weigao Sun, et al.*2023 · 33 citations*](https://md.research.elamur.ai/free/en/papers/qin-2023-transnormerllm-faster-better-large/)
73. **[TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Text](https://md.research.elamur.ai/free/en/papers/lu-2025-turborag-accelerating-retrieval-augmented-generation/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Songshuo Lu, Hua Wang, Yutian Rong, et al.*2025 · 32 citations*
74. [**Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models arXiv:2401.04658**Zhen Qin, Weigao Sun, Dong Li, et al.*2024 · 31 citations*](https://md.research.elamur.ai/free/en/papers/qin-2024-lightning-attention-2-free/)
75. **[Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language Processing](https://md.research.elamur.ai/free/en/papers/ghaddar-2022-revisiting-pre-trained-language-models/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Abbas Ghaddar, Yimeng Wu, Sunyam Bagga, et al.*2022 · 27 citations*
76. [**Block-State Transformers arXiv:2306.09539**Mahan Fathi, Jonathan Pilault, Orhan Firat, et al.*2023 · 20 citations*](https://md.research.elamur.ai/free/en/papers/fathi-2023-block-state-transformers/)
77. [**Linearizing Large Language Models arXiv:2405.06640**Jean Mercat, Igor Vasiljevic, Sedrick Keh, et al.*2024 · 16 citations*](https://md.research.elamur.ai/free/en/papers/mercat-2024-linearizing-large-language-models/)
78. **[Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2025-attention-entropy-is-key-factor/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Zhisong Zhang, Yan Wang, Xinting Huang, et al.*2025 · 14 citations*
79. [**PanGu-π: Enhancing Language Model Architectures via Nonlinearity Compensation arXiv:2312.17276**Yunhe Wang, Hanting Chen, Yehui Tang, et al.*2023 · 10 citations*](https://md.research.elamur.ai/free/en/papers/wang-2023-pangu-pi-enhancing-language/)
80. [**Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language Understanding arXiv:2205.10687**Abbas Ghaddar, Yimeng Wu, Sunyam Bagga, et al.*2022 · 9 citations*](https://md.research.elamur.ai/free/en/papers/ghaddar-2022-revisiting-pre-trained-language-models-their/)
81. [**Just read twice: closing the recall gap for recurrent language models arXiv:2407.05483**Simran Arora; Aman Timalsina; Aaryan Singhal; Benjamin Spector; Sabri Eyuboglu; Xinyi Zhao; Ashish Rao; Atri Rudra;, Christopher Re*2024 · 8 citations*](https://md.research.elamur.ai/free/en/papers/arora-2024-just-read-twice-closing-recall/)
82. [**An End-to-End OCR Framework for Robust Arabic-Handwriting Recognition using a Novel Transformers-based Model and an Innovative 270 Million-Words Multi-Font Corpus of Classical Arabic with Diacritics arXiv:2208.11484**Aly Mostafa, Omar Mohamed, Ali Ashraf, et al.*2022 · 3 citations*](https://md.research.elamur.ai/free/en/papers/mostafa-2022-end-end-ocr-framework-robust/)
83. **[Cross-layer Attention Sharing for Pre-trained Large Language Models](https://md.research.elamur.ai/free/en/papers/mu-2026-cross-layer-attention-sharing-pre/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Yongyu Mu, Yuzhang Wu, Yuchun Fan, et al.*2026 · 2 citations*
84. [**MiniMax Sparse Attention arXiv:2606.13392**Xunhao Lai, Weiqi Xu, Yufeng Yang, et al.*2026 · 2 citations*](https://md.research.elamur.ai/free/en/papers/lai-2026-minimax-sparse-attention/)
85. **[A Survey of Retentive Network](https://md.research.elamur.ai/free/en/papers/yang-2026-survey-retentive-network/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Haiqi Yang, Zhiyuan Li, Yi Chang, et al.*2026*
86. **[From Logical to Computational Sparsity: Structure-Aware Block-Sparse Attention for Long-Code Completion](https://md.research.elamur.ai/free/en/papers/wang-2026-logical-computational-sparsity-structure-aware/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Yanli Wang, Yanlin Wang, Bowen Zhang, et al.*2026*
87. **[SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language Models](https://md.research.elamur.ai/free/en/papers/liu-2026-semtoken-semantic-aware-tokenization-efficient/) [Joint Conference on Lexical and Computational Semantics (*SEM)](https://md.research.elamur.ai/free/en/venues/starsem/)**Dong Liu, Yanxuan Yu*2026*

444 more citing papers of our library have no page here yet.

- [← back to](https://md.research.elamur.ai/free/en/authors/peter-hawkins/) — Peter Hawkins