---
title: "Who cites Rethinking Attention with — 387 citing papers"
description: "387 papers of the Elamur Research library cite «Rethinking Attention with Performers», found in their bibliographies; the most cited first."
canonical: "https://research.elamur.ai/free/en/papers/choromanski-2020-rethinking-attention-performers/cited-by/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/choromanski-2020-rethinking-attention-performers/cited-by/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [Rethinking Attention with Performers](https://md.research.elamur.ai/free/en/papers/choromanski-2020-rethinking-attention-performers/) / cited by

# Who cites «Rethinking Attention with Performers»

387 papers of our library cite it (of the 149,439 whose bibliographies we read) · 49 with a page here, the most cited first · [← back to «Rethinking Attention with Performers»](https://md.research.elamur.ai/free/en/papers/choromanski-2020-rethinking-attention-performers/)

1. **[Swin Transformer: Hierarchical Vision Transformer using Shifted Windows](https://md.research.elamur.ai/free/en/papers/liu-2021-swin-transformer-hierarchical-vision-transformer/) [IEEE/CVF International Conference on Computer Vision (ICCV)](https://md.research.elamur.ai/free/en/venues/ieee-cvf-international-conference-on-computer-vision-iccv/)**Ze Liu, Yutong Lin, Yue Cao, et al.*2021 · 33,968 citations*
2. [**PaLM: Scaling Language Modeling with Pathways arXiv:2204.02311**Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, et al.*2022 · 4,753 citations*](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
3. [**On the Opportunities and Risks of Foundation Models arXiv:2108.07258**Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, et al.*2021 · 3,160 citations*](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/)
4. [**RoFormer: Enhanced Transformer with Rotary Position Embedding arXiv:2104.09864**Jianlin Su, Yu Lu, Shengfeng Pan, et al.*2021 · 3,028 citations*](https://md.research.elamur.ai/free/en/papers/su-2021-roformer-enhanced-transformer-rotary-position/)
5. **[Efficient Transformers: A Survey](https://md.research.elamur.ai/free/en/papers/tay-2020-efficient-transformers-survey/) [ACM Computing Surveys](https://md.research.elamur.ai/free/en/venues/acm-computing-surveys/)**Yi Tay, Mostafa Dehghani, Dara Bahri, et al.*2020 · 1,539 citations*
6. [**FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning arXiv:2307.08691**Tri Dao*2023 · 1,240 citations*](https://md.research.elamur.ai/free/en/papers/dao-2023-flashattention-2-faster-attention-better/)
7. **[FNet: Mixing Tokens with Fourier Transforms](https://md.research.elamur.ai/free/en/papers/thorp-2022-fnet-mixing-tokens-fourier-transforms/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**James Lee-Thorp, Joshua Ainslie, Ilya Eckstein, et al.*2022 · 597 citations*
8. **[AudioLM: a Language Modeling Approach to Audio Generation](https://md.research.elamur.ai/free/en/papers/borsos-2022-audiolm-language-modeling-approach-audio/) [IEEE/ACM Transactions on Audio Speech and Language Processing](https://md.research.elamur.ai/free/en/venues/ieee-acm-transactions-on-audio-speech-and-language-processing/)**Zalan Borsos, Raphael Marinier, Damien Vincent, et al.*2022 · 581 citations*
9. [**Long Range Arena: A Benchmark for Efficient Transformers arXiv:2011.04006**Yi Tay, Mostafa Dehghani, Samira Abnar, et al.*2020 · 414 citations*](https://md.research.elamur.ai/free/en/papers/tay-2020-long-range-arena-benchmark-efficient/)
10. **[FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision](https://md.research.elamur.ai/free/en/papers/shah-2024-flashattention-3-fast-accurate-attention/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Jay Shah, Ganesh Bikshandi, Ying Zhang, et al.*2024 · 400 citations*
11. **[LongT5: Efficient Text-To-Text Transformer for Long Sequences](https://md.research.elamur.ai/free/en/papers/guo-2022-longt5-efficient-text-text-transformer/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Mandy Guo, Joshua Ainslie, David Uthus, et al.*2022 · 399 citations*
12. **[Efficient Attentions for Long Document Summarization](https://md.research.elamur.ai/free/en/papers/huang-2021-efficient-attentions-long-document-summarization/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Luyang Huang, Shuyang Cao, Nikolaus Parulian, et al.*2021 · 336 citations*
13. [**Socratic Models: Composing Zero-Shot Multimodal Reasoning with Language arXiv:2204.00598**Andy Zeng, Maria Attarian, Brian Ichter, et al.*2022 · 302 citations*](https://md.research.elamur.ai/free/en/papers/zeng-2022-socratic-models-composing-zero-shot/)
14. [**Challenges and Applications of Large Language Models arXiv:2307.10169**Jean Kaddour, Joshua Harris, Maximilian Mozes, et al.*2023 · 301 citations*](https://md.research.elamur.ai/free/en/papers/kaddour-2023-challenges-applications-large-language-models/)
15. **[Canine: Pre-training an Efficient Tokenization-Free Encoder for Language Representation](https://md.research.elamur.ai/free/en/papers/clark-2022-canine-pre-training-efficient-tokenization/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Jonathan H. Clark, Dan Garrette, Iulia Turc, et al.*2022 · 301 citations*
16. [**Perceiver: General Perception with Iterative Attention arXiv:2103.03206**Andrew Jaegle, Felix Gimeno, Andrew Brock, et al.*2021 · 269 citations*](https://md.research.elamur.ai/free/en/papers/jaegle-2021-perceiver-general-perception-iterative-attention/)
17. [**Transformers learn in-context by gradient descent arXiv:2212.07677**Johannes von Oswald, Eyvind Niklasson, Ettore Randazzo, et al.*2022 · 247 citations*](https://md.research.elamur.ai/free/en/papers/oswald-2022-transformers-learn-context-gradient-descent/)
18. [**Charformer: Fast Character Transformers via Gradient-based Subword Tokenization arXiv:2106.12672**Yi Tay, Vinh Q. Tran, Sebastian Ruder, et al.*2021 · 186 citations*](https://md.research.elamur.ai/free/en/papers/tay-2021-charformer-fast-character-transformers-gradient/)
19. **[Adapting Language Models to Compress Contexts](https://md.research.elamur.ai/free/en/papers/chevalier-2023-adapting-language-models-compress-contexts/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Alexis Chevalier, Alexander Wettig, Anirudh Ajith, et al.*2023 · 182 citations*
20. **[SCROLLS: Standardized CompaRison Over Long Language Sequences](https://md.research.elamur.ai/free/en/papers/shaham-2022-scrolls-standardized-comparison-over-long/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Uri Shaham, Elad Segal, Maor Ivgi, et al.*2022 · 165 citations*
21. **[Efficient Methods for Natural Language Processing: A Survey](https://md.research.elamur.ai/free/en/papers/treviso-2023-efficient-methods-natural-language-processing/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Marcos Treviso, Ji-Ung Lee, Tianchu Ji, et al.*2023 · 145 citations*
22. **[A Length-Extrapolatable Transformer](https://md.research.elamur.ai/free/en/papers/sun-2023-length-extrapolatable-transformer/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Yutao Sun, Li Dong, Barun Patra, et al.*2023 · 118 citations*
23. **[Do Long-Range Language Models Actually Use Long-Range Context?](https://md.research.elamur.ai/free/en/papers/sun-2021-do-long-range-language-models/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Simeng Sun, Kalpesh Krishna, Andrew Mattarella-Micke, et al.*2021 · 101 citations*
24. **[How to Train Long-Context Language Models (Effectively)](https://md.research.elamur.ai/free/en/papers/gao-2025-how-train-long-context-language/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Tianyu Gao, Alexander Wettig, Howard Yen, et al.*2025 · 89 citations*
25. **[Scaling Laws vs Model Architectures: How does Inductive Bias Influence Scaling?](https://md.research.elamur.ai/free/en/papers/tay-2023-scaling-laws-vs-model-architectures/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Yi Tay, Mostafa Dehghani, Samira Abnar, et al.*2023 · 86 citations*
26. **[MoBA: Mixture of Block Attention for Long-Context LLMs](https://md.research.elamur.ai/free/en/papers/lu-2025-moba-mixture-block-attention/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Enzhe Lu, Zhejun Jiang, Jingyuan Liu, et al.*2025 · 80 citations*
27. **[Domain Adaptation with Pre-trained Transformers for Query-Focused Abstractive Text Summarization](https://md.research.elamur.ai/free/en/papers/laskar-2022-domain-adaptation-pre-trained-transformers/) [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/)**Md Tahmid Rahman Laskar, Enamul Hoque, Jimmy Xiangji Huang*2022 · 76 citations*
28. **[The Devil in Linear Transformer](https://md.research.elamur.ai/free/en/papers/qin-2022-devil-linear-transformer/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Zhen Qin, Xiaodong Han, Weixuan Sun, et al.*2022 · 67 citations*
29. [**MiniMax-M1: Scaling Test-Time Compute Efficiently with Lightning Attention arXiv:2506.13585**MiniMax, Aili Chen, Aonian Li, et al.*2025 · 66 citations*](https://md.research.elamur.ai/free/en/papers/minimax-2025-minimax-m1-scaling-test/)
30. [**A Comprehensive Survey on Long Context Language Modeling arXiv:2503.17407**Jiaheng Liu, Dawei Zhu, Zhiqi Bai, et al.*2025 · 64 citations*](https://md.research.elamur.ai/free/en/papers/liu-2025-comprehensive-survey-long-context-language/)
31. [**Beyond the Limits: A Survey of Techniques to Extend the Context Length in Large Language Models arXiv:2402.02244**Xindi Wang, Mahsa Salmani, Parsa Omidi, et al.*2024 · 60 citations*](https://md.research.elamur.ai/free/en/papers/wang-2024-limits-survey-techniques-extend-context/)
32. [**Simple linear attention language models balance the recall-throughput tradeoff arXiv:2402.18668**Simran Arora, Sabri Eyuboglu, Michael Zhang, et al.*2024 · 59 citations*](https://md.research.elamur.ai/free/en/papers/arora-2024-simple-linear-attention-language-models/)
33. [**Scaling Laws of RoPE-based Extrapolation arXiv:2310.05209**Xiaoran Liu, Hang Yan, Shuo Zhang, et al.*2023 · 54 citations*](https://md.research.elamur.ai/free/en/papers/liu-2023-scaling-laws-rope-based/)
34. **[Long-Context Language Modeling with Parallel Context Encoding](https://md.research.elamur.ai/free/en/papers/yen-2024-long-context-language-modeling-parallel/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Howard Yen, Tianyu Gao, Danqi Chen*2024 · 44 citations*
35. [**TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer arXiv:2307.14995**Zhen Qin, Dong Li, Weigao Sun, et al.*2023 · 33 citations*](https://md.research.elamur.ai/free/en/papers/qin-2023-transnormerllm-faster-better-large/)
36. **[TurboRAG: Accelerating Retrieval-Augmented Generation with Precomputed KV Caches for Chunked Text](https://md.research.elamur.ai/free/en/papers/lu-2025-turborag-accelerating-retrieval-augmented-generation/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Songshuo Lu, Hua Wang, Yutian Rong, et al.*2025 · 32 citations*
37. [**Lightning Attention-2: A Free Lunch for Handling Unlimited Sequence Lengths in Large Language Models arXiv:2401.04658**Zhen Qin, Weigao Sun, Dong Li, et al.*2024 · 31 citations*](https://md.research.elamur.ai/free/en/papers/qin-2024-lightning-attention-2-free/)
38. **[Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language Processing](https://md.research.elamur.ai/free/en/papers/ghaddar-2022-revisiting-pre-trained-language-models/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Abbas Ghaddar, Yimeng Wu, Sunyam Bagga, et al.*2022 · 27 citations*
39. [**Block-State Transformers arXiv:2306.09539**Mahan Fathi, Jonathan Pilault, Orhan Firat, et al.*2023 · 20 citations*](https://md.research.elamur.ai/free/en/papers/fathi-2023-block-state-transformers/)
40. [**Linearizing Large Language Models arXiv:2405.06640**Jean Mercat, Igor Vasiljevic, Sedrick Keh, et al.*2024 · 16 citations*](https://md.research.elamur.ai/free/en/papers/mercat-2024-linearizing-large-language-models/)
41. **[Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2025-attention-entropy-is-key-factor/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Zhisong Zhang, Yan Wang, Xinting Huang, et al.*2025 · 14 citations*
42. [**PanGu-π: Enhancing Language Model Architectures via Nonlinearity Compensation arXiv:2312.17276**Yunhe Wang, Hanting Chen, Yehui Tang, et al.*2023 · 10 citations*](https://md.research.elamur.ai/free/en/papers/wang-2023-pangu-pi-enhancing-language/)
43. [**Revisiting Pre-trained Language Models and their Evaluation for Arabic Natural Language Understanding arXiv:2205.10687**Abbas Ghaddar, Yimeng Wu, Sunyam Bagga, et al.*2022 · 9 citations*](https://md.research.elamur.ai/free/en/papers/ghaddar-2022-revisiting-pre-trained-language-models-their/)
44. [**Just read twice: closing the recall gap for recurrent language models arXiv:2407.05483**Simran Arora; Aman Timalsina; Aaryan Singhal; Benjamin Spector; Sabri Eyuboglu; Xinyi Zhao; Ashish Rao; Atri Rudra;, Christopher Re*2024 · 8 citations*](https://md.research.elamur.ai/free/en/papers/arora-2024-just-read-twice-closing-recall/)
45. [**An End-to-End OCR Framework for Robust Arabic-Handwriting Recognition using a Novel Transformers-based Model and an Innovative 270 Million-Words Multi-Font Corpus of Classical Arabic with Diacritics arXiv:2208.11484**Aly Mostafa, Omar Mohamed, Ali Ashraf, et al.*2022 · 3 citations*](https://md.research.elamur.ai/free/en/papers/mostafa-2022-end-end-ocr-framework-robust/)
46. **[Cross-layer Attention Sharing for Pre-trained Large Language Models](https://md.research.elamur.ai/free/en/papers/mu-2026-cross-layer-attention-sharing-pre/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Yongyu Mu, Yuzhang Wu, Yuchun Fan, et al.*2026 · 2 citations*
47. [**MiniMax Sparse Attention arXiv:2606.13392**Xunhao Lai, Weiqi Xu, Yufeng Yang, et al.*2026 · 2 citations*](https://md.research.elamur.ai/free/en/papers/lai-2026-minimax-sparse-attention/)
48. **[From Logical to Computational Sparsity: Structure-Aware Block-Sparse Attention for Long-Code Completion](https://md.research.elamur.ai/free/en/papers/wang-2026-logical-computational-sparsity-structure-aware/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Yanli Wang, Yanlin Wang, Bowen Zhang, et al.*2026*
49. **[SemToken: Semantic-Aware Tokenization for Efficient Long-Context Language Models](https://md.research.elamur.ai/free/en/papers/liu-2026-semtoken-semantic-aware-tokenization-efficient/) [Joint Conference on Lexical and Computational Semantics (*SEM)](https://md.research.elamur.ai/free/en/venues/starsem/)**Dong Liu, Yanxuan Yu*2026*

338 more papers of our library cite it; they have no page here yet.

- [← back to](https://md.research.elamur.ai/free/en/papers/choromanski-2020-rethinking-attention-performers/) — Rethinking Attention with Performers