---
title: "Who cites Generating Long Sequences — 543 citing papers"
description: "543 papers of the Elamur Research library cite «Generating Long Sequences with Sparse Transformers», found in their bibliographies; the…"
canonical: "https://research.elamur.ai/free/en/papers/child-2019-generating-long-sequences-sparse-transformers/cited-by/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/child-2019-generating-long-sequences-sparse-transformers/cited-by/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [Generating Long Sequences with Sparse Transformers](https://md.research.elamur.ai/free/en/papers/child-2019-generating-long-sequences-sparse-transformers/) / cited by

# Who cites «Generating Long Sequences with Sparse Transformers»

543 papers of our library cite it (of the 149,439 whose bibliographies we read) · 86 with a page here, the most cited first · [← back to «Generating Long Sequences with Sparse Transformers»](https://md.research.elamur.ai/free/en/papers/child-2019-generating-long-sequences-sparse-transformers/)

1. [**An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale arXiv:2010.11929**Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, et al.*2020 · 23,726 citations*](https://md.research.elamur.ai/free/en/papers/dosovitskiy-2020-image-worth-16x16-words-transformers/)
2. [**Language Models are Few-Shot Learners arXiv:2005.14165**Tom B. Brown, Benjamin Mann, Nick Ryder, et al.*2020 · 14,178 citations*](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
3. [**PaLM: Scaling Language Modeling with Pathways arXiv:2204.02311**Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, et al.*2022 · 4,753 citations*](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
4. [**Longformer: The Long-Document Transformer arXiv:2004.05150**Iz Beltagy, Matthew E. Peters, Arman Cohan*2020 · 3,880 citations*](https://md.research.elamur.ai/free/en/papers/beltagy-2020-longformer-long-document-transformer/)
5. [**Scaling Laws for Neural Language Models arXiv:2001.08361**Jared Kaplan, Sam McCandlish, Tom Henighan, et al.*2020 · 3,434 citations*](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/)
6. [**On the Opportunities and Risks of Foundation Models arXiv:2108.07258**Rishi Bommasani, Drew A. Hudson, Ehsan Adeli, et al.*2021 · 3,160 citations*](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/)
7. **[A Survey of Large Language Models](https://md.research.elamur.ai/free/en/papers/zhao-2023-survey-large-language-models/) [Frontiers of Computer Science](https://md.research.elamur.ai/free/en/venues/frontiers-of-computer-science/)**Wayne Xin Zhao, Kun Zhou, Junyi Li, et al.*2023 · 2,875 citations*
8. [**ALBERT: A Lite BERT for Self-supervised Learning of Language Representations arXiv:1909.11942**Zhenzhong Lan, Mingda Chen, Sebastian Goodman, et al.*2019 · 2,806 citations*](https://md.research.elamur.ai/free/en/papers/lan-2019-albert-lite-bert-self-supervised/)
9. **[Scalable Diffusion Models with Transformers](https://md.research.elamur.ai/free/en/papers/peebles-2022-scalable-diffusion-models-transformers/) [IEEE/CVF International Conference on Computer Vision (ICCV)](https://md.research.elamur.ai/free/en/venues/ieee-cvf-international-conference-on-computer-vision-iccv/)**William Peebles, Saining Xie*2022 · 2,670 citations*
10. [**Evaluating Large Language Models Trained on Code arXiv:2107.03374**Mark Chen, Jerry Tworek, Heewoo Jun, et al.*2021 · 2,659 citations*](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
11. [**Robust Speech Recognition via Large-Scale Weak Supervision arXiv:2212.04356**Alec Radford, Jong Wook Kim, Tao Xu, et al.*2022 · 2,546 citations*](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/)
12. [**DeBERTa: Decoding-enhanced BERT with Disentangled Attention arXiv:2006.03654**Pengcheng He, Xiaodong Liu, Jianfeng Gao, et al.*2020 · 1,828 citations*](https://md.research.elamur.ai/free/en/papers/he-2020-deberta-decoding-enhanced-bert-disentangled/)
13. [**Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models arXiv:2206.04615 · Transactions on Machine Learning Research, May/2022, https://openreview.net/forum?id=uyTL5Bvosj**Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, et al.*2022 · 1,756 citations*](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/)
14. **[Efficient Transformers: A Survey](https://md.research.elamur.ai/free/en/papers/tay-2020-efficient-transformers-survey/) [ACM Computing Surveys](https://md.research.elamur.ai/free/en/venues/acm-computing-surveys/)**Yi Tay, Mostafa Dehghani, Dara Bahri, et al.*2020 · 1,539 citations*
15. [**Zero-Shot Text-to-Image Generation arXiv:2102.12092**Aditya Ramesh, Mikhail Pavlov, Gabriel Goh, et al.*2021 · 1,528 citations*](https://md.research.elamur.ai/free/en/papers/ramesh-2021-zero-shot-text-image-generation/)
16. [**CTRL: A Conditional Transformer Language Model for Controllable Generation arXiv:1909.05858**Nitish Shirish Keskar, Bryan McCann, Lav R. Varshney, et al.*2019 · 1,291 citations*](https://md.research.elamur.ai/free/en/papers/keskar-2019-ctrl-conditional-transformer-language-model/)
17. **[A Comprehensive Overview of Large Language Models](https://md.research.elamur.ai/free/en/papers/naveed-2023-comprehensive-overview-large-language-models/) [ACM Transactions on Intelligent Systems and Technology](https://md.research.elamur.ai/free/en/venues/acm-transactions-on-intelligent-systems-and-technology/)**Humza Naveed, Asad Ullah Khan, Shi Qiu, et al.*2023 · 1,287 citations*
18. **[Big Bird: Transformers for Longer Sequences](https://md.research.elamur.ai/free/en/papers/zaheer-2020-big-bird-transformers-longer-sequences/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Manzil Zaheer, Guru Guruganesh, Avinava Dubey, et al.*2020 · 1,023 citations*
19. **[GPT-NeoX-20B: An Open-Source Autoregressive Language Model](https://md.research.elamur.ai/free/en/papers/black-2022-gpt-neox-20b-open-source/) [BigScience Episode – Workshop on Challenges & Perspectives in Creating Large Language Models](https://md.research.elamur.ai/free/en/venues/bigscience/)**Sidney Black, Stella Biderman, Eric Hallahan, et al.*2022 · 831 citations*
20. [**Reformer: The Efficient Transformer arXiv:2001.04451**Nikita Kitaev, Łukasz Kaiser, Anselm Levskaya*2020 · 829 citations*](https://md.research.elamur.ai/free/en/papers/kitaev-2020-reformer-efficient-transformer/)
21. **[Can Large Language Models Transform Computational Social Science?](https://md.research.elamur.ai/free/en/papers/ziems-2023-large-language-models-transform-computational/) [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/)**Caleb Ziems, William Held, Omar Shaikh, et al.*2023 · 790 citations*
22. **[Efficient Content-Based Sparse Attention with Routing Transformers](https://md.research.elamur.ai/free/en/papers/roy-2020-efficient-content-based-sparse-attention/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Aurko Roy, Mohammad Saffar, Ashish Vaswani, et al.*2020 · 677 citations*
23. [**gpt-oss-120b & gpt-oss-20b Model Card arXiv:2508.10925**OpenAI: Sandhini Agarwal, Lama Ahmad, Jason Ai, et al.*2025 · 576 citations*](https://md.research.elamur.ai/free/en/papers/agarwal-2025-gpt-oss-120b-gpt-oss/)
24. [**LongBench: A Bilingual, Multitask Benchmark for Long Context Understanding arXiv:2308.14508**Yushi Bai, Xin Lv, Jiajie Zhang, et al.*2023 · 514 citations*](https://md.research.elamur.ai/free/en/papers/bai-2023-longbench-bilingual-multitask-benchmark/)
25. [**Rethinking Attention with Performers arXiv:2009.14794**Krzysztof Choromanski, Valerii Likhosherstov, David Dohan, et al.*2020 · 491 citations*](https://md.research.elamur.ai/free/en/papers/choromanski-2020-rethinking-attention-performers/)
26. **[BERT for Coreference Resolution: Baselines and Analysis](https://md.research.elamur.ai/free/en/papers/joshi-2019-bert-coreference-resolution-baselines-analysis/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Mandar Joshi, Omer Levy, Luke Zettlemoyer, et al.*2019 · 490 citations*
27. [**Analyzing the Structure of Attention in a Transformer Language Model arXiv:1906.04284**Jesse Vig, Yonatan Belinkov*2019 · 438 citations*](https://md.research.elamur.ai/free/en/papers/vig-2019-analyzing-structure-attention-transformer-language/)
28. [**Long Range Arena: A Benchmark for Efficient Transformers arXiv:2011.04006**Yi Tay, Mostafa Dehghani, Samira Abnar, et al.*2020 · 414 citations*](https://md.research.elamur.ai/free/en/papers/tay-2020-long-range-arena-benchmark-efficient/)
29. **[FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision](https://md.research.elamur.ai/free/en/papers/shah-2024-flashattention-3-fast-accurate-attention/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Jay Shah, Ganesh Bikshandi, Ying Zhang, et al.*2024 · 400 citations*
30. [**RULER: What's the Real Context Size of Your Long-Context Language Models? arXiv:2404.06654**Cheng-Ping Hsieh, Simeng Sun, Samuel Kriman, et al.*2024 · 400 citations*](https://md.research.elamur.ai/free/en/papers/hsieh-2024-ruler-s-real-context-size/)
31. [**A Simple Language Model for Task-Oriented Dialogue arXiv:2005.00796**Ehsan Hosseini-Asl, Bryan McCann, Chien-Sheng Wu, et al.*2020 · 399 citations*](https://md.research.elamur.ai/free/en/papers/hosseini-asl-2020-simple-language-model-task-oriented/)
32. **[ETC: Encoding Long and Structured Inputs in Transformers](https://md.research.elamur.ai/free/en/papers/ainslie-2020-etc-encoding-long-structured-inputs/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Joshua Ainslie, Santiago Ontañón, Chris Alberti, et al.*2020 · 386 citations*
33. [**Compressive Transformers for Long-Range Sequence Modelling arXiv:1911.05507**Jack W. Rae, Anna Potapenko, Siddhant M. Jayakumar, et al.*2019 · 340 citations*](https://md.research.elamur.ai/free/en/papers/rae-2019-compressive-transformers-long-range-sequence/)
34. **[Efficient Attentions for Long Document Summarization](https://md.research.elamur.ai/free/en/papers/huang-2021-efficient-attentions-long-document-summarization/) [Conference of the North American Chapter of the Association for Computational Linguistics (NAACL)](https://md.research.elamur.ai/free/en/venues/naacl/)**Luyang Huang, Shuyang Cao, Nikolaus Parulian, et al.*2021 · 336 citations*
35. [**Structured Denoising Diffusion Models in Discrete State-Spaces arXiv:2107.03006**Jacob Austin, Daniel D. Johnson, Jonathan Ho, et al.*2021 · 284 citations*](https://md.research.elamur.ai/free/en/papers/austin-2021-structured-denoising-diffusion-models-discrete/)
36. [**Scaling Laws for Autoregressive Generative Modeling arXiv:2010.14701**Tom Henighan, Jared Kaplan, Mor Katz, et al.*2020 · 277 citations*](https://md.research.elamur.ai/free/en/papers/henighan-2020-scaling-laws-autoregressive-generative-modeling/)
37. **[Adaptively Sparse Transformers](https://md.research.elamur.ai/free/en/papers/correia-2019-adaptively-sparse-transformers/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Gonçalo M. Correia, Vlad Niculae, André F. T. Martins*2019 · 275 citations*
38. [**Perceiver: General Perception with Iterative Attention arXiv:2103.03206**Andrew Jaegle, Felix Gimeno, Andrew Brock, et al.*2021 · 269 citations*](https://md.research.elamur.ai/free/en/papers/jaegle-2021-perceiver-general-perception-iterative-attention/)
39. [**LongLoRA: Efficient Fine-tuning of Long-Context Large Language Models arXiv:2309.12307**Yukang Chen, Shengju Qian, Haotian Tang, et al.*2023 · 268 citations*](https://md.research.elamur.ai/free/en/papers/chen-2023-longlora-efficient-fine-tuning-long/)
40. [**DeepSpeed-MoE: Advancing Mixture-of-Experts Inference and Training to Power Next-Generation AI Scale arXiv:2201.05596**Samyam Rajbhandari, Conglong Li, Zhewei Yao, et al.*2022 · 261 citations*](https://md.research.elamur.ai/free/en/papers/rajbhandari-2022-deepspeed-moe-advancing-mixture-experts/)
41. [**TrustLLM: Trustworthiness in Large Language Models arXiv:2401.05561**Yue Huang, Lichao Sun, Haoran Wang, et al.*2024 · 237 citations*](https://md.research.elamur.ai/free/en/papers/huang-2024-trustllm-trustworthiness-large-language-models/)
42. **[Compressing Context to Enhance Inference Efficiency of Large Language Models](https://md.research.elamur.ai/free/en/papers/li-2023-compressing-context-enhance-inference-efficiency/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Yucheng Li, Bo Dong, Frank Guerin, et al.*2023 · 212 citations*
43. [**mGPT: Few-Shot Learners Go Multilingual arXiv:2204.07580**Oleh Shliazhko, Alena Fenogenova, Maria Tikhonova, et al.*2022 · 203 citations*](https://md.research.elamur.ai/free/en/papers/shliazhko-2022-mgpt-few-shot-learners-go/)
44. **[MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention](https://md.research.elamur.ai/free/en/papers/jiang-2024-minference-1-0-accelerating-pre/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Huiqiang Jiang, Yucheng Li, Chengruidong Zhang, et al.*2024 · 202 citations*
45. [**Model Tells You What to Discard: Adaptive KV Cache Compression for LLMs arXiv:2310.01801**Suyu Ge, Yunan Zhang, Liyuan Liu, et al.*2023 · 202 citations*](https://md.research.elamur.ai/free/en/papers/ge-2023-model-tells-discard-adaptive-kv/)
46. [**Effective Long-Context Scaling of Foundation Models arXiv:2309.16039**Wenhan Xiong, Jingyu Liu, Igor Molybog, et al.*2023 · 184 citations*](https://md.research.elamur.ai/free/en/papers/xiong-2023-effective-long-context-scaling-foundation/)
47. **[Adapting Language Models to Compress Contexts](https://md.research.elamur.ai/free/en/papers/chevalier-2023-adapting-language-models-compress-contexts/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Alexis Chevalier, Alexander Wettig, Anirudh Ajith, et al.*2023 · 182 citations*
48. **[Learning to Compress Prompts with Gist Tokens](https://md.research.elamur.ai/free/en/papers/mu-2023-learning-compress-prompts-gist-tokens/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Jesse Mu, Xiang Lisa Li, Noah Goodman*2023 · 178 citations*
49. [**Generative Representational Instruction Tuning arXiv:2402.09906**Niklas Muennighoff, Hongjin Su, Liang Wang, et al.*2024 · 172 citations*](https://md.research.elamur.ai/free/en/papers/muennighoff-2024-generative-representational-instruction-tuning/)
50. [**DeepSpeed Ulysses: System Optimizations for Enabling Training of Extreme Long Sequence Transformer Models arXiv:2309.14509**Sam Ade Jacobs, Masahiro Tanaka, Chengming Zhang, et al.*2023 · 162 citations*](https://md.research.elamur.ai/free/en/papers/jacobs-2023-deepspeed-ulysses-system-optimizations-enabling/)
51. **[Efficient Methods for Natural Language Processing: A Survey](https://md.research.elamur.ai/free/en/papers/treviso-2023-efficient-methods-natural-language-processing/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Marcos Treviso, Ji-Ung Lee, Tianchu Ji, et al.*2023 · 145 citations*
52. [**Retrieval meets Long Context Large Language Models arXiv:2310.03025**Peng Xu, Wei Ping, Xianchao Wu, et al.*2023 · 130 citations*](https://md.research.elamur.ai/free/en/papers/xu-2023-retrieval-meets-long-context-large/)
53. **[InfLLM: Training-Free Long-Context Extrapolation for LLMs with an Efficient Context Memory](https://md.research.elamur.ai/free/en/papers/xiao-2024-infllm-training-free-long-context/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Chaojun Xiao, Pengle Zhang, Xu Han, et al.*2024 · 126 citations*
54. **[A Length-Extrapolatable Transformer](https://md.research.elamur.ai/free/en/papers/sun-2023-length-extrapolatable-transformer/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Yutao Sun, Li Dong, Barun Patra, et al.*2023 · 118 citations*
55. [**DuoAttention: Efficient Long-Context LLM Inference with Retrieval and Streaming Heads arXiv:2410.10819**Guangxuan Xiao, Jiaming Tang, Jingwei Zuo, et al.*2024 · 114 citations*](https://md.research.elamur.ai/free/en/papers/xiao-2024-duoattention-efficient-long-context-llm/)
56. **[Rethinking Document-level Neural Machine Translation](https://md.research.elamur.ai/free/en/papers/sun-2022-rethinking-document-level-neural-machine/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Zewei Sun, Mingxuan Wang, Hao Zhou, et al.*2022 · 107 citations*
57. **[Do Long-Range Language Models Actually Use Long-Range Context?](https://md.research.elamur.ai/free/en/papers/sun-2021-do-long-range-language-models/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Simeng Sun, Kalpesh Krishna, Andrew Mattarella-Micke, et al.*2021 · 101 citations*
58. **[CoLT5: Faster Long-Range Transformers with Conditional Computation](https://md.research.elamur.ai/free/en/papers/ainslie-2023-colt5-faster-long-range-transformers/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Joshua Ainslie, Tao Lei, Michiel de Jong, et al.*2023 · 84 citations*
59. **[MoBA: Mixture of Block Attention for Long-Context LLMs](https://md.research.elamur.ai/free/en/papers/lu-2025-moba-mixture-block-attention/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Enzhe Lu, Zhejun Jiang, Jingyuan Liu, et al.*2025 · 80 citations*
60. **[What Context Features Can Transformer Language Models Use?](https://md.research.elamur.ai/free/en/papers/connor-2021-what-context-features-can-transformer/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Joe O'Connor, Jacob Andreas*2021 · 77 citations*
61. **[Conditional Generation with a Question-Answering Blueprint](https://md.research.elamur.ai/free/en/papers/narayan-2023-conditional-generation-question-answering-blueprint/) [Transactions of the Association for Computational Linguistics (TACL)](https://md.research.elamur.ai/free/en/venues/tacl/)**Shashi Narayan, Joshua Maynez, Reinald Kim Amplayo, et al.*2023 · 70 citations*
62. **[The Devil in Linear Transformer](https://md.research.elamur.ai/free/en/papers/qin-2022-devil-linear-transformer/) [Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/)**Zhen Qin, Xiaodong Han, Weixuan Sun, et al.*2022 · 67 citations*
63. [**A Comprehensive Survey on Long Context Language Modeling arXiv:2503.17407**Jiaheng Liu, Dawei Zhu, Zhiqi Bai, et al.*2025 · 64 citations*](https://md.research.elamur.ai/free/en/papers/liu-2025-comprehensive-survey-long-context-language/)
64. [**XAttention: Block Sparse Attention with Antidiagonal Scoring arXiv:2503.16428**Ruyi Xu, Guangxuan Xiao, Haofeng Huang, et al.*2025 · 59 citations*](https://md.research.elamur.ai/free/en/papers/xu-2025-xattention-block-sparse-attention-antidiagonal/)
65. **[BAMBOO: A Comprehensive Benchmark for Evaluating Long Text Modeling Capacities of Large Language Models](https://md.research.elamur.ai/free/en/papers/dong-2024-bamboo-comprehensive-benchmark-evaluating-long/) [Language Resources and Evaluation Conference (LREC)](https://md.research.elamur.ai/free/en/venues/lrec/)**Zican Dong, Tianyi Tang, Junyi Li, et al.*2024 · 59 citations*
66. [**Samba: Simple Hybrid State Space Models for Efficient Unlimited Context Language Modeling arXiv:2406.07522**Liliang Ren, Yang Liu, Yadong Lu, et al.*2024 · 59 citations*](https://md.research.elamur.ai/free/en/papers/ren-2024-samba-simple-hybrid-state-space/)
67. [**Simple linear attention language models balance the recall-throughput tradeoff arXiv:2402.18668**Simran Arora, Sabri Eyuboglu, Michael Zhang, et al.*2024 · 59 citations*](https://md.research.elamur.ai/free/en/papers/arora-2024-simple-linear-attention-language-models/)
68. [**RecurrentGPT: Interactive Generation of (Arbitrarily) Long Text arXiv:2305.13304**Wangchunshu Zhou, Yuchen Eleanor Jiang, Peng Cui, et al.*2023 · 57 citations*](https://md.research.elamur.ai/free/en/papers/zhou-2023-recurrentgpt-interactive-generation-arbitrarily-long/)
69. [**Zoology: Measuring and Improving Recall in Efficient Language Models arXiv:2312.04927**Simran Arora; Sabri Eyuboglu; Aman Timalsina; Isys Johnson; Michael Poli; James Zou; Atri Rudra;, Christopher Re*2023 · 55 citations*](https://md.research.elamur.ai/free/en/papers/arora-2023-zoology-measuring-improving-recall-efficient/)
70. [**Incremental Neural Coreference Resolution in Constant Memory**Patrick Xia, João Sedoc, Benjamin Van Durme*2020 · 55 citations*](https://md.research.elamur.ai/free/en/papers/xia-2020-incremental-neural-coreference-resolution-constant/)
71. [**MiniMax-01: Scaling Foundation Models with Lightning Attention arXiv:2501.08313**MiniMax, Aonian Li, Bangwei Gong, et al.*2025 · 50 citations*](https://md.research.elamur.ai/free/en/papers/minimax-2025-minimax-01-scaling-foundation/)
72. [**TransNormerLLM: A Faster and Better Large Language Model with Improved TransNormer arXiv:2307.14995**Zhen Qin, Dong Li, Weigao Sun, et al.*2023 · 33 citations*](https://md.research.elamur.ai/free/en/papers/qin-2023-transnormerllm-faster-better-large/)
73. [**Linear algebra with transformers arXiv:2112.01898**François Charton*2021 · 31 citations*](https://md.research.elamur.ai/free/en/papers/charton-2021-linear-algebra-transformers/)
74. [**MEMORYLLM: Towards Self-Updatable Large Language Models arXiv:2402.04624**Yu Wang, Yifan Gao, Xiusi Chen, et al.*2024 · 27 citations*](https://md.research.elamur.ai/free/en/papers/wang-2024-memoryllm-towards-self-updatable-large/)
75. [**ERNIE-Doc: A Retrospective Long-Document Modeling Transformer arXiv:2012.15688**Siyu Ding, Junyuan Shang, Shuohuan Wang, et al.*2020 · 27 citations*](https://md.research.elamur.ai/free/en/papers/ding-2020-ernie-doc-retrospective-long/)
76. [**Block-State Transformers arXiv:2306.09539**Mahan Fathi, Jonathan Pilault, Orhan Firat, et al.*2023 · 20 citations*](https://md.research.elamur.ai/free/en/papers/fathi-2023-block-state-transformers/)
77. **[Attention Entropy is a Key Factor: An Analysis of Parallel Context Encoding with Full-attention-based Pre-trained Language Models](https://md.research.elamur.ai/free/en/papers/zhang-2025-attention-entropy-is-key-factor/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Zhisong Zhang, Yan Wang, Xinting Huang, et al.*2025 · 14 citations*
78. [**PanGu-π: Enhancing Language Model Architectures via Nonlinearity Compensation arXiv:2312.17276**Yunhe Wang, Hanting Chen, Yehui Tang, et al.*2023 · 10 citations*](https://md.research.elamur.ai/free/en/papers/wang-2023-pangu-pi-enhancing-language/)
79. [**GRIN: GRadient-INformed MoE arXiv:2409.12136**Liyuan Liu, Young Jin Kim, Shuohang Wang, et al.*2024 · 7 citations*](https://md.research.elamur.ai/free/en/papers/liu-2024-grin-gradient-informed-moe/)
80. **[DELTA: Dynamic Layer-Aware Token Attention for Efficient Long-Context Reasoning](https://md.research.elamur.ai/free/en/papers/zarch-2026-delta-dynamic-layer-aware-token/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Hossein Entezari Zarch, Lei Gao, Chaoyi Jiang, et al.*2026 · 1 citation*
81. **[AdapShot: Adaptive Many-Shot In-Context Learning with Semantic-Aware KV Cache Reuse](https://md.research.elamur.ai/free/en/papers/ou-2026-adapshot-adaptive-many-shot-context/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Jie Ou, Jinyu Guo, Shiyao Guo, et al.*2026*
82. **[From Logical to Computational Sparsity: Structure-Aware Block-Sparse Attention for Long-Code Completion](https://md.research.elamur.ai/free/en/papers/wang-2026-logical-computational-sparsity-structure-aware/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Yanli Wang, Yanlin Wang, Bowen Zhang, et al.*2026*
83. **[Less is More: Knowledge-Aware Compression for Long Legal Judgment Prediction](https://md.research.elamur.ai/free/en/papers/lou-2026-less-is-more-knowledge-aware/) [Findings of the Association for Computational Linguistics](https://md.research.elamur.ai/free/en/venues/findings/)**Fanghao Lou, Qiqi Wang, Guanyu Chen, et al.*2026*
84. **[MATCH: Modulating Attention via In-Context Retrieval for Long-Context Transformers](https://md.research.elamur.ai/free/en/papers/ma-2026-match-modulating-attention-context-retrieval/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Linrui Ma, Chun Hei Lo, Xinyu Wang, et al.*2026*
85. **[RoBSA: RoPE-based Blockwise Sparse Multi-head Latent Attention](https://md.research.elamur.ai/free/en/papers/shi-2026-robsa-rope-based-blockwise-sparse/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Xinyu Shi, Kairong Luo, Zhen Zheng, et al.*2026*
86. **[SCOPE: Boosting LLM Efficiency with Scoped Position Encoding](https://md.research.elamur.ai/free/en/papers/qi-2026-scope-boosting-llm-efficiency-scoped/) [Annual Meeting of the Association for Computational Linguistics (ACL)](https://md.research.elamur.ai/free/en/venues/acl/)**Qingguo Qi, Hongyang Chen, Zhao Li*2026*

457 more papers of our library cite it; they have no page here yet.

- [← back to](https://md.research.elamur.ai/free/en/papers/child-2019-generating-long-sequences-sparse-transformers/) — Generating Long Sequences with Sparse Transformers