---
title: "Language Models Use Trigonometry to Do Addition (2025)"
description: "Large language models handle arithmetic reasonably well, but it remains unclear what happens inside… 38 references, cited by 14 papers here."
authors: ["Subhash Kantamneni", "Max Tegmark"]
year: 2025
venue: "arXiv:2502.00873"
source: "https://arxiv.org/abs/2502.00873"
doi: "10.48550/arxiv.2502.00873"
arxiv: "2502.00873"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/kantamneni-2025-language-models-use-trigonometry-addition/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/kantamneni-2025-language-models-use-trigonometry-addition/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/) / Language Models Use Trigonometry to Do Addition

# Language Models Use Trigonometry to Do Addition

**Authors:** Subhash Kantamneni and [Max Tegmark](https://md.research.elamur.ai/free/en/authors/max-tegmark/) · **Year:** 2025 · **Venue:** arXiv:2502.00873 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2502.00873) · [PDF](https://arxiv.org/pdf/2502.00873) · [DOI 10.48550/arxiv.2502.00873](https://doi.org/10.48550/arxiv.2502.00873) · [arXiv 2502.00873](https://arxiv.org/abs/2502.00873) · [OpenAlex](https://openalex.org/W4407124196)

**Citations:** 5 citations (OpenAlex, 2026-10-14) · [14](https://md.research.elamur.ai/free/en/papers/kantamneni-2025-language-models-use-trigonometry-addition/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Large language models handle arithmetic reasonably well, but it remains unclear what happens inside them when they add two numbers. Kantamneni and Tegmark reverse engineer integer addition in three mid-sized models: GPT-J, Pythia-6.9B and Llama3.1-8B. They use a dataset of all 10,000 problems a + b with operands from 0 to 99, so every answer fits in a single token. Fourier analysis and PCA of hidden states show that each number is encoded by a linear component plus several periodic (circular) components. The authors call this a generalized helix and fit it to the residual stream with regression. Causal interventions then show that the models apply a variant of the "Clock" algorithm. The helices of both operands are combined into a helix for their sum, and later layers decode that helix into output logits. The study traces this process down to MLPs, attention heads and individual neurons. It is presented as the first explanation of an LLM's math skill that links internal number representations to the computation performed on them.

## Key points
- Accuracy on the 10,000 addition problems: GPT-J 80.5%, Pythia-6.9B 77.2%, Llama3.1-8B 98.0%. The main analysis uses GPT-J because its plain (non-gated) MLP makes individual neurons easier to interpret.
- The helix fit uses Fourier periods T = [2, 5, 10, 100] plus one linear direction. Neuron preactivations show these same periods.
- In GPT-J, MLPs 14–18 mostly read the operand helices and build the sum. MLPs 19–27 mostly turn the a + b helix into the answer logits.
- The helix representation also matters causally for subtraction, multiplication, integer division and modular arithmetic.
- Limitations:
  - The exact trigonometric step that produces the sum helix could not be isolated.
  - The sum helix plays a weaker causal role in Llama3.1-8B, possibly because of its gated MLPs.
  - Models that tokenize digit by digit, such as Gemma-2-9B, likely need different algorithms.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 4,145 works

## Cite

Cite as: Subhash Kantamneni and Max Tegmark. 2025. Language Models Use Trigonometry to Do Addition. arXiv:2502.00873.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{kantamneni2025language,
  title = {{Language Models Use Trigonometry to Do Addition}},
  author = {Subhash Kantamneni and Max Tegmark},
  year = {2025},
  doi = {10.48550/arxiv.2502.00873},
  eprint = {2502.00873},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2502.00873}
}
```

- [← previous JSONSchemaBench: A Rigorous Benchmark of Structured Outputs for Language Models](https://md.research.elamur.ai/free/en/papers/geng-2025-jsonschemabench-rigorous-benchmark-structured-outputs/)
- [next → MegaMath: Pushing the Limits of Open Math Corpora](https://md.research.elamur.ai/free/en/papers/zhou-2025-megamath-pushing-limits-open-math/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/)

## References

38 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahn, J., Verma, R., Lou, R., Liu, D., Zhang, R., and Yin, W. [Large language models for mathematical reasoning: Progresses and challenges](https://md.research.elamur.ai/free/en/papers/ahn-2024-large-language-models-mathematical-reasoning/), 2024. URL https://arxiv.org/abs/2402.00157. [↗](https://arxiv.org/abs/2402.00157)
2. Gao, L., la Tour, T. D., Tillman, H., Goh, G., Troll, R., Radford, A., Sutskever, I., Leike, J., and Wu, J. Scaling and evaluating sparse autoencoders, 2024. URL https://arxiv.org/abs/2406.04093. [↗](https://arxiv.org/abs/2406.04093)
3. Biderman, S., Schoelkopf, H., Anthony, Q. G., Bradley, H., O’Brien, K., Hallahan, E., Khan, M. A., Purohit, S., Prashanth, U. S., Raff, E., et al. Pythia: A suite for analyzing large language models across training and scaling. In International Conference on Machine Learning, pp. 2397–2430. PMLR, 20… [find](https://scholar.google.com/scholar?q=Biderman%2C+S%2E%2C+Schoelkopf%2C+H%2E%2C+Anthony%2C+Q%2E+G%2E%2C+Bradley%2C+H%2E%2C+O%E2%80%99Brien%2C+K%2E%2C+Hallahan%2C+E%2E%2C+Khan%2C+M%2E+A%2E%2C+Purohit%2C+S%2E%2C+Prashanth%2C+U%2E+S%2E%2C+Raff%2C+E%2E%2C+et+al%2E+Pythia%3A+A+suite+for+analyzing+large+language+models+a)
4. Glazer, E., Erdil, E., Besiroglu, T., Chicharro, D., Chen, E., Gunning, A., Olsson, C. F., Denain, J.-S., Ho, A., de Oliveira Santos, E., Järviniemi, O., Barnett, M., Sandler, R., Vrzala, M., Sevilla, J., Ren, Q., Pratt, E., Levine, L., Barkley, G., Stewart, N., Grechuk, B., Grechuk, T., Enugandla,… — [FrontierMath: A Benchmark for Evaluating Advanced Mathematical Reasoning in AI](https://md.research.elamur.ai/free/en/papers/glazer-2024-frontiermath-benchmark-evaluating-advanced-mathematical/)
5. Bricken, T., Templeton, A., Batson, J., Chen, B., Jermyn, A., Conerly, T., Turner, N., Anil, C., Denison, C., Askell, A., Lasenby, R., Wu, Y., Kravec, S., Schiefer, N., Maxwell, T., Joseph, N., Hatfield-Dodds, Z., Tamkin, A., Nguyen, K., McLean, B., Burke, J. E., Hume, T., Carter, S., Henighan, T.,… [find](https://scholar.google.com/scholar?q=Bricken%2C+T%2E%2C+Templeton%2C+A%2E%2C+Batson%2C+J%2E%2C+Chen%2C+B%2E%2C+Jermyn%2C+A%2E%2C+Conerly%2C+T%2E%2C+Turner%2C+N%2E%2C+Anil%2C+C%2E%2C+Denison%2C+C%2E%2C+Askell%2C+A%2E%2C+Lasenby%2C+R%2E%2C+Wu%2C+Y%2E%2C+Kravec%2C+S%2E%2C+Schiefer%2C+N%2E%2C+Maxwell%2C+T%2E%2C+Joseph%2C+N%2E%2C+Hatfie)
6. Goldowsky-Dill, N., MacLeod, C., Sato, L., and Arora, A. [Localizing model behavior with path patching](https://md.research.elamur.ai/free/en/papers/goldowsky-dill-2023-localizing-model-behavior-path-patching/), 2023. URL https://arxiv.org/abs/2304.05969. [↗](https://arxiv.org/abs/2304.05969)
7. Grattafiori, A., Dubey, A., Jauhri, A., Pandey, A., Kadian, A., et al. [The llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/), 2024. URL https://arxiv.org/abs/2407.21783. [↗](https://arxiv.org/abs/2407.21783)
8. Elhage, N., Nanda, N., Olsson, C., Henighan, T., Joseph, N., Mann, B., Askell, A., Bai, Y., Chen, A., Conerly, T., DasSarma, N., Drain, D., Ganguli, D., Hatfield-Dodds, Z., Hernandez, D., Jones, A., Kernion, J., Lovitt, L., Ndousse, K., Amodei, D., Brown, T., Clark, J., Kaplan, J., McCandlish, S., a… [find](https://scholar.google.com/scholar?q=Elhage%2C+N%2E%2C+Nanda%2C+N%2E%2C+Olsson%2C+C%2E%2C+Henighan%2C+T%2E%2C+Joseph%2C+N%2E%2C+Mann%2C+B%2E%2C+Askell%2C+A%2E%2C+Bai%2C+Y%2E%2C+Chen%2C+A%2E%2C+Conerly%2C+T%2E%2C+DasSarma%2C+N%2E%2C+Drain%2C+D%2E%2C+Ganguli%2C+D%2E%2C+Hatfield%2DDodds%2C+Z%2E%2C+Hernandez%2C+D%2E%2C+Jones%2C+A%2E%2C+K)
9. Hanna, M., Liu, O., and Variengien, A. [How does GPT-2 compute greater-than?: Interpreting mathematical abilities in a pre-trained language model](https://md.research.elamur.ai/free/en/papers/hanna-2023-gpt-2-compute-greater-than/). In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=p4PckNQR8k. [↗](https://openreview.net/forum?)
10. Heimersheim, S. and Nanda, N. How to use and interpret activation patching, 2024. URL [https://arxiv.org/](https://arxiv.org/) abs/2404.15255. — [arXiv:2404.15255](https://arxiv.org/abs/2404.15255) [↗](https://arxiv.org/)
11. Elhage, N., Hume, T., Olsson, C., Schiefer, N., Henighan, T., Kravec, S., Hatfield-Dodds, Z., Lasenby, R., Drain, D., Chen, C., Grosse, R., McCandlish, S., Kaplan, J., Amodei, D., Wattenberg, M., and Olah, C. Toy models of superposition. Transformer Circuits Thread, 2022. URL https://transformer-cir… [find](https://scholar.google.com/scholar?q=Elhage%2C+N%2E%2C+Hume%2C+T%2E%2C+Olsson%2C+C%2E%2C+Schiefer%2C+N%2E%2C+Henighan%2C+T%2E%2C+Kravec%2C+S%2E%2C+Hatfield%2DDodds%2C+Z%2E%2C+Lasenby%2C+R%2E%2C+Drain%2C+D%2E%2C+Chen%2C+C%2E%2C+Grosse%2C+R%2E%2C+McCandlish%2C+S%2E%2C+Kaplan%2C+J%2E%2C+Amodei%2C+D%2E%2C+Wattenberg%2C+M%2E%2C+and+)
12. Huben, R., Cunningham, H., Smith, L. R., Ewart, A., and Sharkey, L. Sparse autoencoders find highly interpretable features in language models. In The Twelfth International Conference on Learning Representations, 2024. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=F76bwRSLeK. [↗](https://openreview.net/forum?)
13. Engels, J., Michaud, E. J., Liao, I., Gurnee, W., and Tegmark, M. Not all language model features are linear, 2024. URL https://arxiv.org/abs/2405. 14860. [↗](https://arxiv.org/abs/2405)
14. Kramár, J., Lieberum, T., Shah, R., and Nanda, N. [Atp*: An efficient and scalable method for localizing llm behaviour to components](https://md.research.elamur.ai/free/en/papers/kramar-2024-atp-efficient-scalable-method-localizing/), 2024. URL [https://arxiv.org/](https://arxiv.org/) abs/2403.00745. [↗](https://arxiv.org/)
15. Levy, A. A. and Geva, M. Language models encode numbers using digit representations in base 10, 2024. URL https://arxiv.org/abs/2410.11781. nostalgebraist. interpreting GPT: the logit lens — LessWrong — lesswrong.com. https://www.lesswrong. com/posts/AcKRB8wDpdaN6v6ru/ interpreting-gpt-the-logit-len… [↗](https://arxiv.org/abs/2410.11781)
16. Liu, H., Dai, Z., So, D., and Le, Q. V. Pay attention to MLPs. In Beygelzimer, A., Dauphin, Y., Liang, P., and Vaughan, J. W. (eds.), Advances in Neural Information Processing Systems, 2021. URL [https://openreview.net/](https://openreview.net/) forum?id=KBnXrODoBW. [↗](https://openreview.net/)
17. Olah, C. and Jermyn, A. What is a linear representation? what is a multidimensional feature?, 2024. URL https://transformer-circuits. pub/2024/july-update/index.html\# linear-representations. [find](https://scholar.google.com/scholar?q=Olah%2C+C%2E+and+Jermyn%2C+A%2E+What+is+a+linear+representation%3F+what+is+a+multidimensional+feature%3F%2C+2024%2E+URL+https%3A%2F%2Ftransformer%2Dcircuits%2E+pub%2F2024%2Fjuly%2Dupdate%2Findex%2Ehtml%5C%23+linear%2Drepresentations%2E)
18. Liu, Z., Kitouni, O., Nolte, N., Michaud, E. J., Tegmark, M., and Williams, M. [Towards understanding grokking: An effective theory of representation learning](https://md.research.elamur.ai/free/en/papers/liu-2022-towards-understanding-grokking-effective-theory/). In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022. URL [https://openreview.…](https://openreview.…) [↗](https://openreview.…)
19. Olah, C., Cammarata, N., Schubert, L., Goh, G., Petrov, M., and Carter, S. Zoom in: An introduction to circuits. Distill, 2020. doi: 10.23915/distill.00024.001. [https://distill.pub/2020/circuits/zoom-in](https://distill.pub/2020/circuits/zoom-in). — [doi:10.23915/distill.00024.001](https://doi.org/10.23915/distill.00024.001) [↗](https://distill.pub/2020/circuits/zoom-in)
20. Makelov, A., Lange, G., and Nanda, N. Towards principled evaluations of sparse autoencoders for interpretability and control. In ICLR 2024 Workshop on Secure and Trustworthy Large Language Models, 2024. URL [https://openreview.net/forum?id=MHIX9H8aYF](https://openreview.net/forum?id=MHIX9H8aYF). [↗](https://openreview.net/forum?id=MHIX9H8aYF)
21. Marks, S., Rager, C., Michaud, E. J., Belinkov, Y., Bau, D., and Mueller, A. Sparse feature circuits: Discovering and editing interpretable causal graphs in language models, 2024. URL https://arxiv.org/abs/2403. 19647. [↗](https://arxiv.org/abs/2403)
22. Meng, K., Bau, D., Andonian, A. J., and Belinkov, Y. Locating and editing factual associations in GPT. In Oh, A. H., Agarwal, A., Belgrave, D., and Cho, K. (eds.), Advances in Neural Information Processing Systems, 2022. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=-h6WAS6eE4. [↗](https://openreview.net/forum?)
23. Olsson, C., Elhage, N., Nanda, N., Joseph, N., DasSarma, N., Henighan, T., Mann, B., Askell, A., Bai, Y., Chen, A., Conerly, T., Drain, D., Ganguli, D., Hatfield-Dodds, Z., Hernandez, D., Johnston, S., Jones, A., Kernion, J., Lovitt, L., Ndousse, K., Amodei, D., Brown, T., Clark, J., Kaplan, J., McC… — [In-context Learning and Induction Heads](https://md.research.elamur.ai/free/en/papers/olsson-2022-context-learning-induction-heads/)
24. OpenAI. URL — [The Linear Representation Hypothesis and the Geometry of Large Language Models](https://md.research.elamur.ai/free/en/papers/park-2023-linear-representation-hypothesis-geometry-large/) [https://openai.com/index/](https://openai.com/index/) learning-to-reason-with-llms. Park, K., Choe, Y. J., and Veitch, V. The linear representation hypothesis and the geometry of large language models. In Causal Representation Learning Workshop at NeurIPS 2023, 2023. URL https://openreview. net/forum?id=T0PoOJg8cK. [↗](https://openai.com/index/)
25. Rajamanoharan, S., Conmy, A., Smith, L., Lieberum, T., Varma, V., Kramár, J., Shah, R., and Nanda, N. [Improving dictionary learning with gated sparse autoencoders](https://md.research.elamur.ai/free/en/papers/rajamanoharan-2024-improving-dictionary-learning-gated-sparse/), 2024. URL https://arxiv.org/abs/2404.16014. [↗](https://arxiv.org/abs/2404.16014)
26. Nanda, N., Chan, L., Lieberum, T., Smith, J., and Steinhardt, J. [Progress measures for grokking via mechanistic interpretability](https://md.research.elamur.ai/free/en/papers/nanda-2023-progress-measures-grokking-mechanistic-interpretability/). In The Eleventh International Conference on Learning Representations, 2023a. URL [https://openreview.net/forum?id=9XFSbDPmdW](https://openreview.net/forum?id=9XFSbDPmdW). [↗](https://openreview.net/forum?id=9XFSbDPmdW)
27. Riviere, M., Pathak, S., Sessa, P. G., Hardin, C., Bhupatiraju, S., Hussenot, L., Mesnard, T., Shahriari, B., et al. Gemma 2: Improving open language models at a practical size, 2024. [find](https://scholar.google.com/scholar?q=Riviere%2C+M%2E%2C+Pathak%2C+S%2E%2C+Sessa%2C+P%2E+G%2E%2C+Hardin%2C+C%2E%2C+Bhupatiraju%2C+S%2E%2C+Hussenot%2C+L%2E%2C+Mesnard%2C+T%2E%2C+Shahriari%2C+B%2E%2C+et+al%2E+Gemma+2%3A+Improving+open+language+models+at+a+practical+size%2C+2024%2E)
28. Nanda, N., Rajamanoharan, S., Kramar, J., and Shah, R. Fact finding: Attempting to reverse-engineer factual recall on the neuron level, Dec 2023b. URL [https://www.alignmentforum](https://www.alignmentforum). Satpute, A., Gießing, N., Greiner-Petter, A., Schubotz, M., Teschke, O., Aizawa, A., and Gipp, B. Can org/posts/iGuwZTHWb… — [doi:10.1145/3626772.3657945](https://doi.org/10.1145/3626772.3657945) [↗](https://www.alignmentforum)
29. Stolfo, A., Belinkov, Y., and Sachan, M. A mechanistic interpretation of arithmetic reasoning in language models using causal mediation analysis. pp. 7035–7052, Singapore, 2023. Association for Computational Linguistics. doi: 10.18653/v1/2023.emnlp-main.435. URL https://aclanthology.org/2023.emnlp-… [↗](https://aclanthology.org/2023.emnlp-…)
30. Zhu, F., Dai, D., and Sui, Z. [Language models encode the value of numbers linearly](https://md.research.elamur.ai/free/en/papers/zhu-2025-language-models-encode-value-numbers/). In Rambow, O., Wanner, L., Apidianaki, M., Al-Khalifa, H., Eugenio, B. D., and Schockaert, S. (eds.), Proceedings of the 31st International Conference on Computational Linguistics, pp. 693–709, Abu Dhabi, UAE, Januar…
31. Templeton, A., Conerly, T., Marcus, J., Lindsey, J., Bricken, T., Chen, B., Pearce, A., Citro, C., Ameisen, E., Jones, A., Cunningham, H., Turner, N. L., McDougall, C., MacDiarmid, M., Freeman, C. D., Sumers, T. R., Rees, E., Batson, J., Jermyn, A., Carter, S., Olah, C., and Henighan, T. Scaling mon… [find](https://scholar.google.com/scholar?q=Templeton%2C+A%2E%2C+Conerly%2C+T%2E%2C+Marcus%2C+J%2E%2C+Lindsey%2C+J%2E%2C+Bricken%2C+T%2E%2C+Chen%2C+B%2E%2C+Pearce%2C+A%2E%2C+Citro%2C+C%2E%2C+Ameisen%2C+E%2E%2C+Jones%2C+A%2E%2C+Cunningham%2C+H%2E%2C+Turner%2C+N%2E+L%2E%2C+McDougall%2C+C%2E%2C+MacDiarmid%2C+M%2E%2C+Freeman%2C+C%2E+D%2E%2C)
32. Zlokapa, A., Tan, A. K., Martyn, J. M., Fiete, I. R., Tegmark, M., and Chuang, I. L. Fault-tolerant neural networks from biological error correction codes. Phys. Rev. E, 110:054303, Nov 2024. doi: 10.1103/PhysRevE.110. 054303. URL [https://link.aps.org/doi/10](https://link.aps.org/doi/10). 1103/PhysRevE.110.054303. — [doi:10.1103/physreve.110](https://doi.org/10.1103/physreve.110) [↗](https://link.aps.org/doi/10)
33. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, L. u., and Polosukhin, I. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., and Garnett, R. (eds.), Advances in Neural Information Processing Systems, v…
34. Wang, B. and Komatsuzaki, A. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. [https://github.com/kingoflolz/](https://github.com/kingoflolz/) mesh-transformer-jax, May 2021. [↗](https://github.com/kingoflolz/)
35. Wang, K. R., Variengien, A., Conmy, A., Shlegeris, B., and Steinhardt, J. Interpretability in the wild: a circuit for indirect object identification in GPT-2 small. In The Eleventh International Conference on Learning Representations, 2023. URL [https://openreview.net/forum?](https://openreview.net/forum?) id=NpsVSN6o4ul. [↗](https://openreview.net/forum?)
36. Yip, C. H., Agrawal, R., Chan, L., and Gross, J. Modular addition without black-boxes: Compressing explanations of mlps that compute numerical integration, 2024. [find](https://scholar.google.com/scholar?q=Yip%2C+C%2E+H%2E%2C+Agrawal%2C+R%2E%2C+Chan%2C+L%2E%2C+and+Gross%2C+J%2E+Modular+addition+without+black%2Dboxes%3A+Compressing+explanations+of+mlps+that+compute+numerical+integration%2C+2024%2E)
37. Zhong, Z., Liu, Z., Tegmark, M., and Andreas, J. The clock and the pizza: Two stories in mechanistic explanation of neural networks. In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL [https://openreview.net/forum?id=S5wmbQc1We](https://openreview.net/forum?id=S5wmbQc1We). [↗](https://openreview.net/forum?id=S5wmbQc1We)
38. Zhou, T., Fu, D., Sharan, V., and Jia, R. Pre-trained large language models use fourier features to compute addition. In The Thirty-eighth Annual Conference on Neural Information Processing Systems, 2024. URL [https://openreview.net/forum?id=i4MutM2TZb](https://openreview.net/forum?id=i4MutM2TZb). [↗](https://openreview.net/forum?id=i4MutM2TZb)

*Updated 3 November 2026*