---
title: "Deep Symbolic Regression for Recurrent Sequences (2022)"
description: "They build random unary-binary expression trees from operator sets (a compact integer-closed set and… 30 references, cited by 7 papers here."
authors: ["Stéphane d'Ascoli", "Pierre-Alexandre Kamienny", "Guillaume Lample", "François Charton"]
year: 2022
venue: "arXiv:2201.04600"
source: "https://arxiv.org/abs/2201.04600"
doi: "10.48550/arxiv.2201.04600"
arxiv: "2201.04600"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "https://arxiv.org/licenses/nonexclusive-distrib/1.0/license.html"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ascoli-2022-deep-symbolic-regression-recurrent-sequences/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ascoli-2022-deep-symbolic-regression-recurrent-sequences/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/6/) / Deep Symbolic Regression for Recurrent Sequences

# Deep Symbolic Regression for Recurrent Sequences

**Authors:** [Stéphane d'Ascoli](https://md.research.elamur.ai/free/en/authors/stephane-d-ascoli/), [Pierre-Alexandre Kamienny](https://md.research.elamur.ai/free/en/authors/pierre-alexandre-kamienny/), [Guillaume Lample](https://md.research.elamur.ai/free/en/authors/guillaume-lample/) and [François Charton](https://md.research.elamur.ai/free/en/authors/francois-charton/) · **Year:** 2022 · **Venue:** arXiv:2201.04600 · **Licence:** [arXiv — non-exclusive licence to distribute](https://arxiv.org/licenses/nonexclusive-distrib/1.0/license.html)

**Links:** [Publisher's page](https://arxiv.org/abs/2201.04600) · [PDF](https://arxiv.org/pdf/2201.04600v2) · [DOI 10.48550/arxiv.2201.04600](https://doi.org/10.48550/arxiv.2201.04600) · [arXiv 2201.04600](https://arxiv.org/abs/2201.04600) · [OpenAlex](https://openalex.org/W4226117689)

**Citations:** 20 citations (OpenAlex, 2026-10-14) · [7](https://md.research.elamur.ai/free/en/papers/ascoli-2022-deep-symbolic-regression-recurrent-sequences/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper tackles symbolic regression for sequences. Given the first terms of an integer or float sequence, the goal is to recover a recurrence relation u_n = f(n, earlier terms) that generates it. This is the familiar "guess the next number" puzzle from IQ tests, and earlier symbolic regression work, which targeted plain y = f(x) mappings, had mostly left it aside. The authors pretrain Transformers on a large synthetic corpus. They build random unary-binary expression trees from operator sets (a compact integer-closed set and a broader float set), draw random initial terms and unroll each sequence. A symbolic model emits the formula, while a numeric counterpart with the same architecture directly forecasts the upcoming values. On unseen synthetic recurrences, extrapolating with the predicted formulas is more precise than using the numeric model. On a subset of OEIS sequences, both models beat Mathematica's built-in tools. The float model can also express constants and special functions missing from its vocabulary through approximations such as 1.64493 ≈ π²/6. The approach replaces costly search with a single forward pass and could assist mathematical discovery.

## Key points
- On OEIS sequences with 25 input terms, next-term accuracy (tolerance 10⁻¹⁰) was:
  - symbolic model: 34.5%
  - numeric model: 54.9%
  - Mathematica FindLinearRecurrence: 21.2%
  - Mathematica FindSequenceFunction: 8.1%
- Operator vocabularies differ by setting:
  - Integer generator: abs, sqr, sign, relu, add, sub, mul, intdiv, mod.
  - Float generator: adds sqrt, inv, log, exp, trig functions, atan and real division.
- When constants are drawn from a continuous range outside the training vocabulary, the symbolic model's approximate accuracy (≤10 operators, tolerance 0.01) falls from about 60% to 42%.
- The model finds asymptotic approximations for scipy.special functions, e.g. bessel0 and Catalan numbers, and outperforms the numeric model even on non-polynomial families.
- Published at ICML 2022. Code is at github.com/facebookresearch/recur, and an interactive demo is available online.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Sorbonne Université](https://md.research.elamur.ai/free/en/universities/sorbonne-universite/) — University, Paris, France · 35 works · 7,416 citations
- [École Normale Supérieure - PSL](https://md.research.elamur.ai/free/en/labs/ecole-normale-superieure-psl/) — Research institute, Paris, France · 25 works · 4,704 citations
- [LIP6](https://md.research.elamur.ai/free/en/labs/lip6/) — Research institute, Paris, France · 7 works · 1,665 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [➗ Mathematics](https://md.research.elamur.ai/free/en/shelves/mathematics/) · 233 works · [⚛️ Physics and Astronomy](https://md.research.elamur.ai/free/en/shelves/physics-and-astronomy/) · 70 works · [⚛️ Statistical and Nonlinear Physics](https://md.research.elamur.ai/free/en/shelves/physics-and-astronomy/statistical-and-nonlinear-physics/) · 60 works · [➗ Modeling and Simulation](https://md.research.elamur.ai/free/en/shelves/mathematics/modeling-and-simulation/) · 12 works

## Cite

Cite as: Stéphane d'Ascoli, Pierre-Alexandre Kamienny, Guillaume Lample and François Charton. 2022. Deep Symbolic Regression for Recurrent Sequences. arXiv:2201.04600.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{dascoli2022deep,
  title = {{Deep Symbolic Regression for Recurrent Sequences}},
  author = {Stéphane d'Ascoli, Pierre-Alexandre Kamienny, Guillaume Lample and François Charton},
  year = {2022},
  doi = {10.48550/arxiv.2201.04600},
  eprint = {2201.04600},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2201.04600}
}
```

- [← previous Camelira: An Arabic Multi-Dialect Morphological Disambiguator](https://md.research.elamur.ai/free/en/papers/obeid-2022-camelira-arabic-multi-dialect-morphological/)
- [next → DuReader_retrieval: A Large-scale Chinese Benchmark for Passage Retrieval from Web Search Engine](https://md.research.elamur.ai/free/en/papers/qiu-2022-dureader-retrieval-large-scale/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/6/)

## References

30 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Allamanis, M., Chanthirasegaran, P., Kohli, P., and Sutton, C. [Learning continuous semantic representations of symbolic expressions](https://md.research.elamur.ai/free/en/papers/allamanis-2016-learning-continuous-semantic-representations-symbolic/). In International Conference on Machine Learning, pp. 80–88. PMLR, 2017.
2. Arabshahi, F., Singh, S., and Anandkumar, A. Towards solving differential equations through neural programming. In ICML Workshop on Neural Abstract Machines and Program Induction (NAMPI), 2018. [find](https://scholar.google.com/scholar?q=Arabshahi%2C+F%2E%2C+Singh%2C+S%2E%2C+and+Anandkumar%2C+A%2E+Towards+solving+differential+equations+through+neural+programming%2E+In+ICML+Workshop+on+Neural+Abstract+Machines+and+Program+Induction+%28NAMPI%29%2C+2018%2E)
3. Augusto, D. A. and Barbosa, H. J. Symbolic regression via genetic programming. In Proceedings. Vol. 1. Sixth Brazilian Symposium on Neural Networks, pp. 173–178. IEEE, 2000. [find](https://scholar.google.com/scholar?q=Augusto%2C+D%2E+A%2E+and+Barbosa%2C+H%2E+J%2E+Symbolic+regression+via+genetic+programming%2E+In+Proceedings%2E+Vol%2E+1%2E+Sixth+Brazilian+Symposium+on+Neural+Networks%2C+pp%2E+173%E2%80%93178%2E+IEEE%2C+2000%2E)
4. Biggio, L., Bendinelli, T., Neitz, A., Lucchi, A., and Parascandolo, G. Neural symbolic regression that scales. arXiv preprint arXiv:2106.06427, 2021. — [arXiv:2106.06427](https://arxiv.org/abs/2106.06427) [find](https://scholar.google.com/scholar?q=Biggio%2C+L%2E%2C+Bendinelli%2C+T%2E%2C+Neitz%2C+A%2E%2C+Lucchi%2C+A%2E%2C+and+Parascandolo%2C+G%2E+Neural+symbolic+regression+that+scales%2E+arXiv+preprint+arXiv%3A2106%2E06427%2C+2021%2E+%E2%80%94+%5BarXiv%3A2106%2E06427%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2106%2E0)
5. Charton, F. [Linear algebra with transformers](https://md.research.elamur.ai/free/en/papers/charton-2021-linear-algebra-transformers/). arXiv preprint arXiv:2112.01898, 2021.
6. Charton, F., Hayat, A., and Lample, G. [Learning advanced mathematical computations from examples](https://md.research.elamur.ai/free/en/papers/charton-2020-learning-advanced-mathematical-computations-examples/). arXiv preprint arXiv:2006.06462, 2020.
7. Cobbe, K., Kosaraju, V., Bavarian, M., Hilton, J., Nakano, R., Hesse, C., and Schulman, J. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168, 2021.
8. Davies, A., Velickovic, P., Buesing, L., Blackwell, S., Zheng, D., Tomasev, N., Tanburn, R., Battaglia, P., Blundell, C., Juhasz, A., et al. Advancing mathematics by guiding human intuition with ai. Nature, 2021. [find](https://scholar.google.com/scholar?q=Davies%2C+A%2E%2C+Velickovic%2C+P%2E%2C+Buesing%2C+L%2E%2C+Blackwell%2C+S%2E%2C+Zheng%2C+D%2E%2C+Tomasev%2C+N%2E%2C+Tanburn%2C+R%2E%2C+Battaglia%2C+P%2E%2C+Blundell%2C+C%2E%2C+Juhasz%2C+A%2E%2C+et+al%2E+Advancing+mathematics+by+guiding+human+intuition+with+ai%2E+N)
9. Kaiser, Ł. and Sutskever, I. Neural gpus learn algorithms. arXiv preprint arXiv:1511.08228, 2015. [find](https://scholar.google.com/scholar?q=Kaiser%2C+%C5%81%2E+and+Sutskever%2C+I%2E+Neural+gpus+learn+algorithms%2E+arXiv+preprint+arXiv%3A1511%2E08228%2C+2015%2E)
10. Kamienny, P.-A., d’Ascoli, S., Lample, G., and Charton, F. [End-to-end symbolic regression with transformers](https://md.research.elamur.ai/free/en/papers/kamienny-2022-end-end-symbolic-regression-transformers/). arXiv preprint arXiv:2204.10532, 2022.
11. Kim, S., Lu, P. Y., Mukherjee, S., Gilbert, M., Jing, L., Čeperić, V., and Soljačić, M. Integration of neural network-based symbolic regression in deep learning for scientific discovery. IEEE Transactions on Neural Networks and Learning Systems, 2020. [find](https://scholar.google.com/scholar?q=Kim%2C+S%2E%2C+Lu%2C+P%2E+Y%2E%2C+Mukherjee%2C+S%2E%2C+Gilbert%2C+M%2E%2C+Jing%2C+L%2E%2C+C%CC%8Ceperic%CC%81%2C+V%2E%2C+and+Soljac%CC%8Cic%CC%81%2C+M%2E+Integration+of+neural+network%2Dbased+symbolic+regression+in+deep+learning+for+scientific+discovery%2E+IEEE+Tran)
12. Lample, G. and Charton, F. [Deep learning for symbolic mathematics](https://md.research.elamur.ai/free/en/papers/lample-2019-deep-learning-symbolic-mathematics/). arXiv preprint arXiv:1912.01412, 2019.
13. McKay, B., Willis, M. J., and Barton, G. W. Using a tree structured genetic algorithm to perform symbolic regression. In First international conference on genetic algorithms in engineering systems: innovations and applications, pp. 487–492. IET, 1995. [find](https://scholar.google.com/scholar?q=McKay%2C+B%2E%2C+Willis%2C+M%2E+J%2E%2C+and+Barton%2C+G%2E+W%2E+Using+a+tree+structured+genetic+algorithm+to+perform+symbolic+regression%2E+In+First+international+conference+on+genetic+algorithms+in+engineering+systems%3A+in)
14. Meurer, A., Smith, C. P., Paprocki, M., Čertı́k, O., Kirpichev, S. B., Rocklin, M., Kumar, A., Ivanov, S., Moore, J. K., Singh, S., et al. Sympy: symbolic computing in python. PeerJ Computer Science, 3:e103, 2017. [find](https://scholar.google.com/scholar?q=Meurer%2C+A%2E%2C+Smith%2C+C%2E+P%2E%2C+Paprocki%2C+M%2E%2C+C%CC%8Cert%C4%B1%CC%81k%2C+O%2E%2C+Kirpichev%2C+S%2E+B%2E%2C+Rocklin%2C+M%2E%2C+Kumar%2C+A%2E%2C+Ivanov%2C+S%2E%2C+Moore%2C+J%2E+K%2E%2C+Singh%2C+S%2E%2C+et+al%2E+Sympy%3A+symbolic+computing+in+python%2E+PeerJ+Computer+Science%2C)
15. Murari, A., Peluso, E., Gelfusa, M., Lupelli, I., Lunga-roni, M., and Gaudio, P. Symbolic regression via genetic programming for data driven derivation of confinement scaling laws without any assumption on their mathematical form. Plasma Physics and Controlled Fusion, 57(1): 014008, 2014. [find](https://scholar.google.com/scholar?q=Murari%2C+A%2E%2C+Peluso%2C+E%2E%2C+Gelfusa%2C+M%2E%2C+Lupelli%2C+I%2E%2C+Lunga%2Droni%2C+M%2E%2C+and+Gaudio%2C+P%2E+Symbolic+regression+via+genetic+programming+for+data+driven+derivation+of+confinement+scaling+laws+without+any+assumpti)
16. Nam, H., Kim, S., and Jung, K. Number sequence prediction problems for evaluating computational powers of neural networks, 2018. [find](https://scholar.google.com/scholar?q=Nam%2C+H%2E%2C+Kim%2C+S%2E%2C+and+Jung%2C+K%2E+Number+sequence+prediction+problems+for+evaluating+computational+powers+of+neural+networks%2C+2018%2E)
17. Paccanaro, A. and Hinton, G. E. Learning distributed representations of concepts using linear relational embedding. IEEE Transactions on Knowledge and Data Engineering, 13(2):232–244, 2001. [find](https://scholar.google.com/scholar?q=Paccanaro%2C+A%2E+and+Hinton%2C+G%2E+E%2E+Learning+distributed+representations+of+concepts+using+linear+relational+embedding%2E+IEEE+Transactions+on+Knowledge+and+Data+Engineering%2C+13%282%29%3A232%E2%80%93244%2C+2001%2E)
18. Petersen, B. K., Larma, M. L., Mundhenk, T. N., Santiago, C. P., Kim, S. K., and Kim, J. T. Deep symbolic regression: Recovering mathematical expressions from data via risk-seeking policy gradients. arXiv preprint arXiv:1912.04871, 2019. — [arXiv:1912.04871](https://arxiv.org/abs/1912.04871) [find](https://scholar.google.com/scholar?q=Petersen%2C+B%2E+K%2E%2C+Larma%2C+M%2E+L%2E%2C+Mundhenk%2C+T%2E+N%2E%2C+Santiago%2C+C%2E+P%2E%2C+Kim%2C+S%2E+K%2E%2C+and+Kim%2C+J%2E+T%2E+Deep+symbolic+regression%3A+Recovering+mathematical+expressions+from+data+via+risk%2Dseeking+policy+gradients%2E+a)
19. Raayoni, G., Gottlieb, S., Manor, Y., Pisha, G., Harris, Y., Mendlovic, U., Haviv, D., Hadad, Y., and Kaminer, I. Generating conjectures on fundamental constants with the ramanujan machine. Nature, 590(7844):67–73, 2021. [find](https://scholar.google.com/scholar?q=Raayoni%2C+G%2E%2C+Gottlieb%2C+S%2E%2C+Manor%2C+Y%2E%2C+Pisha%2C+G%2E%2C+Harris%2C+Y%2E%2C+Mendlovic%2C+U%2E%2C+Haviv%2C+D%2E%2C+Hadad%2C+Y%2E%2C+and+Kaminer%2C+I%2E+Generating+conjectures+on+fundamental+constants+with+the+ramanujan+machine%2E+Nature%2C+59)
20. Ragni, M. and Klein, A. Predicting numbers: An ai approach to solving number series. In Bach, J. and Edelkamp, S. (eds.), KI 2011: Advances in Artificial Intelligence, pp. 255–259. [find](https://scholar.google.com/scholar?q=Ragni%2C+M%2E+and+Klein%2C+A%2E+Predicting+numbers%3A+An+ai+approach+to+solving+number+series%2E+In+Bach%2C+J%2E+and+Edelkamp%2C+S%2E+%28eds%2E%29%2C+KI+2011%3A+Advances+in+Artificial+Intelligence%2C+pp%2E+255%E2%80%93259%2E)
21. Springer Berlin Heidelberg, 2011. [find](https://scholar.google.com/scholar?q=Springer+Berlin+Heidelberg%2C+2011%2E)
22. Ryskina, M. and Knight, K. Learning mathematical properties of integers. arXiv preprint arXiv:2109.07230, 2021. [find](https://scholar.google.com/scholar?q=Ryskina%2C+M%2E+and+Knight%2C+K%2E+Learning+mathematical+properties+of+integers%2E+arXiv+preprint+arXiv%3A2109%2E07230%2C+2021%2E)
23. Sahoo, S., Lampert, C., and Martius, G. Learning equations for extrapolation and control. In International Conference on Machine Learning, pp. 4442–4450. PMLR, 2018. [find](https://scholar.google.com/scholar?q=Sahoo%2C+S%2E%2C+Lampert%2C+C%2E%2C+and+Martius%2C+G%2E+Learning+equations+for+extrapolation+and+control%2E+In+International+Conference+on+Machine+Learning%2C+pp%2E+4442%E2%80%934450%2E+PMLR%2C+2018%2E)
24. Saxton, D., Grefenstette, E., Hill, F., and Kohli, P. [Analysing mathematical reasoning abilities of neural models](https://md.research.elamur.ai/free/en/papers/saxton-2019-analysing-mathematical-reasoning-abilities-neural/). arXiv preprint arXiv:1904.01557, 2019.
25. Schmidt, M. and Lipson, H. Distilling free-form natural laws from experimental data. science, 324(5923):81–85, 2009. [find](https://scholar.google.com/scholar?q=Schmidt%2C+M%2E+and+Lipson%2C+H%2E+Distilling+free%2Dform+natural+laws+from+experimental+data%2E+science%2C+324%285923%29%3A81%E2%80%9385%2C+2009%2E)
26. Sloane, N. J. The on-line encyclopedia of integer sequences. In Towards mechanized mathematical assistants, pp. 130– 130. Springer, 2007. [find](https://scholar.google.com/scholar?q=Sloane%2C+N%2E+J%2E+The+on%2Dline+encyclopedia+of+integer+sequences%2E+In+Towards+mechanized+mathematical+assistants%2C+pp%2E+130%E2%80%93+130%2E+Springer%2C+2007%2E)
27. Trask, A., Hill, F., Reed, S., Rae, J., Dyer, C., and Blunsom, P. Neural arithmetic logic units. arXiv preprint arXiv:1808.00508, 2018. [find](https://scholar.google.com/scholar?q=Trask%2C+A%2E%2C+Hill%2C+F%2E%2C+Reed%2C+S%2E%2C+Rae%2C+J%2E%2C+Dyer%2C+C%2E%2C+and+Blunsom%2C+P%2E+Neural+arithmetic+logic+units%2E+arXiv+preprint+arXiv%3A1808%2E00508%2C+2018%2E)
28. Valipour, M., You, B., Panju, M., and Ghodsi, A. Symbolicgpt: A generative transformer model for symbolic regression. arXiv preprint arXiv:2106.14131, 2021. van der Maaten, L. and Hinton, G. Visualizing data using t-SNE. Journal of Machine Learning Research, 9:2579– 2605, 2008. — [arXiv:2106.14131](https://arxiv.org/abs/2106.14131) [find](https://scholar.google.com/scholar?q=Valipour%2C+M%2E%2C+You%2C+B%2E%2C+Panju%2C+M%2E%2C+and+Ghodsi%2C+A%2E+Symbolicgpt%3A+A+generative+transformer+model+for+symbolic+regression%2E+arXiv+preprint+arXiv%3A2106%2E14131%2C+2021%2E+van+der+Maaten%2C+L%2E+and+Hinton%2C+G%2E+Visualizi)
29. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in neural information processing systems, pp. 5998–6008, 2017.
30. Wu, C. W. Can machine learning identify interesting mathematics? an exploration using empirically observed laws. arXiv preprint arXiv:1805.07431, 2018. — [arXiv:1805.07431](https://arxiv.org/abs/1805.07431) [find](https://scholar.google.com/scholar?q=Wu%2C+C%2E+W%2E+Can+machine+learning+identify+interesting+mathematics%3F+an+exploration+using+empirically+observed+laws%2E+arXiv+preprint+arXiv%3A1805%2E07431%2C+2018%2E+%E2%80%94+%5BarXiv%3A1805%2E07431%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1805%2E)

*Updated 3 November 2026*