---
title: "AutoReproduce — paper by Zhao et al. (2025)"
description: "The authors present AutoReproduce, a multi-agent system that works through three stages in order: it… 42 references, cited by 12 papers here."
authors: ["Xuanle Zhao", "Zilin Sang", "Yuxuan Li", "Qi Shi", "Weilun Zhao", "Shuo Wang", "Duzhen Zhang", "Xu Han", "Zhiyuan Liu", "Maosong Sun"]
year: 2025
venue: "arXiv:2505.20662"
source: "https://arxiv.org/abs/2505.20662"
doi: "10.48550/arxiv.2505.20662"
arxiv: "2505.20662"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhao-2025-autoreproduce-automatic-ai-experiment/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhao-2025-autoreproduce-automatic-ai-experiment/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/9/) / AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage

# AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage

**Authors:** [Xuanle Zhao](https://md.research.elamur.ai/free/en/authors/xuanle-zhao/), Zilin Sang, [Yuxuan Li](https://md.research.elamur.ai/free/en/authors/yuxuan-li/), [Qi Shi](https://md.research.elamur.ai/free/en/authors/qi-shi/), [Weilun Zhao](https://md.research.elamur.ai/free/en/authors/weilun-zhao/), [Shuo Wang](https://md.research.elamur.ai/free/en/authors/shuo-wang/), [Duzhen Zhang](https://md.research.elamur.ai/free/en/authors/duzhen-zhang/), [Xu Han](https://md.research.elamur.ai/free/en/authors/xu-han/), [Zhiyuan Liu](https://md.research.elamur.ai/free/en/authors/zhiyuan-liu/) and [Maosong Sun](https://md.research.elamur.ai/free/en/authors/maosong-sun/) · **Year:** 2025 · **Venue:** [arXiv:2505.20662](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2025](https://md.research.elamur.ai/free/en/venues/acl/2025/)) · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2505.20662) · [PDF](https://arxiv.org/pdf/2505.20662) · [DOI 10.48550/arxiv.2505.20662](https://doi.org/10.48550/arxiv.2505.20662) · [arXiv 2505.20662](https://arxiv.org/abs/2505.20662) · [OpenAlex](https://openalex.org/W4415036172)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [12](https://md.research.elamur.ai/free/en/papers/zhao-2025-autoreproduce-automatic-ai-experiment/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper addresses automatically reproducing AI experiments from a paper's text. This is difficult because papers often leave out implementation details that a field treats as common knowledge. The authors present AutoReproduce, a multi-agent system that works through three stages in order: it first reviews the literature, then traces the paper's lineage, and finally develops the code.

The paper lineage algorithm fills in unstated practices, such as module designs or data pipelines, by mining the cited papers and their code repositories. During code development, the system runs unit tests on a small sampled batch of data so it can confirm quickly that the generated code actually executes.

For evaluation, the authors built ReproduceBench, which has 13 papers from different AI sub-domains, each with a manually written and executed reference implementation. Five metrics score the generated code, ranging from structural fidelity to final execution accuracy. AutoReproduce beat the baselines on every metric on both ReproduceBench and PaperBench.

The work matters because it moves automated reproduction past paper-level summaries and toward code that runs and approaches reported results.

## Key points
- ReproduceBench has 13 papers with human-curated ground-truth code and five evaluation metrics. The task instructions give the agent the dataset path, the target metric, and ready-made preprocessing code.
- In human evaluation, total scores were:
  - ChatDev (GPT-4o): 8.86
  - PaperCoder (o3-mini): 13.24
  - AutoReproduce (Claude-3.5-Sonnet): 14.19
  - AutoReproduce (o3-mini): 14.61
- To validate lineage selection, experts chose five gold references per source paper. Matching the system's k ∈ {2, 3, 5} picks against them with Top-1 Recall@k and Hits@N showed strong agreement between the LLM and the humans.
- Agents reproduce high-level architectures well but get fine details wrong, such as convolution stride and padding. Learning-rate schedules that papers omit also limit how closely results can be matched.
- With EDIT-based debugging, o3-mini and Gemini-2.5-Pro converged in fewer turns than Claude-3.5-Sonnet. The system works at the level of single experiments rather than whole repositories, and it does not automate data preprocessing.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Tsinghua University](https://md.research.elamur.ai/free/en/universities/tsinghua-university/) — University, Beijing, China · 439 works · 94,783 citations
- [Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/government/chinese-academy-of-sciences/) — Government body, Beijing, China · 256 works · 25,136 citations
- [University of Chinese Academy of Sciences](https://md.research.elamur.ai/free/en/universities/university-of-chinese-academy-of-sciences/) — University, Beijing, China · 196 works · 8,907 citations
- [Xidian University](https://md.research.elamur.ai/free/en/universities/xidian-university/) — University, Xi'an, China · 19 works · 1,378 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,959 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,789 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,888 works · [🎯 Decision Sciences](https://md.research.elamur.ai/free/en/shelves/decision-sciences/) · 306 works · [🎯 Information Systems and Management](https://md.research.elamur.ai/free/en/shelves/decision-sciences/information-systems-and-management/) · 96 works

## Cite

Cite as: Xuanle Zhao, Zilin Sang, Yuxuan Li, Qi Shi, Weilun Zhao, Shuo Wang, Duzhen Zhang, Xu Han, Zhiyuan Liu and Maosong Sun. 2025. AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage. arXiv:2505.20662.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{zhao2025autoreproduce,
  title = {{AutoReproduce: Automatic AI Experiment Reproduction with Paper Lineage}},
  author = {Xuanle Zhao, Zilin Sang, Yuxuan Li, Qi Shi, Weilun Zhao, Shuo Wang, Duzhen Zhang, Xu Han, Zhiyuan Liu and Maosong Sun},
  year = {2025},
  doi = {10.48550/arxiv.2505.20662},
  eprint = {2505.20662},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2505.20662}
}
```

- [← previous ArithmAttack: Evaluating Robustness of LLMs to Noisy Context in Math Problem Solving](https://md.research.elamur.ai/free/en/papers/abedin-2025-arithmattack-evaluating-robustness-llms-noisy/)
- [next → Benchmarking Open-ended Audio Dialogue Understanding for Large Audio-Language Models](https://md.research.elamur.ai/free/en/papers/gao-2025-benchmarking-open-ended-audio-dialogue/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/9/)

## References

42 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jinheon Baek, Sujay Kumar Jauhar, Silviu Cucerzan, and Sung Ju Hwang. 2024. [Researchagent: Iterative research idea generation over scientific literature with large language models](https://md.research.elamur.ai/free/en/papers/baek-2024-researchagent-iterative-research-idea/). arXiv preprint arXiv:2404.07738.
2. Ben Bogin, Kejuan Yang, Shashank Gupta, Kyle Richardson, Erin Bransom, Peter Clark, Ashish Sabharwal, and Tushar Khot. 2024. Super: Evaluating agents on setting up and executing tasks from research repositories. arXiv preprint arXiv:2409.07440. [find](https://scholar.google.com/scholar?q=Ben+Bogin%2C+Kejuan+Yang%2C+Shashank+Gupta%2C+Kyle+Richardson%2C+Erin+Bransom%2C+Peter+Clark%2C+Ashish+Sabharwal%2C+and+Tushar+Khot%2E+2024%2E+Super%3A+Evaluating+agents+on+setting+up+and+executing+tasks+from+research+re)
3. Hu Cao, Yueyue Wang, Joy Chen, Dongsheng Jiang, Xiaopeng Zhang, Qi Tian, and Manning Wang. 2022. Swin-unet: Unet-like pure transformer for medical image segmentation. In European conference on computer vision, pages 205–218. Springer. [find](https://scholar.google.com/scholar?q=Hu+Cao%2C+Yueyue+Wang%2C+Joy+Chen%2C+Dongsheng+Jiang%2C+Xiaopeng+Zhang%2C+Qi+Tian%2C+and+Manning+Wang%2E+2022%2E+Swin%2Dunet%3A+Unet%2Dlike+pure+transformer+for+medical+image+segmentation%2E+In+European+conference+on+compute)
4. Jieneng Chen, Jieru Mei, Xianhang Li, Yongyi Lu, Qihang Yu, Qingyue Wei, Xiangde Luo, Yutong Xie, Ehsan Adeli, Yan Wang, et al. 2024. Transunet: Rethinking the u-net architecture design for medical image segmentation through the lens of transformers. Medical Image Analysis, 97:103280. [find](https://scholar.google.com/scholar?q=Jieneng+Chen%2C+Jieru+Mei%2C+Xianhang+Li%2C+Yongyi+Lu%2C+Qihang+Yu%2C+Qingyue+Wei%2C+Xiangde+Luo%2C+Yutong+Xie%2C+Ehsan+Adeli%2C+Yan+Wang%2C+et+al%2E+2024%2E+Transunet%3A+Rethinking+the+u%2Dnet+architecture+design+for+medical+im)
5. Ling-Hao Chen, Jiawei Zhang, Yewen Li, Yiren Pang, Xiaobo Xia, and Tongliang Liu. 2023a. Humanmac: Masked motion completion for human motion prediction. In Proceedings of the IEEE/CVF international conference on computer vision, pages 9544–9555. [find](https://scholar.google.com/scholar?q=Ling%2DHao+Chen%2C+Jiawei+Zhang%2C+Yewen+Li%2C+Yiren+Pang%2C+Xiaobo+Xia%2C+and+Tongliang+Liu%2E+2023a%2E+Humanmac%3A+Masked+motion+completion+for+human+motion+prediction%2E+In+Proceedings+of+the+IEEE%2FCVF+international+co)
6. Zheng Chen, Yulun Zhang, Jinjin Gu, Linghe Kong, Xiaokang Yang, and Fisher Yu. 2023b. Dual aggregation transformer for image super-resolution. In Proceedings of the IEEE/CVF international conference on computer vision, pages 12312–12321. [find](https://scholar.google.com/scholar?q=Zheng+Chen%2C+Yulun+Zhang%2C+Jinjin+Gu%2C+Linghe+Kong%2C+Xiaokang+Yang%2C+and+Fisher+Yu%2E+2023b%2E+Dual+aggregation+transformer+for+image+super%2Dresolution%2E+In+Proceedings+of+the+IEEE%2FCVF+international+conference+o)
7. Jeongwhan Choi, Seoyoung Hong, Noseong Park, and Sung-Bae Cho. 2023. Blurring-sharpening process models for collaborative filtering. In Proceedings of the 46th international ACM SIGIR conference on research and development in information retrieval, pages 1096–1106. [find](https://scholar.google.com/scholar?q=Jeongwhan+Choi%2C+Seoyoung+Hong%2C+Noseong+Park%2C+and+Sung%2DBae+Cho%2E+2023%2E+Blurring%2Dsharpening+process+models+for+collaborative+filtering%2E+In+Proceedings+of+the+46th+international+ACM+SIGIR+conference+on+re)
8. Yuning Cui, Yi Tao, Zhenshan Bing, Wenqi Ren, Xinwei Gao, Xiaochun Cao, Kai Huang, and Alois Knoll. 2023. Selective frequency network for image restoration. In The eleventh international conference on learning representations. [find](https://scholar.google.com/scholar?q=Yuning+Cui%2C+Yi+Tao%2C+Zhenshan+Bing%2C+Wenqi+Ren%2C+Xinwei+Gao%2C+Xiaochun+Cao%2C+Kai+Huang%2C+and+Alois+Knoll%2E+2023%2E+Selective+frequency+network+for+image+restoration%2E+In+The+eleventh+international+conference+on)
9. Abhyuday Desai, Cynthia Freeman, Zuhui Wang, and Ian Beaver. 2021. Timevae: A variational autoencoder for multivariate time series generation. arXiv preprint arXiv:2111.08095. — [arXiv:2111.08095](https://arxiv.org/abs/2111.08095) [find](https://scholar.google.com/scholar?q=Abhyuday+Desai%2C+Cynthia+Freeman%2C+Zuhui+Wang%2C+and+Ian+Beaver%2E+2021%2E+Timevae%3A+A+variational+autoencoder+for+multivariate+time+series+generation%2E+arXiv+preprint+arXiv%3A2111%2E08095%2E+%E2%80%94+%5BarXiv%3A2111%2E08095%5D%28htt)
10. Ege Erdil and Tamay Besiroglu. 2023. Explosive growth from ai automation: A review of the arguments. arXiv preprint arXiv:2309.11690. — [arXiv:2309.11690](https://arxiv.org/abs/2309.11690) [find](https://scholar.google.com/scholar?q=Ege+Erdil+and+Tamay+Besiroglu%2E+2023%2E+Explosive+growth+from+ai+automation%3A+A+review+of+the+arguments%2E+arXiv+preprint+arXiv%3A2309%2E11690%2E+%E2%80%94+%5BarXiv%3A2309%2E11690%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2309%2E11690%29)
11. Shubham Gandhi, Dhruv Shah, Manasi Patwardhan, Lovekesh Vig, and Gautam Shroff. 2025. Researchcodeagent: An llm multi-agent system for automated codification of research methodologies. arXiv preprint arXiv:2504.20117. — [arXiv:2504.20117](https://arxiv.org/abs/2504.20117) [find](https://scholar.google.com/scholar?q=Shubham+Gandhi%2C+Dhruv+Shah%2C+Manasi+Patwardhan%2C+Lovekesh+Vig%2C+and+Gautam+Shroff%2E+2025%2E+Researchcodeagent%3A+An+llm+multi%2Dagent+system+for+automated+codification+of+research+methodologies%2E+arXiv+preprint+)
12. Zhangyang Gao, Cheng Tan, Lirong Wu, and Stan Z Li. 2022. Simvp: Simpler yet better video prediction. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pages 3170–3180. [find](https://scholar.google.com/scholar?q=Zhangyang+Gao%2C+Cheng+Tan%2C+Lirong+Wu%2C+and+Stan+Z+Li%2E+2022%2E+Simvp%3A+Simpler+yet+better+video+prediction%2E+In+Proceedings+of+the+IEEE%2FCVF+conference+on+computer+vision+and+pattern+recognition%2C+pages+3170%E2%80%933)
13. Yang Gu, Hengyu You, Jian Cao, Muran Yu, Haoran Fan, and Shiyou Qian. 2024. Large language models for constructing and optimizing machine learning workflows: A survey. arXiv preprint arXiv:2411.10478. — [arXiv:2411.10478](https://arxiv.org/abs/2411.10478) [find](https://scholar.google.com/scholar?q=Yang+Gu%2C+Hengyu+You%2C+Jian+Cao%2C+Muran+Yu%2C+Haoran+Fan%2C+and+Shiyou+Qian%2E+2024%2E+Large+language+models+for+constructing+and+optimizing+machine+learning+workflows%3A+A+survey%2E+arXiv+preprint+arXiv%3A2411%2E10478%2E)
14. Noah Hollmann, Samuel Müller, Katharina Eggensperger, and Frank Hutter. 2022. Tabpfn: A transformer that solves small tabular classification problems in a second. arXiv preprint arXiv:2207.01848. [find](https://scholar.google.com/scholar?q=Noah+Hollmann%2C+Samuel+M%C3%BCller%2C+Katharina+Eggensperger%2C+and+Frank+Hutter%2E+2022%2E+Tabpfn%3A+A+transformer+that+solves+small+tabular+classification+problems+in+a+second%2E+arXiv+preprint+arXiv%3A2207%2E01848%2E)
15. Daniel P Jeong, Zachary C Lipton, and Pradeep Ravikumar. 2024. Llm-select: Feature selection with large language models. arXiv preprint arXiv:2407.02694. — [arXiv:2407.02694](https://arxiv.org/abs/2407.02694) [find](https://scholar.google.com/scholar?q=Daniel+P+Jeong%2C+Zachary+C+Lipton%2C+and+Pradeep+Ravikumar%2E+2024%2E+Llm%2Dselect%3A+Feature+selection+with+large+language+models%2E+arXiv+preprint+arXiv%3A2407%2E02694%2E+%E2%80%94+%5BarXiv%3A2407%2E02694%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F240)
16. Hai Jiang, Ao Luo, Haoqiang Fan, Songchen Han, and Shuaicheng Liu. 2023. Low-light image enhancement with wavelet-based diffusion models. ACM Transactions on Graphics (TOG), 42(6):1–14. [find](https://scholar.google.com/scholar?q=Hai+Jiang%2C+Ao+Luo%2C+Haoqiang+Fan%2C+Songchen+Han%2C+and+Shuaicheng+Liu%2E+2023%2E+Low%2Dlight+image+enhancement+with+wavelet%2Dbased+diffusion+models%2E+ACM+Transactions+on+Graphics+%28TOG%29%2C+42%286%29%3A1%E2%80%9314%2E)
17. Long Li, Weiwen Xu, Jiayan Guo, Ruochen Zhao, Xingxuan Li, Yuqian Yuan, Boqiang Zhang, Yuming Jiang, Yifei Xin, Ronghao Dang, et al. 2024. [Chain of ideas: Revolutionizing research via novel idea development with llm agents](https://md.research.elamur.ai/free/en/papers/li-2024-chain-ideas-revolutionizing-research/). arXiv preprint arXiv:2410.13185.
18. Zhong-Zhi Li, Duzhen Zhang, Ming-Liang Zhang, Jiaxin Zhang, Zengyan Liu, Yuxuan Yao, Haotian Xu, Junhao Zheng, Pei-Jie Wang, Xiuyi Chen, et al. 2025. [From system 1 to system 2: A survey of reasoning large language models](https://md.research.elamur.ai/free/en/papers/li-2025-system-1-system-2-survey/). arXiv preprint arXiv:2502.17419.
19. Zijie Lin, Yiqing Shen, Qilin Cai, He Sun, Jinrui Zhou, and Mingjun Xiao. 2025. Autop2c: An llm-based agent framework for code repository generation from multimodal content in academic papers. Preprint, arXiv:2504.20115. — [arXiv:2504.20115](https://arxiv.org/abs/2504.20115) [find](https://scholar.google.com/scholar?q=Zijie+Lin%2C+Yiqing+Shen%2C+Qilin+Cai%2C+He+Sun%2C+Jinrui+Zhou%2C+and+Mingjun+Xiao%2E+2025%2E+Autop2c%3A+An+llm%2Dbased+agent+framework+for+code+repository+generation+from+multimodal+content+in+academic+papers%2E+Preprin)
20. Shang-Ching Liu, ShengKun Wang, Wenqi Lin, Chung-Wei Hsiung, Yi-Chen Hsieh, Yu-Ping Cheng, Sian-Hong Luo, Tsungyao Chang, and Jianwei Zhang. 2023a. Jarvix: A llm no code platform for tabular data analysis and optimization. arXiv preprint arXiv:2312.02213. [find](https://scholar.google.com/scholar?q=Shang%2DChing+Liu%2C+ShengKun+Wang%2C+Wenqi+Lin%2C+Chung%2DWei+Hsiung%2C+Yi%2DChen+Hsieh%2C+Yu%2DPing+Cheng%2C+Sian%2DHong+Luo%2C+Tsungyao+Chang%2C+and+Jianwei+Zhang%2E+2023a%2E+Jarvix%3A+A+llm+no+code+platform+for+tabular+data+anal)
21. Yuqi Nie, Nam H Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. 2022. [A time series is worth 64 words: Long-term forecasting with transformers](https://md.research.elamur.ai/free/en/papers/nie-2022-time-series-worth-64-words/). arXiv preprint arXiv:2211.14730.
22. Chen Qian, Wei Liu, Hongzhang Liu, Nuo Chen, Yufan Dang, Jiahao Li, Cheng Yang, Weize Chen, Yusheng Su, Xin Cong, et al. 2023. [Chatdev: Communicative agents for software development](https://md.research.elamur.ai/free/en/papers/qian-2023-chatdev-communicative-agents-software/). arXiv preprint arXiv:2307.07924.
23. Samuel Schmidgall and Michael Moor. 2025. [Agentrxiv: Towards collaborative autonomous research](https://md.research.elamur.ai/free/en/papers/schmidgall-2025-agentrxiv-collaborative-autonomous-research/). arXiv preprint arXiv:2503.18102.
24. Samuel Schmidgall, Yusheng Su, Ze Wang, Ximeng Sun, Jialian Wu, Xiaodong Yu, Jiang Liu, Zicheng Liu, and Emad Barsoum. 2025. [Agent laboratory: Using llm agents as research assistants](https://md.research.elamur.ai/free/en/papers/schmidgall-2025-agent-laboratory-using-llm/). arXiv preprint arXiv:2501.04227.
25. Minju Seo, Jinheon Baek, Seongyun Lee, and Sung Ju Hwang. 2025. [Paper2code: Automating code generation from scientific papers in machine learning](https://md.research.elamur.ai/free/en/papers/seo-2025-paper2code-automating-code-generation/). arXiv preprint arXiv:2504.17192.
26. Shuwei Shao, Zhongcai Pei, Xingming Wu, Zhong Liu, Weihai Chen, and Zhengguo Li. 2023. Iebins: Iterative elastic bins for monocular depth estimation. Advances in Neural Information Processing Systems, 36:53025–53037. [find](https://scholar.google.com/scholar?q=Shuwei+Shao%2C+Zhongcai+Pei%2C+Xingming+Wu%2C+Zhong+Liu%2C+Weihai+Chen%2C+and+Zhengguo+Li%2E+2023%2E+Iebins%3A+Iterative+elastic+bins+for+monocular+depth+estimation%2E+Advances+in+Neural+Information+Processing+Systems%2C)
27. Chenglei Si, Diyi Yang, and Tatsunori Hashimoto. 2024. [Can llms generate novel research ideas? a large-scale human study with 100+ nlp researchers](https://md.research.elamur.ai/free/en/papers/si-2024-llms-generate-novel-research/). arXiv preprint arXiv:2409.04109.
28. Zachary S Siegel, Sayash Kapoor, Nitya Nagdir, Benedikt Stroebl, and Arvind Narayanan. 2024. [Core-bench: Fostering the credibility of published research through a computational reproducibility agent benchmark](https://md.research.elamur.ai/free/en/papers/siegel-2024-core-bench-fostering-credibility/). arXiv preprint arXiv:2409.11363.
29. Giulio Starace, Oliver Jaffe, Dane Sherburn, James Aung, Jun Shern Chan, Leon Maksin, Rachel Dias, Evan Mays, Benjamin Kinsella, Wyatt Thompson, et al. 2025. [Paperbench: Evaluating ai’s ability to replicate ai research](https://md.research.elamur.ai/free/en/papers/starace-2025-paperbench-evaluating-ai-s/). arXiv preprint arXiv:2504.01848.
30. Zihao Tang, Zheqi Lv, Shengyu Zhang, Fei Wu, and Kun Kuang. 2024. Modelgpt: Unleashing llm’s capabilities for tailored model generation. arXiv preprint arXiv:2402.12408. — [arXiv:2402.12408](https://arxiv.org/abs/2402.12408) [find](https://scholar.google.com/scholar?q=Zihao+Tang%2C+Zheqi+Lv%2C+Shengyu+Zhang%2C+Fei+Wu%2C+and+Kun+Kuang%2E+2024%2E+Modelgpt%3A+Unleashing+llm%E2%80%99s+capabilities+for+tailored+model+generation%2E+arXiv+preprint+arXiv%3A2402%2E12408%2E+%E2%80%94+%5BarXiv%3A2402%2E12408%5D%28https%3A%2F%2Fa)
31. Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. 2023b. itransformer: Inverted transformers are effective for time series forecasting. arXiv preprint arXiv:2310.06625. — [arXiv:2310.06625](https://arxiv.org/abs/2310.06625) [find](https://scholar.google.com/scholar?q=Yong+Liu%2C+Tengge+Hu%2C+Haoran+Zhang%2C+Haixu+Wu%2C+Shiyu+Wang%2C+Lintao+Ma%2C+and+Mingsheng+Long%2E+2023b%2E+itransformer%3A+Inverted+transformers+are+effective+for+time+series+forecasting%2E+arXiv+preprint+arXiv%3A2310%2E)
32. Bin Wang, Chao Xu, Xiaomeng Zhao, Linke Ouyang, Fan Wu, Zhiyuan Zhao, Rui Xu, Kaiwen Liu, Yuan Qu, Fukai Shang, et al. 2024a. [Mineru: An open-source solution for precise document content extraction](https://md.research.elamur.ai/free/en/papers/wang-2024-mineru-open-source-solution-precise/). arXiv preprint arXiv:2409.18839.
33. Daqin Luo, Chengjian Feng, Yuxuan Nong, and Yiqing Shen. 2024. Autom3l: An automated multimodal machine learning framework with large language models. In Proceedings of the 32nd ACM International Conference on Multimedia, pages 8586–8594. [find](https://scholar.google.com/scholar?q=Daqin+Luo%2C+Chengjian+Feng%2C+Yuxuan+Nong%2C+and+Yiqing+Shen%2E+2024%2E+Autom3l%3A+An+automated+multimodal+machine+learning+framework+with+large+language+models%2E+In+Proceedings+of+the+32nd+ACM+International+Conf)
34. Qingyun Wang, Doug Downey, Heng Ji, and Tom Hope. 2024b. [Scimon: Scientific inspiration machines optimized for novelty](https://md.research.elamur.ai/free/en/papers/wang-2023-scimon-scientific-inspiration-machines/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 279–299.
35. Xingyao Wang, Yangyi Chen, Lifan Yuan, Yizhe Zhang, Yunzhu Li, Hao Peng, and Heng Ji. 2024c. [Executable code actions elicit better llm agents](https://md.research.elamur.ai/free/en/papers/wang-2024-executable-code-actions-elicit-better/). In Forty-first International Conference on Machine Learning.
36. Yu Wang and Tyler Derr. 2021. Tree decomposed graph neural network. In Proceedings of the 30th ACM international conference on information & knowledge management, pages 2040–2049. [find](https://scholar.google.com/scholar?q=Yu+Wang+and+Tyler+Derr%2E+2021%2E+Tree+decomposed+graph+neural+network%2E+In+Proceedings+of+the+30th+ACM+international+conference+on+information+%26+knowledge+management%2C+pages+2040%E2%80%932049%2E)
37. Yixuan Weng, Minjun Zhu, Guangsheng Bao, Hongbo Zhang, Jindong Wang, Yue Zhang, and Linyi Yang. 2024. [Cycleresearcher: Improving automated research via automated review](https://md.research.elamur.ai/free/en/papers/weng-2024-cycleresearcher-improving-automated-research/). arXiv preprint arXiv:2411.00816.
38. Haixu Wu, Tengge Hu, Huakun Luo, Jianmin Wang, and Mingsheng Long. 2023. Solving high-dimensional pdes with latent spectral models. arXiv preprint arXiv:2301.12664. — [arXiv:2301.12664](https://arxiv.org/abs/2301.12664) [find](https://scholar.google.com/scholar?q=Haixu+Wu%2C+Tengge+Hu%2C+Huakun+Luo%2C+Jianmin+Wang%2C+and+Mingsheng+Long%2E+2023%2E+Solving+high%2Ddimensional+pdes+with+latent+spectral+models%2E+arXiv+preprint+arXiv%3A2301%2E12664%2E+%E2%80%94+%5BarXiv%3A2301%2E12664%5D%28https%3A%2F%2Farxiv%2E)
39. Zhiheng Xi, Wenxiang Chen, Xin Guo, Wei He, Yiwen Ding, Boyang Hong, Ming Zhang, Junzhe Wang, Senjie Jin, Enyu Zhou, et al. 2025. [The rise and potential of large language model based agents: A survey](https://md.research.elamur.ai/free/en/papers/xi-2023-rise-potential-large-language/). Science China Information Sciences, 68(2):121101.
40. Zekang Yang, Wang Zeng, Sheng Jin, Chen Qian, Ping Luo, and Wentao Liu. 2025. Autommlab: Automatically generating deployable models from language instructions for computer vision tasks. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 22056–22064. [find](https://scholar.google.com/scholar?q=Zekang+Yang%2C+Wang+Zeng%2C+Sheng+Jin%2C+Chen+Qian%2C+Ping+Luo%2C+and+Wentao+Liu%2E+2025%2E+Autommlab%3A+Automatically+generating+deployable+models+from+language+instructions+for+computer+vision+tasks%2E+In+Proceedings)
41. Shujian Zhang, Chengyue Gong, Lemeng Wu, Xingchao Liu, and Mingyuan Zhou. 2023. [Automl-gpt: Automatic machine learning with gpt](https://md.research.elamur.ai/free/en/papers/zhang-2023-automl-gpt-automatic-machine-learning/). arXiv preprint arXiv:2305.02499.
42. Borui Zhao, Quan Cui, Renjie Song, Yiyu Qiu, and Jiajun Liang. 2022. Decoupled knowledge distillation. In Proceedings of the IEEE/CVF Conference on computer vision and pattern recognition, pages 11953–11962. [find](https://scholar.google.com/scholar?q=Borui+Zhao%2C+Quan+Cui%2C+Renjie+Song%2C+Yiyu+Qiu%2C+and+Jiajun+Liang%2E+2022%2E+Decoupled+knowledge+distillation%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+computer+vision+and+pattern+recognition%2C+pages+11953)

*Updated 3 November 2026*