---
title: "DR Tulu — paper by Shao et al. (2025) — summary and ideas"
description: "The authors propose RLER, which uses per-question rubrics as rewards and updates… 41.8 points. 46 references, cited by 28 papers here."
authors: ["Rulin Shao", "Akari Asai", "Shannon Zejiang Shen", "Hamish Ivison", "Varsha Kishore", "Jingming Zhuo", "Xinran Zhao", "Molly Park", "Samuel G. Finlayson", "David Sontag", "Tyler Murray", "Sewon Min", "Pradeep Dasigi", "Luca Soldaini", "Faeze Brahman", "Wen-tau Yih", "Tongshuang Wu", "Luke Zettlemoyer", "Yoon Kim", "Hannaneh Hajishirzi", "Pang Wei Koh"]
year: 2025
venue: "arXiv:2511.19399"
source: "https://arxiv.org/abs/2511.19399"
doi: "10.48550/arxiv.2511.19399"
arxiv: "2511.19399"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shao-2025-dr-tulu-reinforcement-learning-evolving/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shao-2025-dr-tulu-reinforcement-learning-evolving/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/4/) / DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research

# DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research

**Authors:** [Rulin Shao](https://md.research.elamur.ai/free/en/authors/rulin-shao/), [Akari Asai](https://md.research.elamur.ai/free/en/authors/akari-asai/), [Shannon Zejiang Shen](https://md.research.elamur.ai/free/en/authors/shannon-zejiang-shen/), [Hamish Ivison](https://md.research.elamur.ai/free/en/authors/hamish-ivison/), Varsha Kishore, Jingming Zhuo, [Xinran Zhao](https://md.research.elamur.ai/free/en/authors/xinran-zhao/), Molly Park, Samuel G. Finlayson, [David Sontag](https://md.research.elamur.ai/free/en/authors/david-sontag/), Tyler Murray, [Sewon Min](https://md.research.elamur.ai/free/en/authors/sewon-min/), [Pradeep Dasigi](https://md.research.elamur.ai/free/en/authors/pradeep-dasigi/), [Luca Soldaini](https://md.research.elamur.ai/free/en/authors/luca-soldaini/), [Faeze Brahman](https://md.research.elamur.ai/free/en/authors/faeze-brahman/), [Wen-tau Yih](https://md.research.elamur.ai/free/en/authors/wen-tau-yih/), [Tongshuang Wu](https://md.research.elamur.ai/free/en/authors/tongshuang-wu/), [Luke Zettlemoyer](https://md.research.elamur.ai/free/en/authors/luke-zettlemoyer/), [Yoon Kim](https://md.research.elamur.ai/free/en/authors/yoon-kim/), [Hannaneh Hajishirzi](https://md.research.elamur.ai/free/en/authors/hannaneh-hajishirzi/) and [Pang Wei Koh](https://md.research.elamur.ai/free/en/authors/pang-wei-koh/) · **Year:** 2025 · **Venue:** arXiv:2511.19399 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2511.19399) · [PDF](https://arxiv.org/pdf/2511.19399) · [DOI 10.48550/arxiv.2511.19399](https://doi.org/10.48550/arxiv.2511.19399) · [arXiv 2511.19399](https://arxiv.org/abs/2511.19399) · [OpenAlex](https://openalex.org/W7106767948)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [28](https://md.research.elamur.ai/free/en/papers/shao-2025-dr-tulu-reinforcement-learning-evolving/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Open deep research (DR) agents are usually trained with reinforcement learning on short questions whose answers can be checked automatically. That kind of reward does not transfer to open-ended, long-form reports, where quality is loosely defined and checking claims requires current external evidence.

The authors propose RLER, which uses per-question rubrics as rewards and updates them during training. Each prompt starts with persistent rubrics derived from web search results. At every step, a rubric LM inspects the current rollouts and their search traces. It then adds positive criteria, which capture newly discovered useful knowledge, and negative ones, which target reward hacking such as verbatim copying of retrieved text. A buffer keeps only the rubrics whose scores vary most across rollouts.

Starting from Qwen3-8B, the authors run SFT on 16K trajectories, then RL with RLER plus format, search and citation rewards. The result, DR Tulu-8B, beats open agents and rivals proprietary systems on four long-form benchmarks at a tiny fraction of the cost. The model, data, rubrics, code and dr-agent-lib are all released.

## Key points
- **Benchmarks and baselines:** evaluation covers SQAv2 (AstaBench-ScholarQA-CS2), HealthBench, ResearchQA and DeepResearchBench.
  - DR Tulu-8B averages 15.6% above Tongyi DR and 0.7% above OpenAI DR.
  - Its margin over open 8–32B models ranges from 4.8 to 41.8 points.
- **Cost:** on SQAv2, one query costs about USD 0.0019, compared with roughly USD 1.8 for OpenAI DR (about 1000× cheaper).
- **Effect of RLER:** it adds 6.4–16.0 points over the SFT-only model. Substituting Qwen3-8B as judge and rubric generator still yields a gain of more than 4 points, about 1 point below the GPT-4.1 setup.
- **Tool use:** the model learns to choose tools per task. It uses paper search about 90% of the time on SQAv2 and web search/browsing about 55% of the time on DeepResearchBench.
- **New dataset:** GeneticDiseasesQA is a clinical benchmark on whether genetic variants qualify for therapy. DR Tulu-8B is competitive with proprietary agents on it, while other open agents cannot handle the task.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

## Cite

Cite as: Rulin Shao, Akari Asai, Shannon Zejiang Shen, Hamish Ivison, Varsha Kishore, Jingming Zhuo, Xinran Zhao, Molly Park, Samuel G. Finlayson, David Sontag, Tyler Murray, Sewon Min, Pradeep Dasigi, Luca Soldaini, Faeze Brahman, Wen-tau Yih, Tongshuang Wu, Luke Zettlemoyer, Yoon Kim, Hannaneh Hajishirzi and Pang Wei Koh. 2025. DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research. arXiv:2511.19399.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{shao2025dr,
  title = {{DR Tulu: Reinforcement Learning with Evolving Rubrics for Deep Research}},
  author = {Rulin Shao, Akari Asai, Shannon Zejiang Shen, Hamish Ivison, Varsha Kishore, Jingming Zhuo, Xinran Zhao, Molly Park, Samuel G. Finlayson, David Sontag, Tyler Murray, Sewon Min, Pradeep Dasigi, Luca Soldaini, Faeze Brahman, Wen-tau Yih, Tongshuang Wu, Luke Zettlemoyer, Yoon Kim, Hannaneh Hajishirzi and Pang Wei Koh},
  year = {2025},
  doi = {10.48550/arxiv.2511.19399},
  eprint = {2511.19399},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2511.19399}
}
```

- [← previous Chat-Driven Text Generation and Interaction for Person Retrieval](https://md.research.elamur.ai/free/en/papers/xie-2025-chat-driven-text-generation-interaction/)
- [next → Language Model Fine-Tuning on Scaled Survey Data for Predicting Distributions of Public Opinions](https://md.research.elamur.ai/free/en/papers/suh-2025-language-model-fine-tuning-scaled/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/4/)

## References

46 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Fan, R.-Z., Wang, Z., and Liu, P. Megascience: Pushing the frontiers of post-training datasets for science reasoning. arXiv preprint arXiv:2507.16812, 2025. — [arXiv:2507.16812](https://arxiv.org/abs/2507.16812) [find](https://scholar.google.com/scholar?q=Fan%2C+R%2E%2DZ%2E%2C+Wang%2C+Z%2E%2C+and+Liu%2C+P%2E+Megascience%3A+Pushing+the+frontiers+of+post%2Dtraining+datasets+for+science+reasoning%2E+arXiv+preprint+arXiv%3A2507%2E16812%2C+2025%2E+%E2%80%94+%5BarXiv%3A2507%2E16812%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
2. Arora, R. K., Wei, J., Hicks, R. S., Bowman, P., Quiñonero-Candela, J., Tsimpourlas, F., Sharman, M., Shah, M., Vallone, A., Beutel, A., et al. [Healthbench: Evaluating large language models towards improved human health](https://md.research.elamur.ai/free/en/papers/arora-2025-healthbench-evaluating-large-language-models/). arXiv preprint arXiv:2505.08775, 2025.
3. Gao, J., Fu, W., Xie, M., Xu, S., He, C., Mei, Z., Zhu, B., and Wu, Y. Beyond ten turns: Unlocking long-horizon agentic search with large-scale asynchronous rl. arXiv preprint arXiv:2508.07976, 2025. — [arXiv:2508.07976](https://arxiv.org/abs/2508.07976) [find](https://scholar.google.com/scholar?q=Gao%2C+J%2E%2C+Fu%2C+W%2E%2C+Xie%2C+M%2E%2C+Xu%2C+S%2E%2C+He%2C+C%2E%2C+Mei%2C+Z%2E%2C+Zhu%2C+B%2E%2C+and+Wu%2C+Y%2E+Beyond+ten+turns%3A+Unlocking+long%2Dhorizon+agentic+search+with+large%2Dscale+asynchronous+rl%2E+arXiv+preprint+arXiv%3A2508%2E07976%2C+2025%2E+)
4. Asai, A., He, J., Shao, R., Shi, W., Singh, A., Chang, J. C., Lo, K., Soldaini, L., Feldman, S., D’arcy, M., et al. [Openscholar: Synthesizing scientific literature with retrieval-augmented lms](https://md.research.elamur.ai/free/en/papers/asai-2024-openscholar-synthesizing-scientific-literature/). arXiv preprint arXiv:2411.14199, 2024.
5. Gunjal, A., Wang, A., Lau, E., Nath, V., He, Y., Liu, B., and Hendryx, S. [Rubrics as rewards: Reinforcement learning beyond verifiable domains](https://md.research.elamur.ai/free/en/papers/gunjal-2025-rubrics-rewards-reinforcement-learning-verifiable/). arXiv preprint arXiv:2507.17746, 2025.
6. Bragg, J., D’Arcy, M., Balepur, N., Bareket, D., Dalvi, B., Feldman, S., Haddad, D., Hwang, J. D., Jansen, P., Kishore, V., et al. Astabench: Rigorous benchmarking of ai agents with a scientific research suite. arXiv preprint arXiv:2510.21652, 2025. [find](https://scholar.google.com/scholar?q=Bragg%2C+J%2E%2C+D%E2%80%99Arcy%2C+M%2E%2C+Balepur%2C+N%2E%2C+Bareket%2C+D%2E%2C+Dalvi%2C+B%2E%2C+Feldman%2C+S%2E%2C+Haddad%2C+D%2E%2C+Hwang%2C+J%2E+D%2E%2C+Jansen%2C+P%2E%2C+Kishore%2C+V%2E%2C+et+al%2E+Astabench%3A+Rigorous+benchmarking+of+ai+agents+with+a+scientific+resea)
7. Guo, J., Chi, Z., Dong, L., Dong, Q., Wu, X., Huang, S., and Wei, F. Reward reasoning model. arXiv preprint arXiv:2505.14674, 2025. — [arXiv:2505.14674](https://arxiv.org/abs/2505.14674) [find](https://scholar.google.com/scholar?q=Guo%2C+J%2E%2C+Chi%2C+Z%2E%2C+Dong%2C+L%2E%2C+Dong%2C+Q%2E%2C+Wu%2C+X%2E%2C+Huang%2C+S%2E%2C+and+Wei%2C+F%2E+Reward+reasoning+model%2E+arXiv+preprint+arXiv%3A2505%2E14674%2C+2025%2E+%E2%80%94+%5BarXiv%3A2505%2E14674%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2505%2E14674%29)
8. Cao, T., Bhandari, N., Yerukola, A., Asai, A., and Sap, M. Out of style: Rag’s fragility to linguistic variation. arXiv preprint arXiv:2504.08231, 2025. [find](https://scholar.google.com/scholar?q=Cao%2C+T%2E%2C+Bhandari%2C+N%2E%2C+Yerukola%2C+A%2E%2C+Asai%2C+A%2E%2C+and+Sap%2C+M%2E+Out+of+style%3A+Rag%E2%80%99s+fragility+to+linguistic+variation%2E+arXiv+preprint+arXiv%3A2504%2E08231%2C+2025%2E)
9. Ho, X., Duong Nguyen, A.-K., Sugawara, S., and Aizawa, A. [Constructing a multi-hop QA dataset for comprehensive evaluation of reasoning steps](https://md.research.elamur.ai/free/en/papers/ho-2020-constructing-multi-hop-qa-dataset/). In Scott, D., Bel, N., and Zong, C. (eds.), Proceedings of the 28th International Conference on Computational Linguistics, pp. 6609–6625, Barcelona, Spain (O… — [doi:10.18653/v1/2020.coling-main](https://doi.org/10.18653/v1/2020.coling-main)
10. Liu, J., Li, Y., Zhang, C., Li, J., Chen, A., Ji, K., Cheng, W., Wu, Z., Du, C., Xu, Q., et al. Webexplorer: Explore and evolve for training long-horizon web agents. arXiv preprint arXiv:2509.06501, 2025. — [arXiv:2509.06501](https://arxiv.org/abs/2509.06501) [find](https://scholar.google.com/scholar?q=Liu%2C+J%2E%2C+Li%2C+Y%2E%2C+Zhang%2C+C%2E%2C+Li%2C+J%2E%2C+Chen%2C+A%2E%2C+Ji%2C+K%2E%2C+Cheng%2C+W%2E%2C+Wu%2C+Z%2E%2C+Du%2C+C%2E%2C+Xu%2C+Q%2E%2C+et+al%2E+Webexplorer%3A+Explore+and+evolve+for+training+long%2Dhorizon+web+agents%2E+arXiv+preprint+arXiv%3A2509%2E06501%2C+2)
11. Liu, N., Zhang, T., and Liang, P. [Evaluating verifiability in generative search engines](https://md.research.elamur.ai/free/en/papers/liu-2023-evaluating-verifiability-generative-search-engines/). In Bouamor, H., Pino, J., and Bali, K. (eds.), Findings of the Association for Computational Linguistics: EMNLP 2023, pp. 7001–7025, Singapore, December 2023a. Association for Computational Linguistics. doi: 10.…
12. Jayalath, D., Goel, S., Foster, T., Jain, P., Gururangan, S., Zhang, C., Goyal, A., and Schelten, A. Compute as teacher: Turning inference compute into reference-free supervision. arXiv preprint arXiv:2509.14234, 2025. — [arXiv:2509.14234](https://arxiv.org/abs/2509.14234) [find](https://scholar.google.com/scholar?q=Jayalath%2C+D%2E%2C+Goel%2C+S%2E%2C+Foster%2C+T%2E%2C+Jain%2C+P%2E%2C+Gururangan%2C+S%2E%2C+Zhang%2C+C%2E%2C+Goyal%2C+A%2E%2C+and+Schelten%2C+A%2E+Compute+as+teacher%3A+Turning+inference+compute+into+reference%2Dfree+supervision%2E+arXiv+preprint+arXiv)
13. Jiang, D., Lu, Y., Li, Z., Lyu, Z., Nie, P., Wang, H., Su, A., Chen, H., Zou, K., Du, C., et al. Verltool: Towards holistic agentic reinforcement learning with tool use. arXiv preprint arXiv:2509.01055, 2025. — [arXiv:2509.01055](https://arxiv.org/abs/2509.01055) [find](https://scholar.google.com/scholar?q=Jiang%2C+D%2E%2C+Lu%2C+Y%2E%2C+Li%2C+Z%2E%2C+Lyu%2C+Z%2E%2C+Nie%2C+P%2E%2C+Wang%2C+H%2E%2C+Su%2C+A%2E%2C+Chen%2C+H%2E%2C+Zou%2C+K%2E%2C+Du%2C+C%2E%2C+et+al%2E+Verltool%3A+Towards+holistic+agentic+reinforcement+learning+with+tool+use%2E+arXiv+preprint+arXiv%3A2509%2E0105)
14. Liu, Y., Iter, D., Xu, Y., Wang, S., Xu, R., and Zhu, C. [G-eval: Nlg evaluation using gpt-4 with better human alignment](https://md.research.elamur.ai/free/en/papers/liu-2023-g-eval-nlg-evaluation-gpt/). arXiv preprint arXiv:2303.16634, 2023b.
15. Jin, B., Zeng, H., Yue, Z., Yoon, J., Arik, S., Wang, D., Zamani, H., and Han, J. [Search-r1: Training llms to reason and leverage search engines with reinforcement learning](https://md.research.elamur.ai/free/en/papers/jin-2025-search-r1-training-llms-reason/). In COLM, 2025.
16. Mallen, A., Asai, A., Zhong, V., Das, R., Hajishirzi, H., and Khashabi, D. When not to trust language models: Investigating effectiveness and limitations of parametric and non-parametric memories. arXiv preprint, 2022. [find](https://scholar.google.com/scholar?q=Mallen%2C+A%2E%2C+Asai%2C+A%2E%2C+Zhong%2C+V%2E%2C+Das%2C+R%2E%2C+Hajishirzi%2C+H%2E%2C+and+Khashabi%2C+D%2E+When+not+to+trust+language+models%3A+Investigating+effectiveness+and+limitations+of+parametric+and+non%2Dparametric+memories%2E+arX)
17. Krishna, K., Roy, A., and Iyyer, M. [Hurdles to progress in long-form question answering](https://md.research.elamur.ai/free/en/papers/krishna-2021-hurdles-progress-long-form-question/). In Toutanova, K., Rumshisky, A., Zettlemoyer, L., Hakkani-Tur, D., Beltagy, I., Bethard, S., Cotterell, R., Chakraborty, T., and Zhou, Y. (eds.), Proceedings of the 2021 Conference of the North American Chapter…
18. Mallen, A., Asai, A., Zhong, V., Das, R., Khashabi, D., and Hajishirzi, H. When not to trust language models: Investigating effectiveness of parametric and non-parametric memories. In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Proceedings of the 61st Annual Meeting of the Association for C… [find](https://scholar.google.com/scholar?q=Mallen%2C+A%2E%2C+Asai%2C+A%2E%2C+Zhong%2C+V%2E%2C+Das%2C+R%2E%2C+Khashabi%2C+D%2E%2C+and+Hajishirzi%2C+H%2E+When+not+to+trust+language+models%3A+Investigating+effectiveness+of+parametric+and+non%2Dparametric+memories%2E+In+Rogers%2C+A%2E%2C+Boyd)
19. Lambert, N., Morrison, J., Pyatkin, V., Huang, S., Ivison, H., Brahman, F., Miranda, L. J. V., Liu, A., Dziri, N., Lyu, X., Gu, Y., Malik, S., Graf, V., Hwang, J. D., Yang, J., Bras, R. L., Tafjord, O., Wilhelm, C., Soldaini, L., Smith, N. A., Wang, Y., Dasigi, P., and Hajishirzi, H. Tulu 3: Pushing… — [Tulu 3: Pushing Frontiers in Open Language Model Post-Training](https://md.research.elamur.ai/free/en/papers/lambert-2024-tulu-3-pushing-frontiers-open/)
20. Miroyan, M., Wu, T.-H., King, L., Li, T., Pan, J., Hu, X., Chiang, W.-L., Angelopoulos, A. N., Darrell, T., Norouzi, N., et al. Search arena: Analyzing search-augmented llms. arXiv preprint arXiv:2506.05334, 2025. [find](https://scholar.google.com/scholar?q=Miroyan%2C+M%2E%2C+Wu%2C+T%2E%2DH%2E%2C+King%2C+L%2E%2C+Li%2C+T%2E%2C+Pan%2C+J%2E%2C+Hu%2C+X%2E%2C+Chiang%2C+W%2E%2DL%2E%2C+Angelopoulos%2C+A%2E+N%2E%2C+Darrell%2C+T%2E%2C+Norouzi%2C+N%2E%2C+et+al%2E+Search+arena%3A+Analyzing+search%2Daugmented+llms%2E+arXiv+preprint+arXiv%3A2506)
21. Nguyen, X.-P., Pandit, S., Reddy, R. G., Xu, A., Savarese, S., Xiong, C., and Joty, S. Sfr-deepresearch: Towards effective reinforcement learning for autonomously reasoning single agents. arXiv preprint arXiv:2509.06283, 2025. — [arXiv:2509.06283](https://arxiv.org/abs/2509.06283) [find](https://scholar.google.com/scholar?q=Nguyen%2C+X%2E%2DP%2E%2C+Pandit%2C+S%2E%2C+Reddy%2C+R%2E+G%2E%2C+Xu%2C+A%2E%2C+Savarese%2C+S%2E%2C+Xiong%2C+C%2E%2C+and+Joty%2C+S%2E+Sfr%2Ddeepresearch%3A+Towards+effective+reinforcement+learning+for+autonomously+reasoning+single+agents%2E+arXiv+prepri)
22. Li, H., Dong, Q., Chen, J., Su, H., Zhou, Y., Ai, Q., Ye, Z., and Liu, Y. [Llms-as-judges: a comprehensive survey on llm-based evaluation methods](https://md.research.elamur.ai/free/en/papers/li-2024-llms-judges-comprehensive-survey-llm/). arXiv preprint arXiv:2412.05579, 2024.
23. Li, X., Jin, J., Dong, G., Qian, H., Zhu, Y., Wu, Y., Wen, J.-R., and Dou, Z. [Webthinker: Empowering large reasoning models with deep research capability](https://md.research.elamur.ai/free/en/papers/li-2025-webthinker-empowering-large-reasoning-models/). arXiv preprint arXiv:2504.21776, 2025a.
24. Noukhovitch, M., Huang, S., Xhonneux, S., Hosseini, A., Agarwal, R., and Courville, A. Asynchronous RLHF: Faster and More Efficient Off-Policy RL for Language Models, October 2024. URL [http://arxiv.org/](http://arxiv.org/) abs/2410.18252. — [arXiv:2410.18252](https://arxiv.org/abs/2410.18252) [↗](http://arxiv.org/)
25. Li, Z., Guan, X., Zhang, B., Huang, S., Zhou, H., Lai, S., Yan, M., Jiang, Y., Xie, P., Huang, F., et al. Webweaver: Structuring web-scale evidence with dynamic OpenAI. Deep research system card, 2025. URL [https://openai.com/index/deepresearch-system-card/](https://openai.com/index/deepresearch-system-card/). Accessed: 2025-10-21. [↗](https://openai.com/index/deepresearch-system-card/)
26. Prabhakar, A., Ram, R., Chen, Z., Savarese, S., Wang, F., Xiong, C., Wang, H., and Yao, W. Enterprise deep research: Steerable multi-agent deep research for enterprise analytics. arXiv preprint arXiv:2510.17797, 2025. — [arXiv:2510.17797](https://arxiv.org/abs/2510.17797) [find](https://scholar.google.com/scholar?q=Prabhakar%2C+A%2E%2C+Ram%2C+R%2E%2C+Chen%2C+Z%2E%2C+Savarese%2C+S%2E%2C+Wang%2C+F%2E%2C+Xiong%2C+C%2E%2C+Wang%2C+H%2E%2C+and+Yao%2C+W%2E+Enterprise+deep+research%3A+Steerable+multi%2Dagent+deep+research+for+enterprise+analytics%2E+arXiv+preprint+arXiv%3A)
27. Wei, J., Karina, N., Chung, H. W., Jiao, Y. J., Papay, S., Glaese, A., Schulman, J., and Fedus, W. [Measuring short-form factuality in large language models](https://md.research.elamur.ai/free/en/papers/wei-2024-measuring-short-form-factuality-large/). arXiv preprint arXiv:2411.04368, 2024.
28. Rezaei, M., Vacareanu, R., Wang, Z., Wang, C., He, Y., and Akyürek, A. F. Online rubrics elicitation from pairwise comparisons. arXiv preprint arXiv:2510.07284, 2025. — [arXiv:2510.07284](https://arxiv.org/abs/2510.07284) [find](https://scholar.google.com/scholar?q=Rezaei%2C+M%2E%2C+Vacareanu%2C+R%2E%2C+Wang%2C+Z%2E%2C+Wang%2C+C%2E%2C+He%2C+Y%2E%2C+and+Akyu%CC%88rek%2C+A%2E+F%2E+Online+rubrics+elicitation+from+pairwise+comparisons%2E+arXiv+preprint+arXiv%3A2510%2E07284%2C+2025%2E+%E2%80%94+%5BarXiv%3A2510%2E07284%5D%28https%3A%2F%2Farx)
29. Wei, J., Sun, Z., Papay, S., McKinney, S., Han, J., Fulford, I., Chung, H. W., Passos, A. T., Fedus, W., and Glaese, A. [Browsecomp: A simple yet challenging benchmark for browsing agents](https://md.research.elamur.ai/free/en/papers/wei-2025-browsecomp-simple-yet-challenging-benchmark/). arXiv preprint arXiv:2504.12516, 2025.
30. Shao, R., Li, S. S., Xin, R., Geng, S., Wang, Y., Oh, S., Du, S. S., Lambert, N., Min, S., Krishna, R., et al. [Spurious rewards: Rethinking training signals in rlvr](https://md.research.elamur.ai/free/en/papers/shao-2025-spurious-rewards-rethinking-training-signals/). arXiv preprint arXiv:2506.10947, 2025.
31. Wu, J., Yin, W., Jiang, Y., Wang, Z., Xi, Z., Fang, R., Zhang, L., He, Y., Zhou, D., Xie, P., et al. [Webwalker: Benchmarking llms in web traversal](https://md.research.elamur.ai/free/en/papers/wu-2025-webwalker-benchmarking-llms-web-traversal/). arXiv preprint arXiv:2501.07572, 2025a.
32. Shao, Z., Wang, P., Zhu, Q., Xu, R., Song, J., Bi, X., Zhang, H., Zhang, M., Li, Y., Wu, Y., et al. [Deepseekmath: Pushing the limits of mathematical reasoning in open language models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/). arXiv preprint arXiv:2402.03300, 2024.
33. Wu, M., Zhang, G., Min, S., Levine, S., and Kumar, A. Rlac: Reinforcement learning with adversarial critic for free-form generation tasks. arXiv preprint arXiv:2511.01758, 2025b. — [arXiv:2511.01758](https://arxiv.org/abs/2511.01758) [find](https://scholar.google.com/scholar?q=Wu%2C+M%2E%2C+Zhang%2C+G%2E%2C+Min%2C+S%2E%2C+Levine%2C+S%2E%2C+and+Kumar%2C+A%2E+Rlac%3A+Reinforcement+learning+with+adversarial+critic+for+free%2Dform+generation+tasks%2E+arXiv+preprint+arXiv%3A2511%2E01758%2C+2025b%2E+%E2%80%94+%5BarXiv%3A2511%2E01758%5D%28)
34. Sharma, M., Zhang, C. B. C., Bandi, C., Wang, C., Aich, A., Nghiem, H., Rabbani, T., Htet, Y., Jang, B., Basu, S., et al. Researchrubrics: A benchmark of prompts and rubrics for evaluating deep research agents. arXiv preprint arXiv:2511.07685, 2025. — [arXiv:2511.07685](https://arxiv.org/abs/2511.07685) [find](https://scholar.google.com/scholar?q=Sharma%2C+M%2E%2C+Zhang%2C+C%2E+B%2E+C%2E%2C+Bandi%2C+C%2E%2C+Wang%2C+C%2E%2C+Aich%2C+A%2E%2C+Nghiem%2C+H%2E%2C+Rabbani%2C+T%2E%2C+Htet%2C+Y%2E%2C+Jang%2C+B%2E%2C+Basu%2C+S%2E%2C+et+al%2E+Researchrubrics%3A+A+benchmark+of+prompts+and+rubrics+for+evaluating+deep+resear)
35. Xu, F., Song, Y., Iyyer, M., and Choi, E. [A critical evaluation of evaluations for long-form question answering](https://md.research.elamur.ai/free/en/papers/xu-2023-critical-evaluation-evaluations-long-form/). In Rogers, A., Boyd-Graber, J., and Okazaki, N. (eds.), Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3225–3245, Tor…
36. Shi, D., Cao, J., Chen, Q., Sun, W., Li, W., Lu, H., Dong, F., Qin, T., Zhu, K., Liu, M., et al. Taskcraft: Automated generation of agentic tasks. arXiv preprint arXiv:2506.10055, 2025. — [arXiv:2506.10055](https://arxiv.org/abs/2506.10055) [find](https://scholar.google.com/scholar?q=Shi%2C+D%2E%2C+Cao%2C+J%2E%2C+Chen%2C+Q%2E%2C+Sun%2C+W%2E%2C+Li%2C+W%2E%2C+Lu%2C+H%2E%2C+Dong%2C+F%2E%2C+Qin%2C+T%2E%2C+Zhu%2C+K%2E%2C+Liu%2C+M%2E%2C+et+al%2E+Taskcraft%3A+Automated+generation+of+agentic+tasks%2E+arXiv+preprint+arXiv%3A2506%2E10055%2C+2025%2E+%E2%80%94+%5BarXiv%3A2506%2E)
37. Singh, A., Chang, J. C., Haddad, D., Naik, A., Hwang, J. D., Kinney, R., Weld, D. S., Downey, D., and Feldman, S. Ai2 scholar QA: Organized literature synthesis with attribution. In Mishra, P., Muresan, S., and Yu, T. (eds.), Proceedings of the 63rd Annual Meeting of the Association for Computationa… [find](https://scholar.google.com/scholar?q=Singh%2C+A%2E%2C+Chang%2C+J%2E+C%2E%2C+Haddad%2C+D%2E%2C+Naik%2C+A%2E%2C+Hwang%2C+J%2E+D%2E%2C+Kinney%2C+R%2E%2C+Weld%2C+D%2E+S%2E%2C+Downey%2C+D%2E%2C+and+Feldman%2C+S%2E+Ai2+scholar+QA%3A+Organized+literature+synthesis+with+attribution%2E+In+Mishra%2C+P%2E%2C+Muresa)
38. Yang, A., Li, A., Yang, B., Zhang, B., Hui, B., Zheng, B., Yu, B., Gao, C., Huang, C., Lv, C., et al. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388, 2025.
39. Yang, Z., Qi, P., Zhang, S., Bengio, Y., Cohen, W., Salakhutdinov, R., and Manning, C. D. [HotpotQA: A dataset for diverse, explainable multi-hop question answering](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/). In Riloff, E., Chiang, D., Hockenmaier, J., and Tsujii, J. (eds.), Proceedings of the 2018 Conference on Empirical Methods in Natural L…
40. Team, T. D., Li, B., Zhang, B., Zhang, D., Huang, F., Li, G., Chen, G., Yin, H., Wu, J., Zhou, J., et al. Tongyi deepresearch technical report. arXiv preprint arXiv:2510.24701, 2025. — [arXiv:2510.24701](https://arxiv.org/abs/2510.24701) [find](https://scholar.google.com/scholar?q=Team%2C+T%2E+D%2E%2C+Li%2C+B%2E%2C+Zhang%2C+B%2E%2C+Zhang%2C+D%2E%2C+Huang%2C+F%2E%2C+Li%2C+G%2E%2C+Chen%2C+G%2E%2C+Yin%2C+H%2E%2C+Wu%2C+J%2E%2C+Zhou%2C+J%2E%2C+et+al%2E+Tongyi+deepresearch+technical+report%2E+arXiv+preprint+arXiv%3A2510%2E24701%2C+2025%2E+%E2%80%94+%5BarXiv%3A2510%2E247)
41. Yifei, L. S., Chang, A., Malaviya, C., and Yatskar, M. Researchqa: Evaluating scholarly question answering at scale across 75 fields with survey-mined questions and rubrics. arXiv preprint arXiv:2509.00496, 2025. [find](https://scholar.google.com/scholar?q=Yifei%2C+L%2E+S%2E%2C+Chang%2C+A%2E%2C+Malaviya%2C+C%2E%2C+and+Yatskar%2C+M%2E+Researchqa%3A+Evaluating+scholarly+question+answering+at+scale+across+75+fields+with+survey%2Dmined+questions+and+rubrics%2E+arXiv+preprint+arXiv%3A2509%2E)
42. Viswanathan, V., Sun, Y., Ma, S., Kong, X., Cao, M., Neubig, G., and Wu, T. [Checklists are better than reward models for aligning language models](https://md.research.elamur.ai/free/en/papers/viswanathan-2025-checklists-better-than-reward-models/). arXiv preprint arXiv:2507.18624, 2025.
43. Yu, Q., Zhang, Z., Zhu, R., Yuan, Y., Zuo, X., Yue, Y., Dai, W., Fan, T., Liu, G., Liu, L., Liu, X., Lin, H., Lin, Z., Ma, B., Sheng, G., Tong, Y., Zhang, C., Zhang, M., Zhang, W., Zhu, H., Zhu, J., Chen, J., Chen, J., Wang, C., Yu, H., Song, Y., Wei, X., Zhou, H., Liu, J., Ma, W.-Y., Zhang, Y.-Q.,… — [arXiv:2504.15219](https://arxiv.org/abs/2504.15219) [find](https://scholar.google.com/scholar?q=Yu%2C+Q%2E%2C+Zhang%2C+Z%2E%2C+Zhu%2C+R%2E%2C+Yuan%2C+Y%2E%2C+Zuo%2C+X%2E%2C+Yue%2C+Y%2E%2C+Dai%2C+W%2E%2C+Fan%2C+T%2E%2C+Liu%2C+G%2E%2C+Liu%2C+L%2E%2C+Liu%2C+X%2E%2C+Lin%2C+H%2E%2C+Lin%2C+Z%2E%2C+Ma%2C+B%2E%2C+Sheng%2C+G%2E%2C+Tong%2C+Y%2E%2C+Zhang%2C+C%2E%2C+Zhang%2C+M%2E%2C+Zhang%2C+W%2E%2C+Zhu%2C+H%2E%2C+Zhu%2C+J%2E%2C+C)
44. M. [Dapo: An open-source llm reinforcement learning system at scale](https://md.research.elamur.ai/free/en/papers/yu-2025-dapo-open-source-llm-reinforcement/), 2025. URL [https://arxiv.org/](https://arxiv.org/) abs/2503.14476. [↗](https://arxiv.org/)
45. Zeng, Z., Yu, J., Gao, T., Meng, Y., Goyal, T., and Chen, D. Evaluating large language models at evaluating instruction following. In International Conference on Learning Representations (ICLR), 2024. [find](https://scholar.google.com/scholar?q=Zeng%2C+Z%2E%2C+Yu%2C+J%2E%2C+Gao%2C+T%2E%2C+Meng%2C+Y%2E%2C+Goyal%2C+T%2E%2C+and+Chen%2C+D%2E+Evaluating+large+language+models+at+evaluating+instruction+following%2E+In+International+Conference+on+Learning+Representations+%28ICLR%29%2C+2024%2E)
46. Zeng, Z., Ivison, H., Wang, Y., Yuan, L., Li, S. S., Ye, Z., Li, S., He, J., Zhou, R., Chen, T., Zhao, C., Tsvetkov, Y., Du, S. S., Jaques, N., Peng, H., Koh, P. W., and Hajishirzi, H. Rlve: Scaling up reinforcement learning for language models with adaptive verifiable environments. arXiv preprint 2… — [arXiv:2511.07317](https://arxiv.org/abs/2511.07317) [find](https://scholar.google.com/scholar?q=Zeng%2C+Z%2E%2C+Ivison%2C+H%2E%2C+Wang%2C+Y%2E%2C+Yuan%2C+L%2E%2C+Li%2C+S%2E+S%2E%2C+Ye%2C+Z%2E%2C+Li%2C+S%2E%2C+He%2C+J%2E%2C+Zhou%2C+R%2E%2C+Chen%2C+T%2E%2C+Zhao%2C+C%2E%2C+Tsvetkov%2C+Y%2E%2C+Du%2C+S%2E+S%2E%2C+Jaques%2C+N%2E%2C+Peng%2C+H%2E%2C+Koh%2C+P%2E+W%2E%2C+and+Hajishirzi%2C+H%2E+Rlve%3A+Scaling+u)

*Updated 2 November 2026*