---
title: "Sayash Kapoor — author profile and papers — Elamur Research"
description: "Sayash Kapoor, Princeton University: 6 papers on Elamur Research (2023–2025), h-index 15 — affiliations, topics, co-authors and who cites…"
canonical: "https://research.elamur.ai/free/en/authors/sayash-kapoor/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/authors/sayash-kapoor/)

[home](https://md.research.elamur.ai/free/en/) / [authors](https://md.research.elamur.ai/free/en/authors/) / Sayash Kapoor

# Sayash Kapoor

Sayash Kapoor has 6 papers in our library, in 2023–2025, most of them on [Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/); often with [Benedikt Stroebl](https://md.research.elamur.ai/free/en/authors/benedikt-stroebl/), [Arvind Narayanan](https://md.research.elamur.ai/free/en/authors/arvind-narayanan/), [Nitya Nadgir](https://md.research.elamur.ai/free/en/authors/nitya-nadgir/).

6 papers in our library · 2023–2025

[OpenAlex](https://openalex.org/A5063345981) · [ORCID](https://orcid.org/0000-0001-5695-280X) · [Wikidata](https://www.wikidata.org/wiki/Q133824454) · [**h-index 15**](https://openalex.org/authors/A5063345981) · [68 works](https://openalex.org/works?filter=author.id:A5063345981) · [2,209 citations](https://openalex.org/works?filter=author.id:A5063345981&sort=cited_by_count:desc) OpenAlex, 2026-10-14

[📄 6 works in our library](https://md.research.elamur.ai/free/en/authors/sayash-kapoor/#papers) · [🔗 Cited 291 times in our library](https://md.research.elamur.ai/free/en/authors/sayash-kapoor/cited-by/) · [16 more in our library without a page yet](#also) · [⭐ Paper Stars ranking: #7157](https://md.research.elamur.ai/free/en/paper-stars/page/72/#star-7157)

The profile is the library's match of the name; two people with one name are not always told apart.

Elsewhere: [Google Scholar](https://scholar.google.com/citations?view_op=search_authors&mauthors=Sayash+Kapoor) · [Semantic Scholar](https://www.semanticscholar.org/search?q=Sayash+Kapoor) · [dblp](https://dblp.org/search?q=Sayash+Kapoor) · [ACL Anthology](https://aclanthology.org/search/?q=Sayash+Kapoor)

## Affiliations

[🎓**Princeton University** Princeton · United States 2021–2026 · 37 papers*current*](https://md.research.elamur.ai/free/en/universities/princeton-university/) · [🎓**Indian Institute of Technology Kanpur** Kanpur · India 2018–2019 · 5 papers](https://md.research.elamur.ai/free/en/universities/indian-institute-of-technology-kanpur/) · [🎓**École Polytechnique Fédérale de Lausanne** Lausanne · Switzerland 2018 · 2 papers](https://md.research.elamur.ai/free/en/universities/ecole-polytechnique-federale-de-lausanne/)

From the bylines of their papers (OpenAlex, 2026-10-14), ORCID and Wikidata; institutions on papers are matched by machine.

## Fields, topics and concepts

Of its works in our library, with the number of works on each.

📚 Shelves 10 The field of science this work belongs to (OpenAlex fields).[Computer Science · 5 works](https://md.research.elamur.ai/free/en/shelves/computer-science/) · [Artificial Intelligence · 5 works](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · [Decision Sciences · 2 works](https://md.research.elamur.ai/free/en/shelves/decision-sciences/) · [Information Systems · 3 works](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · [Information Systems and Management · 2 works](https://md.research.elamur.ai/free/en/shelves/decision-sciences/information-systems-and-management/) · [Arts and Humanities · 2 works](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · [Safety Research · 2 works](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · [Social Sciences · 2 works](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · [Language and Linguistics · 2 works](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · [Software · 1 work](https://md.research.elamur.ai/free/en/shelves/computer-science/software/)

🧭 Topics 1 Research subjects and languages the work is about.[Evaluation & benchmarks · 3 works](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/)

## Works by year

### 2020s

[**2025** 2 papers](https://md.research.elamur.ai/free/en/authors/sayash-kapoor/year/2025/) · [**2024** 3 papers](https://md.research.elamur.ai/free/en/authors/sayash-kapoor/year/2024/) · [**2023** 1 paper](https://md.research.elamur.ai/free/en/authors/sayash-kapoor/year/2023/)

## Papers

The most cited first (OpenAlex and our library).

1. **[AI Agents That Matter](https://md.research.elamur.ai/free/en/papers/kapoor-2024-ai-agents-that-matter/)** *(2024 · 55 citations)* Benchmarks steer the development of LLM-based agents. The Princeton authors argue that current evaluation habits reward topping leaderboards rather than building systems that work in practice. They identify four problems:
2. **[CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark](https://md.research.elamur.ai/free/en/papers/siegel-2024-core-bench-fostering-credibility/)** *(2024 · 39 citations)* Many published studies cannot be rerun to obtain their reported numbers, even when the authors share code and data, and checking this by hand takes expert effort.
3. **[The Leaderboard Illusion](https://md.research.elamur.ai/free/en/papers/singh-2025-leaderboard-illusion/)** *(2025 · 22 citations)* Chatbot Arena has become the main public leaderboard for ranking LLMs through crowdsourced pairwise votes.
4. **[The Limits of Inference Scaling Through Resampling](https://md.research.elamur.ai/free/en/papers/stroebl-2024-limits-inference-scaling-through-resampling/)** *(2024 · 21 citations)* Resampling is a popular way to spend more compute at inference time: a model keeps generating candidates until one passes a verifier, such as unit tests.
5. **[REFORMS: Reporting Standards for Machine Learning Based Science](https://md.research.elamur.ai/free/en/papers/kapoor-2023-reforms-reporting-standards-machine/)** *(2023 · 19 citations)* Machine learning is spreading quickly across scientific disciplines, and so are problems with validity, reproducibility, and generalizability.
6. **[Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation](https://md.research.elamur.ai/free/en/papers/kapoor-2025-holistic-agent-leaderboard-missing/)** *(2025 · 11 citations)* Evaluating AI agents is slow, inconsistent and error-prone. Benchmarks can take weeks to run serially, costs are rarely reported, scaffolds are seldom compared, and agents may game tasks or act unsafely without detection. The authors built the Holistic Agent Leaderboard (HAL), an open-source harness. It accepts any…

## All works

22 works of theirs in our library, the most cited first; 6 with a page here, the others link their original until we summarise them.

1. **[The Foundation Model Transparency Index ↗](https://arxiv.org/abs/2310.12941) arXiv:2310.12941 · no summary yet**Rishi Bommasani et al.*2023 · 76 citations*
2. [**AI Agents That Matter arXiv:2407.01502**Sayash Kapoor, Benedikt Stroebl, Zachary S. Siegel, et al.*2024 · 55 citations*](https://md.research.elamur.ai/free/en/papers/kapoor-2024-ai-agents-that-matter/)
3. **[Foundation Model Transparency Reports ↗](https://doi.org/10.1609/aies.v7i1.31628) Proceedings of the AAAI/ACM Conference on AI Ethics and Society · no summary yet**Rishi Bommasani et al.*2024 · 53 citations*
4. **[International AI Safety Report ↗](https://arxiv.org/abs/2501.17805) arXiv (Cornell University) · no summary yet**Yoshua Bengio et al.*2025 · 41 citations*
5. **[Promises and pitfalls of artificial intelligence for legal applications ↗](https://arxiv.org/abs/2402.01656) arXiv:2402.01656 · no summary yet**Sayash Kapoor, Peter Henderson, Arvind Narayanan*2024 · 40 citations*
6. [**CORE-Bench: Fostering the Credibility of Published Research Through a Computational Reproducibility Agent Benchmark arXiv:2409.11363**Zachary S. Siegel, Sayash Kapoor, Nitya Nadgir, et al.*2024 · 39 citations*](https://md.research.elamur.ai/free/en/papers/siegel-2024-core-bench-fostering-credibility/)
7. **[Considerations for governing open foundation models ↗](https://doi.org/10.1126/science.adp1848) Science · no summary yet**Rishi Bommasani et al.*2024 · 36 citations*
8. **[On the Societal Impact of Open Foundation Models ↗](https://arxiv.org/abs/2403.07918) arXiv (Cornell University) · no summary yet**Sayash Kapoor et al.*2024 · 30 citations*
9. **[The Leaderboard Illusion](https://md.research.elamur.ai/free/en/papers/singh-2025-leaderboard-illusion/) [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/)**Shivalika Singh, Yiyang Nan, Alex Wang, et al.*2025 · 22 citations*
10. **[A Safe Harbor for AI Evaluation and Red Teaming ↗](https://arxiv.org/abs/2403.04893) arXiv (Cornell University) · no summary yet**Shayne Longpre et al.*2024 · 22 citations*
11. [**The Limits of Inference Scaling Through Resampling arXiv:2411.17501**Benedikt Stroebl, Sayash Kapoor, Arvind Narayanan*2024 · 21 citations*](https://md.research.elamur.ai/free/en/papers/stroebl-2024-limits-inference-scaling-through-resampling/)
12. [**REFORMS: Reporting Standards for Machine Learning Based Science arXiv:2308.07832**Sayash Kapoor, Emily Cantrell, Kenny Peng, et al.*2023 · 19 citations*](https://md.research.elamur.ai/free/en/papers/kapoor-2023-reforms-reporting-standards-machine/)
13. **[The Foundation Model Transparency Index v1.1: May 2024 ↗](https://arxiv.org/abs/2407.12929) arXiv (Cornell University) · no summary yet**Rishi Bommasani et al.*2024 · 14 citations*
14. [**Holistic Agent Leaderboard: The Missing Infrastructure for AI Agent Evaluation arXiv:2510.11977**Sayash Kapoor, Benedikt Stroebl, Peter Kirgis, et al.*2025 · 11 citations*](https://md.research.elamur.ai/free/en/papers/kapoor-2025-holistic-agent-leaderboard-missing/)
15. **[The Responsible Foundation Model Development Cheatsheet: A Review of Tools & Resources ↗](https://arxiv.org/abs/2406.16746) arXiv (Cornell University) · no summary yet**Shayne Longpre et al.*2024 · 11 citations*
16. **[The 2025 Foundation Model Transparency Index ↗](https://doi.org/10.70777/si.v2i4.17165) SuperIntelligence - Robotics - Safety & Alignment · no summary yet**Alexander Wan et al.*2026 · 9 citations*
17. **[Establishing Best Practices for Building Rigorous Agentic Benchmarks ↗](https://arxiv.org/abs/2507.02825) arXiv:2507.02825 · no summary yet**Yuxuan Zhu et al.*2025 · 7 citations*
18. **[Position: On the Societal Impact of Open Foundation Models ↗](https://openalex.org/W7135318688) ANU Open Research (Australian National University) · no summary yet**Sayash Kapoor et al.*2024 · 6 citations*
19. **[Establishing Best Practices in Building Rigorous Agentic Benchmarks ↗](https://doi.org/10.52202/085713-5547) neural information processing systems · no summary yet**Yuxuan Zhu et al.*2025 · 4 citations*
20. **[In-House Evaluation Is Not Enough: Towards Robust Third-Party Flaw Disclosure for General-Purpose AI ↗](https://arxiv.org/abs/2503.16861) arXiv (Cornell University) · no summary yet**Shayne Longpre et al.*2025 · 2 citations*
21. **[Log analysis is necessary for credible evaluation of AI agents ↗](https://arxiv.org/abs/2605.08545) arXiv (Cornell University) · no summary yet**Peter Kirgis et al.*2026 · 1 citation*
22. **[FLARE-AI: Flaw Reporting for AI ↗](https://arxiv.org/abs/2606.31567) arXiv (Cornell University) · no summary yet**Shayne Longpre et al.*2026*

## Co-authors here

12 of 19 co-authors with a page, the most papers together first.

- [Benedikt Stroebl](https://md.research.elamur.ai/free/en/authors/benedikt-stroebl/) ×4
- [Arvind Narayanan](https://md.research.elamur.ai/free/en/authors/arvind-narayanan/) ×3
- [Nitya Nadgir](https://md.research.elamur.ai/free/en/authors/nitya-nadgir/) ×3
- [Zachary S. Siegel](https://md.research.elamur.ai/free/en/authors/zachary-s-siegel/) ×3
- [Ahmet Üstün](https://md.research.elamur.ai/free/en/authors/ahmet-ustun/)
- [Alex Wang](https://md.research.elamur.ai/free/en/authors/alex-wang/)
- [Beyza Ermis](https://md.research.elamur.ai/free/en/authors/beyza-ermis/)
- [Boyi Wei](https://md.research.elamur.ai/free/en/authors/boyi-wei/)
- [Daniel D'Souza](https://md.research.elamur.ai/free/en/authors/daniel-d-souza/)
- [Jake M. Hofman](https://md.research.elamur.ai/free/en/authors/jake-m-hofman/)
- [Marzieh Fadaee](https://md.research.elamur.ai/free/en/authors/marzieh-fadaee/)
- [Noah A. Smith](https://md.research.elamur.ai/free/en/authors/noah-a-smith/)

## Venues

Where their works here were published, the most works first, with the editions of those that have a page.

- [Conference on Neural Information Processing Systems (NeurIPS)](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/): [NeurIPS 2025](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/2025/)
- ANU Open Research (Australian National University)
- Proceedings of the AAAI/ACM Conference on AI Ethics and Society
- Science
- SuperIntelligence - Robotics - Safety & Alignment
- neural information processing systems

## Cited by

Papers of our library cite their works 291 times (of the 149,439 papers whose bibliographies we read); [66 citing papers with a page here →](https://md.research.elamur.ai/free/en/authors/sayash-kapoor/cited-by/).

Updated 3 November 2026