---
title: "PolitNuggets — paper by Zhu (2026) — summary and ideas"
description: "PolitNuggets is a benchmark for a task that existing agent evaluations rarely cover. 25 references."
authors: ["Yifei Zhu"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.2085/"
doi: "10.18653/v1/2026.acl-long.2085"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhu-2026-politnuggets-benchmarking-agentic-discovery-long/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhu-2026-politnuggets-benchmarking-agentic-discovery-long/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/45/) / PolitNuggets: Benchmarking Agentic Discovery of Long-Tail Political Facts

# PolitNuggets: Benchmarking Agentic Discovery of Long-Tail Political Facts

**Authors:** Yifei Zhu · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.2085/) · [PDF](https://aclanthology.org/2026.acl-long.2085.pdf) · [DOI 10.18653/v1/2026.acl-long.2085](https://doi.org/10.18653/v1/2026.acl-long.2085) · [OpenAlex](https://openalex.org/W7166850935)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
PolitNuggets is a benchmark for a task that existing agent evaluations rarely cover. An LLM agent has to search the open web and assemble rare, scattered biographical facts into a coherent, time-resolved career history. The author calls this "reasoning through context," as opposed to reading a fixed context. The benchmark covers 400 political elites with more than 10,000 facts. Half are non-US cabinet members drawn from WhoGov and half are US legislators, so the evidence is often non-English.

Evaluation uses a standardized Supervisor–Searcher multi-agent system, with a Coder agent that outputs the final JSON. FactNet, an evidence-conditional judging protocol, scores discovery, fine-grained accuracy and efficiency.

Agents tend to be precise but miss many facts. Removing Wikipedia as a starting point raises search cost considerably, and non-US figures are covered noticeably worse. Agent success depends on pulling information out of brief passages, coping well with many languages and calling tools dependably, not on skill with long inputs. These findings point to which model capabilities matter for practical multilingual research agents.

## Key points
- **Dataset:** 200 non-US cabinet politicians sampled from WhoGov (over 58,000 cabinet members, 1966–2023) plus 200 US legislators. Scoring uses Event-Level F1 (role, organization and year) and Attribute-Level F1 (months within ±1 month and exact official title).
- **Models tested:** Grok-4-Fast, Gemini-2.5-Flash and Qwen-3 (80B/225B). These were chosen using MRCR and BFCL v3 scores plus cost.
- **Two settings:** "with Wiki" (Enhancement: fill gaps starting from the Wikipedia text) and "without Wiki" (Reconstruction: start from the name only).
- **International Evidence Gap:** non-US entities show up to about a 40% relative F1 drop. An ablation shows that removing the Archive evidence store lowers Event-Level F1 by roughly 0.05.
- **"Long-Context Paradox":** long-context reading scores do not reliably predict agent performance. Closed-book parametric knowledge and BFCL tool-use reliability correlate positively with it.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Hong Kong](https://md.research.elamur.ai/free/en/universities/university-of-hong-kong/) — University, Pok Fu Lam, China · 118 works · 13,496 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works

## Cite

Cite as: Yifei Zhu. 2026. PolitNuggets: Benchmarking Agentic Discovery of Long-Tail Political Facts. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhu2026politnuggets,
  title = {{PolitNuggets: Benchmarking Agentic Discovery of Long-Tail Political Facts}},
  author = {Yifei Zhu},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.2085},
  url = {https://doi.org/10.18653/v1/2026.acl-long.2085}
}
```

- [← previous Polite but Misaligned: Evaluating LLM Politeness Judgments Against Human Pragmatic Norms](https://md.research.elamur.ai/free/en/papers/wang-2026-polite-but-misaligned-evaluating-llm/)
- [next → Poller: Are LLMs Suitable for Evaluating Poetry Understanding Task?](https://md.research.elamur.ai/free/en/papers/wang-2026-poller-are-llms-suitable-evaluating/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/45/)

## References

25 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Chenxin An, Shansan Gong, Ming Zhong, Xingjian Zhao, Mukai Li, Jun Zhang, Lingpeng Kong, and Xipeng Qiu. 2024. [L-Eval: Instituting Standardized Evaluation for Long Context Language Models](https://md.research.elamur.ai/free/en/papers/an-2023-l-eval-instituting-standardized-evaluation/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (ACL), pages 14388–1…
2. Yushi Bai, Shangqing Tu, Jiajie Zhang, Hao Peng, Xiaozhi Wang, Xin Lv, Shulin Cao, Jiazheng Xu, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2025. [Longbench v2: towards deeper understanding and reasoning on realistic long-context multitasks](https://md.research.elamur.ai/free/en/papers/bai-2024-longbench-v2-deeper-understanding/). In Proceedings of the 63rd Annual Meeting of the Associa…
3. Gheorghe Comanici, Eric Bieber, Mike Schaekermann, Ice Pasupat, and 1 others. 2025. [Gemini 2.5: Pushing the Frontier with Advanced Reasoning, Multimodality, Long Context, and Next Generation Agentic Capabilities](https://md.research.elamur.ai/free/en/papers/comanici-2025-gemini-2-5-pushing-frontier/). arXiv preprint arXiv:2507.06261.
4. Mingxuan Du, Benfeng Xu, Chiwei Zhu, Xiaorui Wang, and Zhendong Mao. 2025. [Deepresearch bench: a comprehensive benchmark for deep research agents](https://md.research.elamur.ai/free/en/papers/du-2025-deepresearch-bench-comprehensive-benchmark-deep/). Preprint, arXiv:2506.11763.
5. Kelvin Guu, Kenton Lee, Zora Tung, Panupong Pasupat, and Ming-Wei Chang. 2020. [Retrieval augmented language model pre-training](https://md.research.elamur.ai/free/en/papers/guu-2020-realm-retrieval-augmented-language-model/). In Proceedings of the 37th International Conference on Machine Learning (ICML), volume 119 of Proceedings of Machine Learning Research, pages 3929–3938. PMLR.
6. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). In Advances in Neural Information Proc…
7. Kuan Li, Zhongwang Zhang, Huifeng Yin, Liwen Zhang, Litu Ou, Jialong Wu, Wenbiao Yin, Baixuan Li, Zhengwei Tao, Xinyu Wang, Weizhou Shen, Junkai Zhang, Dingchu Zhang, Xixi Wu, Yong Jiang, Ming Yan, Pengjun Xie, Fei Huang, and Jingren Zhou. 2025. Websailor: navigating super-human reasoning for web ag… [find](https://scholar.google.com/scholar?q=Kuan+Li%2C+Zhongwang+Zhang%2C+Huifeng+Yin%2C+Liwen+Zhang%2C+Litu+Ou%2C+Jialong+Wu%2C+Wenbiao+Yin%2C+Baixuan+Li%2C+Zhengwei+Tao%2C+Xinyu+Wang%2C+Weizhou+Shen%2C+Junkai+Zhang%2C+Dingchu+Zhang%2C+Xixi+Wu%2C+Yong+Jiang%2C+Ming+Yan%2C+Pe)
8. Grégoire Mialon, Clémentine Fourrier, Craig Swift, Thomas Wolf, Yann LeCun, and Thomas Scialom. 2024. [GAIA: a benchmark for general AI assistants](https://md.research.elamur.ai/free/en/papers/mialon-2023-gaia-benchmark-general-ai-assistants/). In The Twelfth International Conference on Learning Representations (ICLR). Accessed: 2026-01-06.
9. Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, Xu Jiang, Karl Cobbe, Tyna Eloundou, Gretchen Krueger, Kevin Button, Matthew Knight, Benjamin Chess, and John Schulman. 2021. [WebGPT: Browser-assis…](https://md.research.elamur.ai/free/en/papers/nakano-2021-webgpt-browser-assisted-question-answering/)
10. OpenAI. 2025a. GraphWalks: a multi-hop long-context graph reasoning benchmark. [https://huggingface.co/datasets/openai/graphwalks](https://huggingface.co/datasets/openai/graphwalks). Released with GPT-4.1; results reported in the GPT-4.1 blog post. [↗](https://huggingface.co/datasets/openai/graphwalks)
11. OpenAI. 2025b. Introducing deep research. [https://openai.com/index/](https://openai.com/index/) introducing-deep-research/. OpenAI release, published February 2, 2025. [↗](https://openai.com/index/)
12. OpenAI. 2025c. OpenAI-MRCR (Multi-Round Coreference). [https://huggingface.co/datasets/](https://huggingface.co/datasets/) openai/mrcr. OpenAI dataset/eval introduced in the GPT-4.1 release on April 14, 2025; results used in this paper are provided in final_eval/mrcr_score.csv. [↗](https://huggingface.co/datasets/)
13. Shishir G. Patil, Huanzhi Mao, Fanjia Yan, Charlie Cheng-Jie Ji, Vishnu Suresh, Ion Stoica, and Joseph E. Gonzalez. 2025. The berkeley function calling leaderboard (bfcl): from tool use to agentic evaluation of large language models. In Forty-second International Conference on Machine Learning. Open… [find](https://scholar.google.com/scholar?q=Shishir+G%2E+Patil%2C+Huanzhi+Mao%2C+Fanjia+Yan%2C+Charlie+Cheng%2DJie+Ji%2C+Vishnu+Suresh%2C+Ion+Stoica%2C+and+Joseph+E%2E+Gonzalez%2E+2025%2E+The+berkeley+function+calling+leaderboard+%28bfcl%29%3A+from+tool+use+to+agentic+eva)
14. Perplexity AI. 2025. Introducing Perplexity Deep Research. [https://www.perplexity.ai/hub/blog/](https://www.perplexity.ai/hub/blog/) introducing-perplexity-deep-research. Perplexity AI Blog. [↗](https://www.perplexity.ai/hub/blog/)
15. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. [Toolformer: Language Models Can Teach Themselves to Use Tools](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/). In Advances in Neural Information Processing Systems (NeurIPS).
16. Uri Shaham, Maor Ivgi, Avia Efrat, Jonathan Berant, and Omer Levy. 2023. [ZeroSCROLLS: A Zero-Shot Benchmark for Long Text Understanding](https://md.research.elamur.ai/free/en/papers/shaham-2023-zeroscrolls-zero-shot-benchmark-long/). In Findings of the Association for Computational Linguistics: EMNLP 2023.
17. Harsh Trivedi, Niranjan Balasubramanian, Tushar Khot, and Ashish Sabharwal. 2022. MuSiQue: multi-hop questions via single-hop question composition. Transactions of the Association for Computational Linguistics, 10:539–554. [find](https://scholar.google.com/scholar?q=Harsh+Trivedi%2C+Niranjan+Balasubramanian%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+2022%2E+MuSiQue%3A+multi%2Dhop+questions+via+single%2Dhop+question+composition%2E+Transactions+of+the+Association+for+Computational+Li)
18. Kiran Vodrahalli, Santiago Ontanon, Nilesh Tripuraneni, Kelvin Xu, Sanil Jain, Rakesh Shivanna, Jeffrey Hui, Nishanth Dikkala, Mehran Kazemi, Bahare Fatemi, Rohan Anil, Ethan Dyer, Siamak Shakeri, Roopali Vij, Harsh Mehta, Vinay Ramasesh, Quoc Le, Ed Chi, Yifeng Lu, and 5 others. 2024. [Michelangelo:…](https://md.research.elamur.ai/free/en/papers/vodrahalli-2024-michelangelo-long-context-evaluations-haystacks/)
19. Jason Wei, Zhiqing Sun, Spencer Papay, Scott McKinney, Jeffrey Han, Isa Fulford, Hyung Won Chung, Alex Tachard Passos, William Fedus, and Amelia Glaese. 2025. [BrowseComp: a simple yet challenging benchmark for browsing agents](https://md.research.elamur.ai/free/en/papers/wei-2025-browsecomp-simple-yet-challenging-benchmark/). Preprint, arXiv:2504.12516. xAI. 2025. Grok 4 Fast Model Card. [https://x.…](https://x.…) [↗](https://x.…)
20. An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, and 1 others. 2025a. [Qwen3 technical report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388.
21. Yijun Yang, Zeyu Huang, Wenhao Zhu, Zihan Qiu, Fei Yuan, Jeff Z. Pan, and Ivan Titov. 2025b. A controllable examination for long-context language models. In The Thirty-ninth Annual Conference on Neural Information Processing Systems Datasets and Benchmarks Track. [find](https://scholar.google.com/scholar?q=Yijun+Yang%2C+Zeyu+Huang%2C+Wenhao+Zhu%2C+Zihan+Qiu%2C+Fei+Yuan%2C+Jeff+Z%2E+Pan%2C+and+Ivan+Titov%2E+2025b%2E+A+controllable+examination+for+long%2Dcontext+language+models%2E+In+The+Thirty%2Dninth+Annual+Conference+on+Neura)
22. Shunyu Yao, Howard Chen, John Yang, and Karthik Narasimhan. 2022. [WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents](https://md.research.elamur.ai/free/en/papers/yao-2022-webshop-towards-scalable-real-world/). In Advances in Neural Information Processing Systems (NeurIPS).
23. Howard Yen, Tianyu Gao, Minmin Hou, Ke Ding, Daniel Fleischer, Peter Izsak, Moshe Wasserblat, and Danqi Chen. 2025. Helmet: How to evaluate long-context models effectively and thoroughly. In The Thirteenth International Conference on Learning Representations. OpenReview: [https://openreview.net/forum…](https://openreview.net/forum…) [↗](https://openreview.net/forum…)
24. Xinrong Zhang, Yingfa Chen, Shengding Hu, Zihang Xu, Junhao Chen, Moo Khai Hao, Xu Han, Zhen Leng Thai, Shuo Wang, Zhiyuan Liu, and Maosong Sun. 2024. inf tyBench: Extending Long Context Evaluation Beyond 100K Tokens. In Proceedings of the 62nd Annual Meeting of the Association for Computational Lin… [find](https://scholar.google.com/scholar?q=Xinrong+Zhang%2C+Yingfa+Chen%2C+Shengding+Hu%2C+Zihang+Xu%2C+Junhao+Chen%2C+Moo+Khai+Hao%2C+Xu+Han%2C+Zhen+Leng+Thai%2C+Shuo+Wang%2C+Zhiyuan+Liu%2C+and+Maosong+Sun%2E+2024%2E+inf+tyBench%3A+Extending+Long+Context+Evaluation+Be)
25. Shuyan Zhou, Frank F. Xu, Hao Zhu, Xuhui Zhou, Robert Lo, Abishek Sridhar, Xianyi Cheng, Yonatan Bisk, Daniel Fried, Uri Alon, and Graham Neubig. 2024. [WebArena: A Realistic Web Environment for Building Autonomous Agents](https://md.research.elamur.ai/free/en/papers/zhou-2023-webarena-realistic-web-environment-building/). In The Twelfth International Conference on Learning Representations (ICLR). \#\#…

*Updated 2 November 2026*