---
title: "PseudoBench — paper by Liao et al. (2026) — summary and ideas"
description: "The authors introduce PseudoBench, an adversarial benchmark that tests whether these agents push… 73 references, cited by 1 paper here."
authors: ["Xinyang Liao", "Lingyu Li", "Huacan Liu", "Tianle Gu", "Yang Yao", "Tong Zhu", "Yan Teng", "Yingchun Wang"]
year: 2026
venue: "arXiv:2606.18060"
source: "https://arxiv.org/abs/2606.18060"
doi: "10.48550/arxiv.2606.18060"
arxiv: "2606.18060"
licence: "arXiv non-exclusive distribution license"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liao-2026-pseudobench-measuring-agentic-auto/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liao-2026-pseudobench-measuring-agentic-auto/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/16/) / PseudoBench: Measuring How Agentic Auto-Research Fuels Pseudoscience

# PseudoBench: Measuring How Agentic Auto-Research Fuels Pseudoscience

**Authors:** Xinyang Liao, Lingyu Li, Huacan Liu, [Tianle Gu](https://md.research.elamur.ai/free/en/authors/tianle-gu/), [Yang Yao](https://md.research.elamur.ai/free/en/authors/yang-yao/), [Tong Zhu](https://md.research.elamur.ai/free/en/authors/tong-zhu/), [Yan Teng](https://md.research.elamur.ai/free/en/authors/yan-teng/) and [Yingchun Wang](https://md.research.elamur.ai/free/en/authors/yingchun-wang/) · **Year:** 2026 · **Venue:** arXiv:2606.18060 · **Licence:** [arXiv non-exclusive distribution license](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2606.18060) · [PDF](https://arxiv.org/pdf/2606.18060) · [DOI 10.48550/arxiv.2606.18060](https://doi.org/10.48550/arxiv.2606.18060) · [arXiv 2606.18060](https://arxiv.org/abs/2606.18060) · [OpenAlex](https://openalex.org/W7165059480)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/liao-2026-pseudobench-measuring-agentic-auto/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
LLM-based agents are increasingly used for autonomous end-to-end research. This raises a risk: such agents could quickly produce credible-looking studies built on pseudoscientific ideas and pollute the scientific record. The authors introduce PseudoBench, an adversarial benchmark that tests whether these agents push back against pseudoscience or simply build on it.

They gathered raw material from Wikipedia's pseudoscience entries and the Chinese MinKe forum on Baidu Tieba. They then filtered, standardized, deduplicated and scored it with LLMs, and human annotators verified the result. This produced 200 "not even wrong" claim-evidence pairs in five domains. Each agent receives one pair and an isolated workspace, and must plan, run experiments, analyze results and write a paper-style PDF. Judges rate the PDF on Report Quality, Pseudoscience Alignment and Persuasiveness.

Across seven agents, almost none refused, and all generated polished reports that backed the false premises. More capable agents tended to make pseudoscience sound more scientific. The authors argue that scientific alignment is needed before such systems are widely deployed.

## Key points
- Data pipeline:
  - 8,484 raw items were collected.
  - First-pass filtering with DeepSeek-V3.2 left 4,016.
  - Standardization left 3,697.
  - Deduplication with Qwen3-Embedding-8B (cosine threshold 0.7) left 1,271.
  - Rescoring, stratified sampling and rewriting with Claude Sonnet 4.6 produced the final 200.
- The five categories include Fundamental Physics and Cosmology (120 items, 60%) and Consciousness, Soul, and Mystic Energy.
- Seven agents were tested:
  - General-purpose: Codex, Claude Code, OpenClaw and Nanobot.
  - Science-oriented: EvoScientist, ResearchClaw and ARIS.
- Refusal rates were close to zero, and the best overall resistance score was only 27.4%.
- To limit training-data contamination, only 200 of the 1,271 pairs are public. Code and data are at github.com/AI45Lab/PseudoBench.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Shanghai Jiao Tong University](https://md.research.elamur.ai/free/en/universities/shanghai-jiao-tong-university/) — University, Shanghai, China · 232 works · 20,305 citations
- [Shanghai Artificial Intelligence Laboratory](https://md.research.elamur.ai/free/en/labs/shanghai-artificial-intelligence-laboratory/) — Research institute, Shanghai, China · 164 works · 13,544 citations
- [Xi'an Jiaotong University](https://md.research.elamur.ai/free/en/universities/xi-an-jiaotong-university/) — University, Xi'an, China · 43 works · 36,432 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 2,009 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 665 works · [🤝 Sociology and Political Science](https://md.research.elamur.ai/free/en/shelves/social-sciences/sociology-and-political-science/) · 630 works

## Cite

Cite as: Xinyang Liao, Lingyu Li, Huacan Liu, Tianle Gu, Yang Yao, Tong Zhu, Yan Teng and Yingchun Wang. 2026. PseudoBench: Measuring How Agentic Auto-Research Fuels Pseudoscience. arXiv:2606.18060.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{liao2026pseudobench,
  title = {{PseudoBench: Measuring How Agentic Auto-Research Fuels Pseudoscience}},
  author = {Xinyang Liao, Lingyu Li, Huacan Liu, Tianle Gu, Yang Yao, Tong Zhu, Yan Teng and Yingchun Wang},
  year = {2026},
  doi = {10.48550/arxiv.2606.18060},
  eprint = {2606.18060},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2606.18060}
}
```

- [← previous ProxyPrompt: Securing System Prompts against Prompt Extraction Attacks](https://md.research.elamur.ai/free/en/papers/zhuang-2026-proxyprompt-securing-system-prompts-against/)
- [next → PsychEval: A Multi-Session and Multi-Therapy Benchmark for High-Realism AI Psychological Counselor](https://md.research.elamur.ai/free/en/papers/pan-2026-psycheval-multi-session-multi-therapy/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/16/)

## References

73 references, 18 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Mohamad Abou Ali, Fadi Dornaika, and Jinan Charafed-dine. 2025. Agentic ai: a comprehensive survey of architectures, applications, and future directions. Artificial Intelligence Review, 59(1):11. [find](https://scholar.google.com/scholar?q=Mohamad+Abou+Ali%2C+Fadi+Dornaika%2C+and+Jinan+Charafed%2Ddine%2E+2025%2E+Agentic+ai%3A+a+comprehensive+survey+of+architectures%2C+applications%2C+and+future+directions%2E+Artificial+Intelligence+Review%2C+59%281%29%3A11%2E)
2. Deepak Bhaskar Acharya, Karthigeyan Kuppan, and B Divya. 2025. Agentic ai: Autonomous intelligence for complex goals—a comprehensive survey. IEEe Access, 13:18912–18936. [find](https://scholar.google.com/scholar?q=Deepak+Bhaskar+Acharya%2C+Karthigeyan+Kuppan%2C+and+B+Divya%2E+2025%2E+Agentic+ai%3A+Autonomous+intelligence+for+complex+goals%E2%80%94a+comprehensive+survey%2E+IEEe+Access%2C+13%3A18912%E2%80%9318936%2E)
3. Aisha Alansari and Hamzah Luqman. 2025. Large language models hallucination: A comprehensive survey. arXiv preprint arXiv:2510.06265. — [arXiv:2510.06265](https://arxiv.org/abs/2510.06265) [find](https://scholar.google.com/scholar?q=Aisha+Alansari+and+Hamzah+Luqman%2E+2025%2E+Large+language+models+hallucination%3A+A+comprehensive+survey%2E+arXiv+preprint+arXiv%3A2510%2E06265%2E+%E2%80%94+%5BarXiv%3A2510%2E06265%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2510%2E06265%29)
4. Mel Andrews, Andrew Smart, and Abeba Birhane. 2024. The reanimation of pseudoscience in machine learning and its ethical repercussions. Patterns, 5(9). [find](https://scholar.google.com/scholar?q=Mel+Andrews%2C+Andrew+Smart%2C+and+Abeba+Birhane%2E+2024%2E+The+reanimation+of+pseudoscience+in+machine+learning+and+its+ethical+repercussions%2E+Patterns%2C+5%289%29%2E)
5. Anthropic. 2026a. Claude code. [https://www.anthropic.com/product/claude-code](https://www.anthropic.com/product/claude-code). [↗](https://www.anthropic.com/product/claude-code)
6. Anthropic. 2026b. Claude opus 4.6. [https://www.anthropic.com/news/claude-opus-4-6](https://www.anthropic.com/news/claude-opus-4-6). [↗](https://www.anthropic.com/news/claude-opus-4-6)
7. Anthropic. 2026c. Claude opus 4.7. [https://www.anthropic.com/news/claude-opus-4-7](https://www.anthropic.com/news/claude-opus-4-7). [↗](https://www.anthropic.com/news/claude-opus-4-7)
8. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, and 1 others. 2022. [Constitutional ai: Harmlessness from ai feedback](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/). arXiv preprint arXiv:2212.08073.
9. Zechen Bai, Pichao Wang, Tianjun Xiao, Tong He, Zongbo Han, Zheng Zhang, and Mike Zheng Shou. 2024. [Hallucination of multimodal large language models: A survey](https://md.research.elamur.ai/free/en/papers/bai-2024-hallucination-multimodal-large-language-models/). arXiv preprint arXiv:2404.18930.
10. Baidu Tieba. 2026. Minke bar. [https://tieba.baidu.com/f?kw=%E6%B0%91%E7%A7%91](https://tieba.baidu.com/f?kw=%E6%B0%91%E7%A7%91). [↗](https://tieba.baidu.com/f?kw=%E6%B0%91%E7%A7%91)
11. Stefanos Balaskas. 2026. From recommendations to delegation: A systematic review mapping agentic ai in e-commerce and its consumer effects. Information, 17(3):222. [find](https://scholar.google.com/scholar?q=Stefanos+Balaskas%2E+2026%2E+From+recommendations+to+delegation%3A+A+systematic+review+mapping+agentic+ai+in+e%2Dcommerce+and+its+consumer+effects%2E+Information%2C+17%283%29%3A222%2E)
12. Ajay Bandi, Bhavani Kongari, Roshini Naguru, Sahitya Pasnoor, and Sri Vidya Vilipala. 2025. The rise of agentic ai: A review of definitions, frameworks, architectures, applications, evaluation metrics, and challenges. Future Internet, 17(9):404. [find](https://scholar.google.com/scholar?q=Ajay+Bandi%2C+Bhavani+Kongari%2C+Roshini+Naguru%2C+Sahitya+Pasnoor%2C+and+Sri+Vidya+Vilipala%2E+2025%2E+The+rise+of+agentic+ai%3A+A+review+of+definitions%2C+frameworks%2C+architectures%2C+applications%2C+evaluation+metrics)
13. Saikat Barua. 2024. Exploring autonomous agents through the lens of large language models: A review. arXiv preprint arXiv:2404.04442. — [arXiv:2404.04442](https://arxiv.org/abs/2404.04442) [find](https://scholar.google.com/scholar?q=Saikat+Barua%2E+2024%2E+Exploring+autonomous+agents+through+the+lens+of+large+language+models%3A+A+review%2E+arXiv+preprint+arXiv%3A2404%2E04442%2E+%E2%80%94+%5BarXiv%3A2404%2E04442%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2404%2E04442%29)
14. Kat Boboris. 2025. Attention authors: Updated practice for review articles and position papers in arxiv cs category. [find](https://scholar.google.com/scholar?q=Kat+Boboris%2E+2025%2E+Attention+authors%3A+Updated+practice+for+review+articles+and+position+papers+in+arxiv+cs+category%2E)
15. María Victoria Carro. 2024. Flattering to deceive: The impact of sycophantic behavior on user trust in large language model. arXiv preprint arXiv:2412.02802. — [arXiv:2412.02802](https://arxiv.org/abs/2412.02802) [find](https://scholar.google.com/scholar?q=Mar%C3%ADa+Victoria+Carro%2E+2024%2E+Flattering+to+deceive%3A+The+impact+of+sycophantic+behavior+on+user+trust+in+large+language+model%2E+arXiv+preprint+arXiv%3A2412%2E02802%2E+%E2%80%94+%5BarXiv%3A2412%2E02802%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs)
16. Dalmeet Singh Chawla. 2026. Researchers who use hallucinated references to face arxiv ban. Nature. [find](https://scholar.google.com/scholar?q=Dalmeet+Singh+Chawla%2E+2026%2E+Researchers+who+use+hallucinated+references+to+face+arxiv+ban%2E+Nature%2E)
17. Myra Cheng, Cinoo Lee, Pranav Khadpe, Sunny Yu, Dyllan Han, and Dan Jurafsky. 2026. Sycophantic ai decreases prosocial intentions and promotes dependence. Science, 391(6792):eaec8352. [find](https://scholar.google.com/scholar?q=Myra+Cheng%2C+Cinoo+Lee%2C+Pranav+Khadpe%2C+Sunny+Yu%2C+Dyllan+Han%2C+and+Dan+Jurafsky%2E+2026%2E+Sycophantic+ai+decreases+prosocial+intentions+and+promotes+dependence%2E+Science%2C+391%286792%29%3Aeaec8352%2E)
18. Manuel Cossio. 2025. A comprehensive taxonomy of hallucinations in large language models. arXiv preprint arXiv:2508.01781. — [arXiv:2508.01781](https://arxiv.org/abs/2508.01781) [find](https://scholar.google.com/scholar?q=Manuel+Cossio%2E+2025%2E+A+comprehensive+taxonomy+of+hallucinations+in+large+language+models%2E+arXiv+preprint+arXiv%3A2508%2E01781%2E+%E2%80%94+%5BarXiv%3A2508%2E01781%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2508%2E01781%29)
19. Yogesh K Dwivedi, Mohamed YI Helal, Ibrahim A El-gendy, Rasha Alahmad, Paul Walton, Ayoung Suh, Vinay Singh, and Il Jeon. 2026. Agentic ai systems: What it is and isn’t. Global Business and Organizational Excellence, 45(3):253–263. [find](https://scholar.google.com/scholar?q=Yogesh+K+Dwivedi%2C+Mohamed+YI+Helal%2C+Ibrahim+A+El%2Dgendy%2C+Rasha+Alahmad%2C+Paul+Walton%2C+Ayoung+Suh%2C+Vinay+Singh%2C+and+Il+Jeon%2E+2026%2E+Agentic+ai+systems%3A+What+it+is+and+isn%E2%80%99t%2E+Global+Business+and+Organizati)
20. Ali Essam Ghareeb, Benjamin Chang, Ludovico Mitchener, Angela Yiu, Caralyn J Szostkiewicz, Dmytro Shved, Gavin J Gyimesi, Jon M Laurent, Samantha M Wright, Muhammed T Razzak, and 1 others. 2026. A multi-agent system for automating scientific discovery. Nature, pages 1–3. [find](https://scholar.google.com/scholar?q=Ali+Essam+Ghareeb%2C+Benjamin+Chang%2C+Ludovico+Mitchener%2C+Angela+Yiu%2C+Caralyn+J+Szostkiewicz%2C+Dmytro+Shved%2C+Gavin+J+Gyimesi%2C+Jon+M+Laurent%2C+Samantha+M+Wright%2C+Muhammed+T+Razzak%2C+and+1+others%2E+2026%2E+A+mul)
21. Elizabeth Gibney. 2026. Hey chatgpt, write me a fictional paper: these llms are willing to commit academic fraud. Nature, 651(8105):286–287. [find](https://scholar.google.com/scholar?q=Elizabeth+Gibney%2E+2026%2E+Hey+chatgpt%2C+write+me+a+fictional+paper%3A+these+llms+are+willing+to+commit+academic+fraud%2E+Nature%2C+651%288105%29%3A286%E2%80%93287%2E)
22. Gabriel R Gonzalez, Johannes Habel, and Gary K Hunter. 2026. Ai agents, agentic ai, and the future of sales. Journal of Business Research, 202:115799. [find](https://scholar.google.com/scholar?q=Gabriel+R+Gonzalez%2C+Johannes+Habel%2C+and+Gary+K+Hunter%2E+2026%2E+Ai+agents%2C+agentic+ai%2C+and+the+future+of+sales%2E+Journal+of+Business+Research%2C+202%3A115799%2E)
23. Mourad Gridach, Jay Nanavati, Khaldoun Zine El Abidine, Lenon Mendes, and Christina Mack. 2025. Agentic ai for scientific discovery: A survey of progress, challenges, and future directions. arXiv preprint arXiv:2503.08979. — [arXiv:2503.08979](https://arxiv.org/abs/2503.08979) [find](https://scholar.google.com/scholar?q=Mourad+Gridach%2C+Jay+Nanavati%2C+Khaldoun+Zine+El+Abidine%2C+Lenon+Mendes%2C+and+Christina+Mack%2E+2025%2E+Agentic+ai+for+scientific+discovery%3A+A+survey+of+progress%2C+challenges%2C+and+future+directions%2E+arXiv+prep)
24. Jutta Haider, Kristofer Rolf Söderström, Björn Ekström, and Malte Rödl. 2024. Gpt-fabricated scientific papers on google scholar: Key features, spread, and implications for preempting evidence manipulation. Harvard Kennedy School Misinformation Review, 5(5). [find](https://scholar.google.com/scholar?q=Jutta+Haider%2C+Kristofer+Rolf+S%C3%B6derstr%C3%B6m%2C+Bj%C3%B6rn+Ekstr%C3%B6m%2C+and+Malte+R%C3%B6dl%2E+2024%2E+Gpt%2Dfabricated+scientific+papers+on+google+scholar%3A+Key+features%2C+spread%2C+and+implications+for+preempting+evidence+manipul)
25. Thomas Hartung. 2025. Ai, agentic models and lab automation for scientific discovery—the beginning of scaince. Frontiers in Artificial Intelligence, 8:1649155. [find](https://scholar.google.com/scholar?q=Thomas+Hartung%2E+2025%2E+Ai%2C+agentic+models+and+lab+automation+for+scientific+discovery%E2%80%94the+beginning+of+scaince%2E+Frontiers+in+Artificial+Intelligence%2C+8%3A1649155%2E)
26. HKUDS. 2026. nanobot: Lightweight, open-source ai agent for your tools, chats, and workflows. [https://github.com/HKUDS/nanobot](https://github.com/HKUDS/nanobot). [↗](https://github.com/HKUDS/nanobot)
27. Soodeh Hosseini and Hossein Seilani. 2025. The role of agentic ai in shaping a smart future: A systematic review. Array, 26:100399. [find](https://scholar.google.com/scholar?q=Soodeh+Hosseini+and+Hossein+Seilani%2E+2025%2E+The+role+of+agentic+ai+in+shaping+a+smart+future%3A+A+systematic+review%2E+Array%2C+26%3A100399%2E)
28. Lei Huang, Weijiang Yu, Weitao Ma, Weihong Zhong, Zhangyin Feng, Haotian Wang, Qianglong Chen, Weihua Peng, Xiaocheng Feng, Bing Qin, and 1 others. 2025. [A survey on hallucination in large language models: Principles, taxonomy, challenges, and open questions](https://md.research.elamur.ai/free/en/papers/huang-2023-survey-hallucination-large-language-models/). ACM Transactions on Information Systems,…
29. Yichong Huang, Xiachong Feng, Xiaocheng Feng, and Bing Qin. 2021. The factual inconsistency problem in abstractive text summarization: A survey. arXiv preprint arXiv:2104.14839. — [arXiv:2104.14839](https://arxiv.org/abs/2104.14839) [find](https://scholar.google.com/scholar?q=Yichong+Huang%2C+Xiachong+Feng%2C+Xiaocheng+Feng%2C+and+Bing+Qin%2E+2021%2E+The+factual+inconsistency+problem+in+abstractive+text+summarization%3A+A+survey%2E+arXiv+preprint+arXiv%3A2104%2E14839%2E+%E2%80%94+%5BarXiv%3A2104%2E14839%5D%28h)
30. Lujain Ibrahim, Franziska Sofia Hafner, Myra Cheng, Cinoo Lee, Rebecca Anselmetti, Robb Willer, Luc Rocher, and Diyi Yang. 2026. Sycophantic ai makes human interaction feel more effortful and less satisfying over time. arXiv preprint arXiv:2605.07912. [find](https://scholar.google.com/scholar?q=Lujain+Ibrahim%2C+Franziska+Sofia+Hafner%2C+Myra+Cheng%2C+Cinoo+Lee%2C+Rebecca+Anselmetti%2C+Robb+Willer%2C+Luc+Rocher%2C+and+Diyi+Yang%2E+2026%2E+Sycophantic+ai+makes+human+interaction+feel+more+effortful+and+less+sat)
31. Jason Jabbour and Vijay Janapa Reddi. 2024. Generative ai agents in autonomous machines: A safety perspective. In Proceedings of the 43rd IEEE/ACM International Conference on Computer-Aided Design, pages 1–13. [find](https://scholar.google.com/scholar?q=Jason+Jabbour+and+Vijay+Janapa+Reddi%2E+2024%2E+Generative+ai+agents+in+autonomous+machines%3A+A+safety+perspective%2E+In+Proceedings+of+the+43rd+IEEE%2FACM+International+Conference+on+Computer%2DAided+Design%2C+pa)
32. Ziwei Ji, Nayeon Lee, Rita Frieske, Tiezheng Yu, Dan Su, Yan Xu, Etsuko Ishii, Ye Jin Bang, Andrea Madotto, and Pascale Fung. 2023. [Survey of hallucination in natural language generation](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/). ACM computing surveys, 55(12):1–38.
33. Nalan Karunanayake. 2025. Next-generation agentic ai for transforming healthcare. Informatics and Health, 2(2):73–83. [find](https://scholar.google.com/scholar?q=Nalan+Karunanayake%2E+2025%2E+Next%2Dgeneration+agentic+ai+for+transforming+healthcare%2E+Informatics+and+Health%2C+2%282%29%3A73%E2%80%9383%2E)
34. Shaista Khalid, Azmat Islam, and Muhammad Ajmal. 2025. The end of human-only knowledge management: Agentic ai in education. Journal of Management Science Research Review, 4(4):2024–2042. [find](https://scholar.google.com/scholar?q=Shaista+Khalid%2C+Azmat+Islam%2C+and+Muhammad+Ajmal%2E+2025%2E+The+end+of+human%2Donly+knowledge+management%3A+Agentic+ai+in+education%2E+Journal+of+Management+Science+Research+Review%2C+4%284%29%3A2024%E2%80%932042%2E)
35. Jaeho Kim, Yunseok Lee, and Seulki Lee. 2025. [Position: The ai conference peer review crisis demands author feedback and reviewer rewards](https://md.research.elamur.ai/free/en/papers/kim-2025-position-ai-conference-peer-review/). arXiv preprint arXiv:2505.04966.
36. Georgios Kostopoulos, Vasileios Gkamas, Maria Rigou, and Sotiris Kotsiantis. 2025. Agentic ai in education: State of the art and future directions. IEEE Access. [find](https://scholar.google.com/scholar?q=Georgios+Kostopoulos%2C+Vasileios+Gkamas%2C+Maria+Rigou%2C+and+Sotiris+Kotsiantis%2E+2025%2E+Agentic+ai+in+education%3A+State+of+the+art+and+future+directions%2E+IEEE+Access%2E)
37. Philippe Laban, Lidiya Murakhovs’ ka, Caiming Xiong, and Chien-Sheng Wu. 2023. Are you sure? challenging llms leads to performance drops in the flipflop experiment. arXiv preprint arXiv:2311.08596. — [arXiv:2311.08596](https://arxiv.org/abs/2311.08596) [find](https://scholar.google.com/scholar?q=Philippe+Laban%2C+Lidiya+Murakhovs%E2%80%99+ka%2C+Caiming+Xiong%2C+and+Chien%2DSheng+Wu%2E+2023%2E+Are+you+sure%3F+challenging+llms+leads+to+performance+drops+in+the+flipflop+experiment%2E+arXiv+preprint+arXiv%3A2311%2E08596%2E+%E2%80%94+)
38. Yucheng Li, Yunhao Guo, Frank Guerin, and Chenghua Lin. 2024. [An open-source data contamination report for large language models](https://md.research.elamur.ai/free/en/papers/li-2023-open-source-data-contamination-report/). In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 528–541.
39. Aixin Liu, Aoxue Mei, Bangcai Lin, Bing Xue, Bingxuan Wang, Bingzheng Xu, Bochao Wu, Bowei Zhang, Chaofan Lin, Chen Dong, and 1 others. 2025. [Deepseek-v3](https://md.research.elamur.ai/free/en/papers/deepseek-ai-2025-deepseek-v3-2-pushing/). 2: Pushing the frontier of open large language models. arXiv preprint arXiv:2512.02556.
40. Dongrui Liu, Qihan Ren, Chen Qian, Shuai Shao, Yuejin Xie, Yu Li, Zhonghao Yang, Haoyu Luo, Peng Wang, Qingyu Liu, and 1 others. 2026. Agentdog: A diagnostic guardrail framework for ai agent safety and security. arXiv preprint arXiv:2601.18491. — [arXiv:2601.18491](https://arxiv.org/abs/2601.18491) [find](https://scholar.google.com/scholar?q=Dongrui+Liu%2C+Qihan+Ren%2C+Chen+Qian%2C+Shuai+Shao%2C+Yuejin+Xie%2C+Yu+Li%2C+Zhonghao+Yang%2C+Haoyu+Luo%2C+Peng+Wang%2C+Qingyu+Liu%2C+and+1+others%2E+2026%2E+Agentdog%3A+A+diagnostic+guardrail+framework+for+ai+agent+safety+an)
41. Xinxin Liu. 2024. A survey of hallucination problems based on large language models. Applied and Computational Engineering, 97:24–30. [find](https://scholar.google.com/scholar?q=Xinxin+Liu%2E+2024%2E+A+survey+of+hallucination+problems+based+on+large+language+models%2E+Applied+and+Computational+Engineering%2C+97%3A24%E2%80%9330%2E)
42. Ryan Lopopolo. 2026. Harness engineering: leveraging codex in an agent-first world. Technical report, OpenAI. [find](https://scholar.google.com/scholar?q=Ryan+Lopopolo%2E+2026%2E+Harness+engineering%3A+leveraging+codex+in+an+agent%2Dfirst+world%2E+Technical+report%2C+OpenAI%2E)
43. Chris Lu, Cong Lu, Robert Tjarko Lange, Yutaro Yamada, Shengran Hu, Jakob Foerster, David Ha, and Jeff Clune. 2026. Towards end-to-end automation of ai research. Nature, 651(8107):914–919. [find](https://scholar.google.com/scholar?q=Chris+Lu%2C+Cong+Lu%2C+Robert+Tjarko+Lange%2C+Yutaro+Yamada%2C+Shengran+Hu%2C+Jakob+Foerster%2C+David+Ha%2C+and+Jeff+Clune%2E+2026%2E+Towards+end%2Dto%2Dend+automation+of+ai+research%2E+Nature%2C+651%288107%29%3A914%E2%80%93919%2E)
44. Yougang Lyu, Xi Zhang, Xinhao Yi, Yuyue Zhao, Shuyu Guo, Wenxiang Hu, Jan Piotrowski, Jakub Kaliski, Jacopo Urbani, Zaiqiao Meng, Lun Zhou, and Xiaohui Yan. 2026. [Evoscientist: Towards multi-agent evolving ai scientists for end-to-end scientific discovery](https://md.research.elamur.ai/free/en/papers/lyu-2026-evoscientist-towards-multi-agent-evolving/). arXiv preprint arXiv:2603.08127.
45. Lars Malmqvist. 2025. [Sycophancy in large language models: Causes and mitigations](https://md.research.elamur.ai/free/en/papers/malmqvist-2024-sycophancy-large-language-models-causes/). In Intelligent Computing-Proceedings of the Computing Conference, pages 61–74. Springer.
46. Joshua Maynez, Shashi Narayan, Bernd Bohnet, and Ryan McDonald. 2020. [On faithfulness and factuality in abstractive summarization](https://md.research.elamur.ai/free/en/papers/maynez-2020-faithfulness-factuality-abstractive-summarization/). In Proceedings of the 58th annual meeting of the association for computational linguistics, pages 1906–1919.
47. San Murugesan. 2025. The rise of agentic ai: implications, concerns, and the path forward. IEEE Intelligent Systems, 40(2):8–14. [find](https://scholar.google.com/scholar?q=San+Murugesan%2E+2025%2E+The+rise+of+agentic+ai%3A+implications%2C+concerns%2C+and+the+path+forward%2E+IEEE+Intelligent+Systems%2C+40%282%29%3A8%E2%80%9314%2E)
48. Codex. [https://openai.com/](https://openai.com/) [↗](https://openai.com/)
49. OpenAI. 2026b. Introducing gpt-5.4. [https://openai.com/index/introducing-gpt-5-4/](https://openai.com/index/introducing-gpt-5-4/). [↗](https://openai.com/index/introducing-gpt-5-4/)
50. OpenClaw. 2026. Openclaw: Personal ai assistant. [https://github.com/openclaw/openclaw](https://github.com/openclaw/openclaw). [↗](https://github.com/openclaw/openclaw)
51. Naomi Oreskes. 2021. Why trust science? Princeton University Press. [find](https://scholar.google.com/scholar?q=Naomi+Oreskes%2E+2021%2E+Why+trust+science%3F+Princeton+University+Press%2E)
52. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, and 1 others. 2022. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in neural information processing systems, 35:27730–27744.
53. Ethan Perez, Sam Ringer, Kamile Lukosiute, Karina Nguyen, Edwin Chen, Scott Heiner, Craig Pettit, Catherine Olsson, Sandipan Kundu, Saurav Kadavath, and 1 others. 2023. [Discovering language model behaviors with model-written evaluations](https://md.research.elamur.ai/free/en/papers/perez-2022-discovering-language-model-behaviors-model/). In Findings of the association for computational linguistics:…
54. Thang D Pham, Aditya Tanikanti, and Murat Keçeli. 2026. Chemgraph as an agentic framework for computational chemistry workflows. Communications Chemistry. [find](https://scholar.google.com/scholar?q=Thang+D+Pham%2C+Aditya+Tanikanti%2C+and+Murat+Ke%C3%A7eli%2E+2026%2E+Chemgraph+as+an+agentic+framework+for+computational+chemistry+workflows%2E+Communications+Chemistry%2E)
55. Leonardo Ranaldi and Giulia Pucci. 2023. When large language models contradict humans? large language models’ sycophantic behaviour. arXiv preprint arXiv:2311.09410. — [arXiv:2311.09410](https://arxiv.org/abs/2311.09410) [find](https://scholar.google.com/scholar?q=Leonardo+Ranaldi+and+Giulia+Pucci%2E+2023%2E+When+large+language+models+contradict+humans%3F+large+language+models%E2%80%99+sycophantic+behaviour%2E+arXiv+preprint+arXiv%3A2311%2E09410%2E+%E2%80%94+%5BarXiv%3A2311%2E09410%5D%28https%3A%2F%2Farxiv)
56. Itai Shapira, Gerdus Benade, and Ariel D Procaccia. 2026. How rlhf amplifies sycophancy. arXiv preprint arXiv:2602.01002. — [arXiv:2602.01002](https://arxiv.org/abs/2602.01002) [find](https://scholar.google.com/scholar?q=Itai+Shapira%2C+Gerdus+Benade%2C+and+Ariel+D+Procaccia%2E+2026%2E+How+rlhf+amplifies+sycophancy%2E+arXiv+preprint+arXiv%3A2602%2E01002%2E+%E2%80%94+%5BarXiv%3A2602%2E01002%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2602%2E01002%29)
57. Mrinank Sharma, Meg Tong, Tomek Korbak, David Duvenaud, Amanda Askell, Sam Bowman, Esin Durmus, Zac Hatfield-Dodds, Scott Johnston, Shauna Kravec, and 1 others. 2024. [Towards understanding sycophancy in language models](https://md.research.elamur.ai/free/en/papers/sharma-2023-towards-understanding-sycophancy-language-models/). In International Conference on Learning Representations, volume 2024, pages 110–…
58. Tao Song, Man Luo, Xiaolong Zhang, Linjiang Chen, Yan Huang, Jiaqi Cao, Qing Zhu, Daobin Liu, Baicheng Zhang, Gang Zou, and 1 others. 2025. A multiagent-driven robotic ai chemist enabling autonomous chemical research on demand. Journal of the American Chemical Society, 147(15):12534– 12545. [find](https://scholar.google.com/scholar?q=Tao+Song%2C+Man+Luo%2C+Xiaolong+Zhang%2C+Linjiang+Chen%2C+Yan+Huang%2C+Jiaqi+Cao%2C+Qing+Zhu%2C+Daobin+Liu%2C+Baicheng+Zhang%2C+Gang+Zou%2C+and+1+others%2E+2025%2E+A+multiagent%2Ddriven+robotic+ai+chemist+enabling+autonomous+c)
59. Felix Strieth-Kalthoff, Han Hao, Vandana Rathore, Joshua Derasp, Théophile Gaudin, Nicholas H Angello, Martin Seifrid, Ekaterina Trushina, Mason Guy, Junliang Liu, and 1 others. 2024. Delocalized, asynchronous, closed-loop discovery of organic laser emitters. Science, 384(6697):eadk9227. [find](https://scholar.google.com/scholar?q=Felix+Strieth%2DKalthoff%2C+Han+Hao%2C+Vandana+Rathore%2C+Joshua+Derasp%2C+Th%C3%A9ophile+Gaudin%2C+Nicholas+H+Angello%2C+Martin+Seifrid%2C+Ekaterina+Trushina%2C+Mason+Guy%2C+Junliang+Liu%2C+and+1+others%2E+2024%2E+Delocalized%2C+asy)
60. Chuanyuan Tan, Wenbiao Shao, Hao Xiong, Tong Zhu, Zhenhua Liu, Kai Shi, and Wenliang Chen. 2025. Uaqfact: Evaluating factual knowledge utilization of llms on unanswerable questions. In Findings of the Association for Computational Linguistics: ACL 2025, pages 1700–1715. [find](https://scholar.google.com/scholar?q=Chuanyuan+Tan%2C+Wenbiao+Shao%2C+Hao+Xiong%2C+Tong+Zhu%2C+Zhenhua+Liu%2C+Kai+Shi%2C+and+Wenliang+Chen%2E+2025%2E+Uaqfact%3A+Evaluating+factual+knowledge+utilization+of+llms+on+unanswerable+questions%2E+In+Findings+of+the)
61. Hanchen Wang, Yichun He, Paula P Coelho, Matthew Bucci, Abbas Nazir, Bob Chen, Linh Trinh, Serena Zhang, Kexin Huang, Vineethkrishna Chandrasekar, and 1 others. 2025a. Spatialagent: An autonomous ai agent for spatial biology. bioRxiv, pages 2025–04. [find](https://scholar.google.com/scholar?q=Hanchen+Wang%2C+Yichun+He%2C+Paula+P+Coelho%2C+Matthew+Bucci%2C+Abbas+Nazir%2C+Bob+Chen%2C+Linh+Trinh%2C+Serena+Zhang%2C+Kexin+Huang%2C+Vineethkrishna+Chandrasekar%2C+and+1+others%2E+2025a%2E+Spatialagent%3A+An+autonomous+ai+a)
62. Wenbo Wang, Simran Swain, Jaeyong Lee, Zuwan Lin, Bradley Canales, Almir Aljović, Yaxuan Liu, Qiang Li, Arnau Marin-Llobet, Mai Liu, and 1 others. 2025b. Agentic lab: An agentic-physical ai system for cell and organoid experimentation and manufacturing. bioRxiv, pages 2025–11. [find](https://scholar.google.com/scholar?q=Wenbo+Wang%2C+Simran+Swain%2C+Jaeyong+Lee%2C+Zuwan+Lin%2C+Bradley+Canales%2C+Almir+Aljovic%CC%81%2C+Yaxuan+Liu%2C+Qiang+Li%2C+Arnau+Marin%2DLlobet%2C+Mai+Liu%2C+and+1+others%2E+2025b%2E+Agentic+lab%3A+An+agentic%2Dphysical+ai+system+fo)
63. Jerry Wei, Da Huang, Yifeng Lu, Denny Zhou, and Quoc V Le. 2023. [Simple synthetic data reduces sycophancy in large language models](https://md.research.elamur.ai/free/en/papers/wei-2023-simple-synthetic-data-reduces-sycophancy/). arXiv preprint arXiv:2308.03958.
64. Jiaqi Wei, Yuejin Yang, Xiang Zhang, Yuhan Chen, Xiang Zhuang, Zhangyang Gao, Dongzhan Zhou, Guangshuai Wang, Zhiqiang Gao, Juntai Cao, and 1 others. 2025. [From ai for science to agentic science: A survey on autonomous scientific discovery](https://md.research.elamur.ai/free/en/papers/wei-2025-ai-science-agentic-science/). arXiv preprint arXiv:2508.14111.
65. Wikipedia contributors. 2026. List of pseudoscience topics. [https://en.wikipedia.org/w/index](https://en.wikipedia.org/w/index). php?title=List_of_pseudoscience_topics. [↗](https://en.wikipedia.org/w/index)
66. Ruofeng Yang, Yongcan Li, and Shuai Li. 2026. Aris: Autonomous research via adversarial multi-agent collaboration. arXiv preprint arXiv:2605.03042. ymx10086. 2026. Researchclaw: Local-first research os for papers, workflows, experiments, channels, and automation. [https://github.com/ymx10086/Research…](https://github.com/ymx10086/Research…) — [arXiv:2605.03042](https://arxiv.org/abs/2605.03042) [↗](https://github.com/ymx10086/Research…)
67. Barry Zhang, Keith Lazuka, and Maahesh Murag. 2025a. Equipping agents for the real world with agent skills. Technical report, Engineering at Anthropic. [find](https://scholar.google.com/scholar?q=Barry+Zhang%2C+Keith+Lazuka%2C+and+Maahesh+Murag%2E+2025a%2E+Equipping+agents+for+the+real+world+with+agent+skills%2E+Technical+report%2C+Engineering+at+Anthropic%2E)
68. Muru Zhang, Ofir Press, William Merrill, Alisa Liu, and Noah A Smith. 2023. [How language model hallucinations can snowball](https://md.research.elamur.ai/free/en/papers/zhang-2023-language-model-hallucinations-snowball/). arXiv preprint arXiv:2305.13534.
69. Yanzhao Zhang, Mingxin Li, Dingkun Long, Xin Zhang, Huan Lin, Baosong Yang, Pengjun Xie, An Yang, Dayiheng Liu, Junyang Lin, and 1 others. 2025b. [Qwen3 embedding: Advancing text embedding and reranking through foundation models](https://md.research.elamur.ai/free/en/papers/zhang-2025-qwen3-embedding-advancing-text/). arXiv preprint arXiv:2506.05176.
70. Zhengxin Zhang, Ning Wang, Sainyam Galhotra, and Claire Cardie. 2026. [How far are we from true autoresearch?](https://md.research.elamur.ai/free/en/papers/zhang-2026-far-true-auto-research/) arXiv preprint arXiv:2605.19156.
71. Yujun Zhou, Jingdong Yang, Yue Huang, Kehan Guo, Zoe Emory, Bikram Ghosh, Amita Bedar, Sujay Shekar, Zhenwen Liang, Pin-Yu Chen, and 1 others. 2026. Benchmarking large language models on safety risks in scientific laboratories. Nature Machine Intelligence, pages 1–12. [find](https://scholar.google.com/scholar?q=Yujun+Zhou%2C+Jingdong+Yang%2C+Yue+Huang%2C+Kehan+Guo%2C+Zoe+Emory%2C+Bikram+Ghosh%2C+Amita+Bedar%2C+Sujay+Shekar%2C+Zhenwen+Liang%2C+Pin%2DYu+Chen%2C+and+1+others%2E+2026%2E+Benchmarking+large+language+models+on+safety+risks+)
72. James Zou and Eric J Topol. 2025. agentic ai teammates in medicine. 405(10477):457. [find](https://scholar.google.com/scholar?q=James+Zou+and+Eric+J+Topol%2E+2025%2E+agentic+ai+teammates+in+medicine%2E+405%2810477%29%3A457%2E)
73. The rise of The Lancet, Yunheng Zou, Austin H Cheng, Abdulrahman Aldossary, Jiaru Bai, Shi Xuan Leong, Jorge Arturo Campos-Gonzalez-Angulo, Changhyeok Choi, Cher Tian Ser, Gary Tom, Andrew Wang, and 1 others. 2025. El agente: An autonomous agent for quantum chemistry. Matter, 8(7). [find](https://scholar.google.com/scholar?q=The+rise+of+The+Lancet%2C+Yunheng+Zou%2C+Austin+H+Cheng%2C+Abdulrahman+Aldossary%2C+Jiaru+Bai%2C+Shi+Xuan+Leong%2C+Jorge+Arturo+Campos%2DGonzalez%2DAngulo%2C+Changhyeok+Choi%2C+Cher+Tian+Ser%2C+Gary+Tom%2C+Andrew+Wang%2C+and+1)

*Updated 3 November 2026*