---
title: "Position — paper by Eulig (2026) — summary and ideas"
description: "The author proposes evaluating systems on three separate axes: outcome, mechanism fidelity and… 19 references, cited by 1 paper here."
authors: ["Steven Young Eulig"]
year: 2026
venue: "arXiv:2606.23175"
source: "https://arxiv.org/abs/2606.23175"
doi: "10.48550/arxiv.2606.23175"
arxiv: "2606.23175"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/eulig-2026-position-correct-answer-wrong/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/eulig-2026-position-correct-answer-wrong/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/16/) / Position: Correct Answer, Wrong Mechanism — When AI Scientists Defend General Claims Their Own Data Contradicts

# Position: Correct Answer, Wrong Mechanism — When AI Scientists Defend General Claims Their Own Data Contradicts

**Authors:** Steven Young Eulig · **Year:** 2026 · **Venue:** arXiv:2606.23175 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2606.23175) · [PDF](https://arxiv.org/pdf/2606.23175) · [DOI 10.48550/arxiv.2606.23175](https://doi.org/10.48550/arxiv.2606.23175) · [arXiv 2606.23175](https://arxiv.org/abs/2606.23175) · [OpenAlex](https://openalex.org/W7165658074)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper argues that judging AI scientist systems only by whether their final answer is right hides a serious failure. An agent can reach a correct-looking result while justifying it with reasoning that is false or that overreaches beyond what it tested. The author calls this pattern Correct Answer, Wrong Mechanism (CAWM). The study ran 28 episodes in which coding agents tried to rediscover a known particle-identification observable, the leading-edge photon fraction f_early, in a Geant4 Cherenkov detector simulation. Twenty episodes used Claude Opus 4.6 under four prompt conditions, ranging from fully open-ended to a deliberately biased prior. Eight more episodes probed two other frontier models.

CAWM occurred in 4/20 primary and 3/8 cross-model episodes. Honesty and mechanism correctness turned out to be separable: an agent could revise a false belief and still defend an observable with physics that its own data refuted. The author proposes evaluating systems on three separate axes: outcome, mechanism fidelity and epistemic honesty. The paper also offers cheap detection checks. The conclusion is that agents are dependable tools but not yet trustworthy co-authors.

## Key points
- **Setup:** Four prompt variants were used, N=5 each. A was open-ended, B added a five-step scaffold, C was a positive control that supplied f_early, and D gave a partially misleading prior. Impact parameters were d = 10–25 m at E = 40 GeV.
- **Why the main alternative misleads:** The competing observable σ(hitTime) fails to discriminate at d = 10 m but separates muons from electrons at 20–25 m. An agent that tests it in only one regime therefore draws conclusions that do not transfer.
- **Discovery versus execution:** No open-ended Prompt A agent chose f_early as its primary observable (0/5), and three defended their choice with incorrect physics. With the formula given (Prompt C), all five agents reproduced it.
- **Scaffolding and honesty:** The scaffold removed CAWM for Claude (0/5) but not in the cross-model probe (2/2). Under Prompt D, all five agents rejected the false claim, yet episode 19 still argued for σ using reasoning its own data contradicted.
- **Detection:** A one-step regime-shift check combined with a recomputation step flags every CAWM case. The simulation data (ROOT files), prompts and verification script are released at github.com/seulig/g4 agent icml 2026.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 2,009 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 665 works · [🎯 Decision Sciences](https://md.research.elamur.ai/free/en/shelves/decision-sciences/) · 306 works · [🎯 Information Systems and Management](https://md.research.elamur.ai/free/en/shelves/decision-sciences/information-systems-and-management/) · 96 works

## Cite

Cite as: Steven Young Eulig. 2026. Position: Correct Answer, Wrong Mechanism — When AI Scientists Defend General Claims Their Own Data Contradicts. arXiv:2606.23175.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{eulig2026position,
  title = {{Position: Correct Answer, Wrong Mechanism — When AI Scientists Defend General Claims Their Own Data Contradicts}},
  author = {Steven Young Eulig},
  year = {2026},
  doi = {10.48550/arxiv.2606.23175},
  eprint = {2606.23175},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2606.23175}
}
```

- [← previous Polynomial Expansion Rank Adaptation: Enhancing Low-Rank Fine-Tuning with High-Order Interactions](https://md.research.elamur.ai/free/en/papers/zhang-2026-polynomial-expansion-rank-adaptation-enhancing/)
- [next → Position: From Noise to Signal to Selbstzweck - Reframing Human Label Variation in the Era of Post-training in NLP](https://md.research.elamur.ai/free/en/papers/xu-2026-position-noise-signal-selbstzweck-reframing/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/16/)

## References

19 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Labs and deployers. (i) Role boundaries matter. LLM coding agents are reliable at tool-level tasks and unreliable at co-author-level observable selection in the simulation-based discovery setting studied here (Table 3). (ii) Scaffolding Agostinelli, S., Allison, J., Amako, K., Apostolakis, J., Arauj… [find](https://scholar.google.com/scholar?q=Labs+and+deployers%2E+%28i%29+Role+boundaries+matter%2E+LLM+coding+agents+are+reliable+at+tool%2Dlevel+tasks+and+unreliable+at+co%2Dauthor%2Dlevel+observable+selection+in+the+simulation%2Dbased+discovery+setting+stud)
2. Lu, C., Lu, C., Lange, R. T., Yamada, Y., Hu, S., Foerster, J., Ha, D., and Clune, J. Towards end-to-end automation of AI research. Nature, 651:914–919, 2026. [find](https://scholar.google.com/scholar?q=Lu%2C+C%2E%2C+Lu%2C+C%2E%2C+Lange%2C+R%2E+T%2E%2C+Yamada%2C+Y%2E%2C+Hu%2C+S%2E%2C+Foerster%2C+J%2E%2C+Ha%2C+D%2E%2C+and+Clune%2C+J%2E+Towards+end%2Dto%2Dend+automation+of+AI+research%2E+Nature%2C+651%3A914%E2%80%93919%2C+2026%2E)
3. Beel, J., Kan, M.-Y., and Baumgart, M. Evaluating Sakana’s AI scientist: Bold claims, mixed results, and a promising future? ACM SIGIR Forum, 2025. [arXiv preprint arXiv:2502.14297](https://md.research.elamur.ai/free/en/papers/beel-2025-evaluating-sakana-s-ai/).
4. Luo, Z., Kasirzadeh, A., and Shah, N. B. [The more you automate, the less you see: Hidden pitfalls of AI scientist systems](https://md.research.elamur.ai/free/en/papers/luo-2025-more-automate-less-see/). arXiv preprint arXiv:2509.08713, 2025.
5. NeurIPS 2025 AI4Science Workshop (Spotlight). [find](https://scholar.google.com/scholar?q=NeurIPS+2025+AI4Science+Workshop+%28Spotlight%29%2E)
6. Boiko, D. A., MacKnight, R., Kline, B., and Gomes, G. [Autonomous chemical research with large language models](https://md.research.elamur.ai/free/en/papers/boiko-2023-autonomous-chemical-research-large-language/). Nature, 624(7992):570–578, 2023.
7. McCoy, R. T., Pavlick, E., and Linzen, T. [Right for the wrong reasons: Diagnosing syntactic heuristics in natural language inference](https://md.research.elamur.ai/free/en/papers/mccoy-2019-right-wrong-reasons-diagnosing-syntactic/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), 2019.
8. Chen, Z., Chen, S., Ning, Y., Zhang, Q., Wang, B., Yu, B., Li, Y., Liao, Z., Wei, C., Lu, Z., Dey, V., Xue, M., Baker, F. N., Burns, B., Adu-Ampratwum, D., Huang, X., Ning, X., Gao, S., Su, Y., and Sun, H. ScienceAgentBench: Toward rigorous assessment of language agents for data-driven scientific di… — [ScienceAgentBench: Toward Rigorous Assessment of Language Agents for Data-Driven Scientific Discovery](https://md.research.elamur.ai/free/en/papers/chen-2024-scienceagentbench-rigorous-assessment-language/)
9. Mitchener, L., Yiu, A., Chang, B., Bourdenx, M., Nadolski, T., Sulovari, A., Landsness, E. C., Barabasi, D. L., Narayanan, S., Evans, N., Reddy, S., Foiani, M., Kamal, A., Shriver, L. P., Cao, F., Wassie, A. T., Laurent, J. M., Melville-Green, E., Caldas, M., Bou, A., Roberts, K. F., Zagorac, S., Or… — [Kosmos: An AI Scientist for Autonomous Discovery](https://md.research.elamur.ai/free/en/papers/mitchener-2025-kosmos-ai-scientist-autonomous/)
10. Geirhos, R., Jacobsen, J.-H., Michaelis, C., Zemel, R., Brendel, W., Bethge, M., and Wichmann, F. A. Shortcut learning in deep neural networks. Nature Machine Intelligence, 2(11):665–673, 2020. [find](https://scholar.google.com/scholar?q=Geirhos%2C+R%2E%2C+Jacobsen%2C+J%2E%2DH%2E%2C+Michaelis%2C+C%2E%2C+Zemel%2C+R%2E%2C+Brendel%2C+W%2E%2C+Bethge%2C+M%2E%2C+and+Wichmann%2C+F%2E+A%2E+Shortcut+learning+in+deep+neural+networks%2E+Nature+Machine+Intelligence%2C+2%2811%29%3A665%E2%80%93673%2C+2020%2E)
11. Gottweis, J., Weng, W.-H., Daryin, A., Tu, T., Palepu, A., Sirkovic, P., Myaskovsky, A., Weissenberger, F., Rong, K., Tanno, R., Saab, K., Popovici, D., Blum, J., Zhang, F., Chou, K., Hassidim, A., Gokturk, B., Vahdat, A., Kohli, P., Matias, Y., Carroll, A., Kulkarni, K., Tomasev, N., Guan, Y., Dhil… — [Accelerating scientific discovery with Co-Scientist](https://md.research.elamur.ai/free/en/papers/gottweis-2025-accelerating-scientific-discovery-co/)
12. Niven, T. and Kao, H.-Y. Probing neural network comprehension of natural language arguments. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (ACL), 2019. [find](https://scholar.google.com/scholar?q=Niven%2C+T%2E+and+Kao%2C+H%2E%2DY%2E+Probing+neural+network+comprehension+of+natural+language+arguments%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28ACL%29%2C+2019%2E)
13. Ren, R., Agarwal, A., Mazeika, M., Menghini, C., Vacareanu, R., Kenstler, B., Yang, M., Barrass, I., Gatti, A., Yin, X., Trevino, E., Geralnik, M., Khoja, A., Lee, D., Yue, S., and Hendrycks, D. The MASK benchmark: Disentangling honesty from accuracy in AI systems. arXiv preprint arXiv:2503.03750, 2… — [arXiv:2503.03750](https://arxiv.org/abs/2503.03750) [find](https://scholar.google.com/scholar?q=Ren%2C+R%2E%2C+Agarwal%2C+A%2E%2C+Mazeika%2C+M%2E%2C+Menghini%2C+C%2E%2C+Vacareanu%2C+R%2E%2C+Kenstler%2C+B%2E%2C+Yang%2C+M%2E%2C+Barrass%2C+I%2E%2C+Gatti%2C+A%2E%2C+Yin%2C+X%2E%2C+Trevino%2C+E%2E%2C+Geralnik%2C+M%2E%2C+Khoja%2C+A%2E%2C+Lee%2C+D%2E%2C+Yue%2C+S%2E%2C+and+Hendrycks%2C+D%2E+The+M)
14. Jansen, P., Côté, M.-A., Khot, T., Bransom, E., Dalvi Mishra, B., Majumder, B. P., Tafjord, O., and Clark, P. [DiscoveryWorld: A virtual environment for developing and evaluating automated scientific discovery agents](https://md.research.elamur.ai/free/en/papers/jansen-2024-discoveryworld-virtual-environment-developing/). Advances in Neural Information Processing Systems, 37, 2024.
15. Turpin, M., Michael, J., Perez, E., and Bowman, S. R. Language models don’t always say what they think: Unfaithful explanations in Chain-of-Thought prompting. Advances in Neural Information Processing Systems, 36, 2023. — [Language Models Don't Always Say What They Think: Unfaithful Explanations in Chain-of-Thought Prompting](https://md.research.elamur.ai/free/en/papers/turpin-2023-language-models-don-t-always/)
16. Lanham, T., Chen, A., Radhakrishnan, A., Steiner, B., Denison, C., Hernandez, D., Li, D., Durmus, E., Hubinger, E., Kernion, J., Lukošiūtė, K., Nguyen, K., Cheng, N., Joseph, N., Schiefer, N., Rausch, O., Larson, R., McCandlish, S., Kundu, S., Kadavath, S., Yang, S., Henighan, T., Maxwell, T., Te… — [Measuring Faithfulness in Chain-of-Thought Reasoning](https://md.research.elamur.ai/free/en/papers/lanham-2023-measuring-faithfulness-chain-thought-reasoning/)
17. Uesato, J., Kushman, N., Kumar, R., Song, F., Siegel, N., Wang, L., Creswell, A., Irving, G., and Higgins, I. [Solving math word problems with process- and outcome-based feedback](https://md.research.elamur.ai/free/en/papers/uesato-2022-solving-math-word-problems-process/). arXiv preprint arXiv:2211.14275, 2022.
18. Yamada, Y., Lange, R. T., Lu, C., Hu, S., Lu, C., Foerster, J., Clune, J., and Ha, D. [The AI scientist-v2: Workshop-level automated scientific discovery via agentic tree search](https://md.research.elamur.ai/free/en/papers/yamada-2025-ai-scientist-v2-workshop/). arXiv preprint arXiv:2504.08066, 2025.
19. Lightman, H., Kosaraju, V., Burda, Y., Edwards, H., Baker, B., Lee, T., Leike, J., Schulman, J., Sutskever, I., and Cobbe, K. Let’s verify step by step. In International Conference on Learning Representations, 2024. — [Let's Verify Step by Step](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/)

*Updated 3 November 2026*