--- title: "Breaking the Illusion of Reasoning in Polish LLMs: Quality over" description: "It remains unclear why explicit reasoning helps LLMs. 1 reference. One explanation is the content of the reasoning traces." authors: ["Dzmitry Pihulski", "Mikołaj Langner", "Jan Eliasz", "Przemyslaw Kazienko", "Jan Kocon", "Teddy Ferdinan"] year: 2026 venue: "Findings of the Association for Computational Linguistics: EACL 2026" source: "https://aclanthology.org/2026.findings-eacl.92/" doi: "10.18653/v1/2026.findings-eacl.92" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/pihulski-2026-breaking-illusion-reasoning-polish-llms/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pihulski-2026-breaking-illusion-reasoning-polish-llms/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/) / Breaking the Illusion of Reasoning in Polish LLMs: Quality over Quantity of Thought # Breaking the Illusion of Reasoning in Polish LLMs: Quality over Quantity of Thought **Authors:** Dzmitry Pihulski, Mikołaj Langner, Jan Eliasz, [Przemyslaw Kazienko](https://md.research.elamur.ai/free/en/authors/przemyslaw-kazienko/), [Jan Kocon](https://md.research.elamur.ai/free/en/authors/jan-kocon/) and Teddy Ferdinan · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: EACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.findings-eacl.92/) · [PDF](https://aclanthology.org/2026.findings-eacl.92.pdf) · [DOI 10.18653/v1/2026.findings-eacl.92](https://doi.org/10.18653/v1/2026.findings-eacl.92) · [OpenAlex](https://openalex.org/W7140086371) **Citations:** 3 citations (OpenAlex, 2026-10-14) ## Summary It remains unclear why explicit reasoning helps LLMs. One explanation is the content of the reasoning traces. Another is simply the extra tokens of computation. The authors separate these two factors in a Polish setting. They machine-translated about 32K samples of Mixture-of-Thoughts into Polish with DeepSeek-V3-0324 and filtered them, producing MoT-PL. They then built four versions that share identical questions and answers and differ only in the reasoning part: - **Detailed:** the original traces. - **Summarized:** compressed traces. - **BabyThink:** simplistic, child-like traces of roughly the original length. - **Lengthy:** traces at least doubled in size. Each version was used to fine-tune Llama-PLLuM-8B-instruct. The models were tested on Belebele, Aya Collection, MoT-PL-eval and LightR1. Only the Detailed model beat the non-reasoning baseline on average. Longer traces of similar quality did not help, and neither did weak traces of similar length. Within each model, correct answers tended to use fewer reasoning tokens. The study concludes that clear, relevant reasoning, including room for self-correction, matters more than verbosity. It also contributes a Polish reasoning dataset for building such models. ## Key points - **Dataset:** MoT-PL has 22,571 training examples (28% math, 17% code, 55% science) and a separate 900-example held-out set, MoT-PL-eval. Filtering used langdetect, BERTScore and blacklisted strings. - **Trace lengths:** average reasoning length is about 2,523 tokens for Detailed, 444 for Summarized, 2,661 for BabyThink and 4,884 for Lengthy. Qwen3-235B and Llama-3.3-70B, used as LLM judges, confirmed that BabyThink has the lowest quality. - **Average scores:** Detailed 0.540, baseline Llama-PLLuM-8B-instruct 0.494, BabyThink 0.392, Summarized 0.358, Lengthy 0.342. Improving quality at a similar length (Detailed vs. BabyThink) gives a 14.8 pp gain. - **Regressions of the Detailed model:** it scores below the baseline on Polish Aya (0.584 vs. 0.656) and LightR1 (0.146 vs. 0.172). The authors attribute this to catastrophic forgetting from training only on MoT-PL. - **Code:** on the MoT-PL-eval code subset, the baseline scores best. Fine-tuned models write real code but sometimes miss task constraints, and the judge accepted the baseline's prose descriptions of solutions. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Wrocław University of Science and Technology](https://md.research.elamur.ai/free/en/universities/wroclaw-university-of-science-and-technology/) — University, Wroclaw, Poland · 8 works · 945 citations 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 2,869 works 🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works ## Cite Cite as: Dzmitry Pihulski, Mikołaj Langner, Jan Eliasz, Przemyslaw Kazienko, Jan Kocon and Teddy Ferdinan. 2026. Breaking the Illusion of Reasoning in Polish LLMs: Quality over Quantity of Thought. Findings of the Association for Computational Linguistics: EACL 2026. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{pihulski2026breaking, title = {{Breaking the Illusion of Reasoning in Polish LLMs: Quality over Quantity of Thought}}, author = {Dzmitry Pihulski, Mikołaj Langner, Jan Eliasz, Przemyslaw Kazienko, Jan Kocon and Teddy Ferdinan}, booktitle = {Findings of the Association for Computational Linguistics: EACL 2026}, year = {2026}, doi = {10.18653/v1/2026.findings-eacl.92}, url = {https://doi.org/10.18653/v1/2026.findings-eacl.92} } ``` - [← previous Blind Spot Navigation in Large Language Model Reasoning with Thought Space Explorer](https://md.research.elamur.ai/free/en/papers/zhang-2026-blind-spot-navigation-large-language/) - [next → Can ChatGPT Really Understand Modern Chinese Poetry?](https://md.research.elamur.ai/free/en/papers/wang-2026-can-chatgpt-really-understand-modern/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/5/) ## References 1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. AI-based tools, including ChatGPT, Grammarly Premium, and Writeful, were used exclusively to support linguistic clarity and improve the readability of the manuscript. This work was supported in part by: (1) the National Science Center, Poland, project no. 2021/41/B/ST6/04471; (2) CLARIN-PL: Common L… [find](https://scholar.google.com/scholar?q=AI%2Dbased+tools%2C+including+ChatGPT%2C+Grammarly+Premium%2C+and+Writeful%2C+were+used+exclusively+to+support+linguistic+clarity+and+improve+the+readability+of+the+manuscript%2E+This+work+was+supported+in+part+b) *Updated 31 October 2026*