--- title: "FEANEL — paper by Ye et al. (2025) — summary and ideas" description: "The authors define a task called Fine-grained Error Analysis for English Learners. 90% accuracy. 3 references." authors: ["Jingheng Ye", "Shen Wang", "Jiaqi Chen", "Hebin Wang", "Deqing Zou", "Yanyu Zhu", "Jiwei Tang", "Hai-Tao Zheng", "Ruitong Liu", "Haoyang Li", "Yanfeng Wang", "Qingsong Wen"] year: 2025 venue: "arXiv:2511.22883" source: "https://arxiv.org/abs/2511.22883" doi: "10.48550/arxiv.2511.22883" arxiv: "2511.22883" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/ye-2025-feanel-benchmark-fine-grained-error/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ye-2025-feanel-benchmark-fine-grained-error/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/16/) / FEANEL: A Benchmark for Fine-Grained Error Analysis in K-12 English Writing # FEANEL: A Benchmark for Fine-Grained Error Analysis in K-12 English Writing **Authors:** [Jingheng Ye](https://md.research.elamur.ai/free/en/authors/jingheng-ye/), [Shen Wang](https://md.research.elamur.ai/free/en/authors/shen-wang/), [Jiaqi Chen](https://md.research.elamur.ai/free/en/authors/jiaqi-chen/), Hebin Wang, [Deqing Zou](https://md.research.elamur.ai/free/en/authors/deqing-zou/), Yanyu Zhu, [Jiwei Tang](https://md.research.elamur.ai/free/en/authors/jiwei-tang/), [Hai-Tao Zheng](https://md.research.elamur.ai/free/en/authors/hai-tao-zheng/), [Ruitong Liu](https://md.research.elamur.ai/free/en/authors/ruitong-liu/), [Haoyang Li](https://md.research.elamur.ai/free/en/authors/haoyang-li/), [Yanfeng Wang](https://md.research.elamur.ai/free/en/authors/yanfeng-wang/) and [Qingsong Wen](https://md.research.elamur.ai/free/en/authors/qingsong-wen/) · **Year:** 2025 · **Venue:** arXiv:2511.22883 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2511.22883) · [PDF](https://arxiv.org/pdf/2511.22883) · [DOI 10.48550/arxiv.2511.22883](https://doi.org/10.48550/arxiv.2511.22883) · [arXiv 2511.22883](https://arxiv.org/abs/2511.22883) · [OpenAlex](https://openalex.org/W4416949680) **Citations:** 0 citations (OpenAlex, 2026-10-14) ## Summary Feedback tools for school-age English learners usually stop at fixing errors (GEC) or giving holistic essay scores. Neither tells a student why a mistake happened or how serious it is. The authors define a task called Fine-grained Error Analysis for English Learners. A system receives an essay prompt, the student text, a corrected version and the list of edits between them. For each edit it must produce three things: an error category, a severity score from 1 to 5, and a written explanation of the problem and how to fix it. To support the task they built FEANEL, a set of 1,000 K-12 essays. Teaching experts corrected the essays and analysed every error using a new taxonomy organised around parts of speech. They then tested a broad set of current LLMs, both reasoning and standard models. The models struggle to categorise complex errors, and their explanations lack the precision and pedagogical care of human teachers. Results also change noticeably with prompt detail, the presence of examples, and the order of sub-tasks. The benchmark therefore measures something that existing GEC and essay-scoring evaluations miss. ## Key points - The dataset has 500 elementary-school essays (students aged 9–11, 3,003 errors) and 500 secondary-level essays (5,671 errors). The secondary essays come from the TECCL Corpus of Chinese EFL writing. The data is published on Hugging Face as Feanel/FEANEL. - Average essay length is 48.2 tokens for elementary and 127.1 for secondary. Average edits per essay are 6.01 and 11.34 respectively. - The taxonomy has 29 error types, mostly defined by part of speech. It avoids a catch-all "Other" category and has a priority scheme for choosing one main label when an edit contains several errors. - Experts made minimal corrections following GEC conventions, and edits were extracted with CLEME. Annotators had to reach at least 90% accuracy on a trial round before formal annotation. - Evaluated models include GPT-4o, o1, o3, o4-mini, Gemini-2.5-pro, DeepSeek-R1, Claude-3.7-Sonnet (with and without Thinking), Grok-3-Beta, Qwen-3, Llama-3 and Mistral-Small-3.1. By default, models output severity first, then type, then explanation. ## Interesting ideas - The error taxonomy is built on part-of-speech categories, has no "Other" bucket, and uses an explicit priority rule for edits with several errors, so every case gets one dominant label while the explanation still covers the rest. - The task gives the system the gold correction and the extracted edits, so detecting an error and explaining it are evaluated separately. - Severity is a 1–5 score for how much an error damages sentence structure and meaning, not just whether it exists. - The paper treats the order of sub-tasks (severity → type → explanation) as a variable and shows that results depend on it. - Annotators had to pass a qualification trial at a fixed accuracy threshold, and those who failed were retrained or replaced. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): Disagreements between witness parsers could be sorted with a similar exclusive, prioritised scheme (e.g. attachment vs. label vs. POS cause), which would make self-induced repair rules easier to target. - **I2** (back): Diffing input text against the text → graph → text reconstruction at the edit level would show exactly which spans the compressor loses, and each loss could then be explained and categorised. - **I3** (world): Parse or frame mistakes could be graded by how much they change the derived world state (entities, holdings, event order), so fixes that alter answers are prioritised. - **I4** (answer): The readers that solve school tests and word problems could be checked for order sensitivity in their steps, which would expose hidden dependencies between steps. - **I5** (learn): LLM teacher tiers could be admitted the same way, by first scoring them on a small gold set and only then trusting their silver labels. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works 🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,220 works ## Cite Cite as: Jingheng Ye, Shen Wang, Jiaqi Chen, Hebin Wang, Deqing Zou, Yanyu Zhu, Jiwei Tang, Hai-Tao Zheng, Ruitong Liu, Haoyang Li, Yanfeng Wang and Qingsong Wen. 2025. FEANEL: A Benchmark for Fine-Grained Error Analysis in K-12 English Writing. arXiv:2511.22883. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{ye2025feanel, title = {{FEANEL: A Benchmark for Fine-Grained Error Analysis in K-12 English Writing}}, author = {Jingheng Ye, Shen Wang, Jiaqi Chen, Hebin Wang, Deqing Zou, Yanyu Zhu, Jiwei Tang, Hai-Tao Zheng, Ruitong Liu, Haoyang Li, Yanfeng Wang and Qingsong Wen}, year = {2025}, doi = {10.48550/arxiv.2511.22883}, eprint = {2511.22883}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2511.22883} } ``` - [← previous Exploring the Interaction of Personality, Strategy, and Learning Environment in Achieving Success in Learning Arabic Grammar](https://md.research.elamur.ai/free/en/papers/bahy-2025-exploring-interaction-personality-strategy-learning/) - [next → Features in Lexicography for Arabic Dialects: Multimedia Dictionary of the Syrian Dialect](https://md.research.elamur.ai/free/en/papers/novospasskaya-2025-features-lexicography-arabic-dialects-multimedia/) - [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/16/) ## References 3 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Mistral AI. 2025. Mistral small 3.1. [https://mistral.ai/news/mistral-small-3-1](https://mistral.ai/news/mistral-small-3-1). [↗](https://mistral.ai/news/mistral-small-3-1) 2. Satanjeev Banerjee and Alon Lavie. 2005. [METEOR: An automatic metric for MT evaluation with improved correlation with human judgments](https://md.research.elamur.ai/free/en/papers/banerjee-2005-meteor-automatic-metric-mt-evaluation/). In Proceedings of the ACL Workshop on Intrinsic and Extrinsic Evaluation Measures for Machine Translation and/or Summarization, pages 65–72, Ann Arbor, Michigan. Association for Computational Linguistics. 3. Giorgio Biancini, Alessio Ferrato, and Carla Limongelli. 2024. Multiple-choice question generation using large language models: Methodology and educator insights. In Adjunct Proceedings of the 32nd ACM Conference on User Modeling, Adaptation and Personalization, pages 584–590. [find](https://scholar.google.com/scholar?q=Giorgio+Biancini%2C+Alessio+Ferrato%2C+and+Carla+Limongelli%2E+2024%2E+Multiple%2Dchoice+question+generation+using+large+language+models%3A+Methodology+and+educator+insights%2E+In+Adjunct+Proceedings+of+the+32nd+AC) *Updated 31 October 2026*