--- title: "GrammarGPT — paper by Li; and Li (2023) — summary and ideas" description: "Native speakers of Chinese make grammatical errors that sound fluent but still break grammar rules. 3 references, cited by 13 papers here." authors: ["Yaxin Fan; Feng Jiang; Peifeng Li;", "Haizhou Li"] year: 2023 venue: "arXiv:2307.13923" source: "https://arxiv.org/abs/2307.13923" doi: "10.48550/arxiv.2307.13923" arxiv: "2307.13923" licence: "CC BY 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/fan-2023-grammargpt-exploring-open-source-llms/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/fan-2023-grammargpt-exploring-open-source-llms/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/) / GrammarGPT: Exploring Open-Source LLMs for Native Chinese Grammatical Error Correction with Supervised Fine-Tuning # GrammarGPT: Exploring Open-Source LLMs for Native Chinese Grammatical Error Correction with Supervised Fine-Tuning **Authors:** Yaxin Fan; Feng Jiang; Peifeng Li; and [Haizhou Li](https://md.research.elamur.ai/free/en/authors/haizhou-li/) · **Year:** 2023 · **Venue:** [arXiv:2307.13923](https://md.research.elamur.ai/free/en/venues/lecture-notes-in-computer-science/) ([Lecture notes in computer science 2023](https://md.research.elamur.ai/free/en/venues/lecture-notes-in-computer-science/2023/)) · **Publisher:** [Springer Science+Business Media](https://md.research.elamur.ai/free/en/publishers/springer-science-business-media/) · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://arxiv.org/abs/2307.13923) · [PDF](https://arxiv.org/pdf/2307.13923) · [DOI 10.48550/arxiv.2307.13923](https://doi.org/10.48550/arxiv.2307.13923) · [arXiv 2307.13923](https://arxiv.org/abs/2307.13923) · [OpenAlex](https://openalex.org/W4387426216) **Citations:** 35 citations (OpenAlex, 2026-10-14) · [13](https://md.research.elamur.ai/free/en/papers/fan-2023-grammargpt-exploring-open-source-llms/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary Native speakers of Chinese make grammatical errors that sound fluent but still break grammar rules. These errors are harder to catch than learner mistakes, and corpora of them are scarce. The authors ask whether an open-source LLM can be adapted to correct them with very little training data. They built a hybrid parallel set of roughly 1k pairs from two sources: - For error types signalled by surface cues, such as redundant pairs like "more than … about", they prompted ChatGPT with those cues so it would generate faulty sentences. - For cue-less errors, they collected real faulty sentences from public websites and corrected them by hand. An error-invariant augmentation step then swaps named entities for similar ones while leaving the error itself unchanged. They used this data to instruction-tune Phoenix (phoenix-inst-chat-7b) in a Seq2seq setup. The resulting model, GrammarGPT, roughly doubles the F0.5 of the S2S BART baseline. That baseline was trained on about 1.2M learner-error pairs, around 1200× more data. GrammarGPT placed 3rd in NLPCC2023 SharedTask1. The result suggests that small amounts of targeted native-error data can be enough to adapt open LLMs for CGEC. ## Key points - **Dataset:** 1,061 parallel pairs covering six native error types: redundant component, structural confusion, improper collocation, improper word order, improper logicality and missing component. Code and data are released at github.com/FreedomIntelligence/GrammarGPT. - **Main result:** On the validation data, GrammarGPT reaches word-level F0.5 32.56 and char-level F0.5 35.84, measured with the MaxMatch (M2) / ChERRANT scorer. S2S BART trained on Lang8 + HSK scores 17.99 and 17.59. - **Data matters more than size for the baseline:** S2S BART fine-tuned on only the authors' ~1k native-error set scores about the same as on 1.2M learner pairs (17.57 / 18.16). This indicates learner data transfers poorly to native errors. - **Ablation:** Without augmentation, the hybrid set scores 27.30 / 28.22 F0.5, and augmentation raises this to 32.56 / 35.84. Most of the gain comes from Recall. ChatGPT-generated data alone beats human-annotated data alone. - **Training setup:** phoenix-inst-chat-7b (about 20× more parameters than BART), AdamW, learning rate 2e-5 with a linear schedule, batch size 64, 3 epochs, max sequence length 256. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [University of Science and Technology of China](https://md.research.elamur.ai/free/en/universities/university-of-science-and-technology-of-china/) — University, Hefei, China · 145 works · 114,409 citations - [Soochow University](https://md.research.elamur.ai/free/en/universities/soochow-university/) — University, Suzhou, China · 69 works · 4,815 citations - [Chinese University of Hong Kong, Shenzhen](https://md.research.elamur.ai/free/en/universities/chinese-university-of-hong-kong-shenzhen/) — University, Shenzhen, China · 56 works · 3,212 citations - [Shenzhen Research Institute of Big Data](https://md.research.elamur.ai/free/en/universities/shenzhen-research-institute-of-big-data/) — University, Shenzhen, China · 20 works · 2,016 citations 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works 🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🖍️ Grammatical error correction & learner language](https://md.research.elamur.ai/free/en/topics/grammatical-error-correction/) · 162 works ## Cite Cite as: Yaxin Fan; Feng Jiang; Peifeng Li; and Haizhou Li. 2023. GrammarGPT: Exploring Open-Source LLMs for Native Chinese Grammatical Error Correction with Supervised Fine-Tuning. arXiv:2307.13923. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{li2023grammargpt, title = {{GrammarGPT: Exploring Open-Source LLMs for Native Chinese Grammatical Error Correction with Supervised Fine-Tuning}}, author = {Yaxin Fan; Feng Jiang; Peifeng Li; and Haizhou Li}, year = {2023}, doi = {10.48550/arxiv.2307.13923}, eprint = {2307.13923}, archivePrefix = {arXiv}, url = {https://doi.org/10.48550/arxiv.2307.13923} } ``` - [← previous Large Language Models as Tax Attorneys: A Case Study in Legal Capabilities Emergence](https://md.research.elamur.ai/free/en/papers/nay-2023-large-language-models-tax-attorneys/) - [next → Linguistic ambiguity analysis in ChatGPT](https://md.research.elamur.ai/free/en/papers/ortega-martin-2023-linguistic-ambiguity-analysis-chatgpt/) - [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/) ## References 3 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. 1. Chen, F., Feng, Y.: Chain-of-Thought Prompt Distillation for Multimodal Named Entity and Multimodal Relation Extraction. ArXiv preprint arXiv:2306.[14122 (2023) 2](https://md.research.elamur.ai/free/en/papers/chen-2023-chain-thought-prompt-distillation-multimodal/). Chen, Z., Jiang, F., Chen, J., Wang, T., Yu, F., Chen, G., Zhang, H., Liang, J., Zhang, C., Zhang, Z., et al.: Phoenix: Democratizing… 2. Fan et al. 15. Rothe, S., Mallinson, J., Malmi, E., Krause, S., Severyn, A.: A Simple Recipe for Multilingual Grammatical Error Correction. In: Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Proc… — [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/) 3. Springer (2018) 29. Zhao, Z., Wang, H.: MaskGEC: Improving Neural Grammatical Error Correction via Dynamic Masking. Proceedings of the AAAI Conference on Artificial Intelligence 34(01), 1226–1233 (2020) [find](https://scholar.google.com/scholar?q=Springer+%282018%29+29%2E+Zhao%2C+Z%2E%2C+Wang%2C+H%2E%3A+MaskGEC%3A+Improving+Neural+Grammatical+Error+Correction+via+Dynamic+Masking%2E+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence+34%2801%29%2C+1226%E2%80%931233+%28202) *Updated 30 October 2026*