--- title: "To What Extent Does Lexical Normalization Help" description: "Ehara builds two readability assessors and checks them on OneStopEnglish, a parallel corpus of… 1 reference, cited by 2 papers here." authors: ["Yo Ehara"] year: 2021 venue: "Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021)" source: "https://aclanthology.org/2021.wnut-1.50/" doi: "10.18653/v1/2021.wnut-1.50" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/ehara-2021-what-extent-does-lexical-normalization/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ehara-2021-what-extent-does-lexical-normalization/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/8/) / To What Extent Does Lexical Normalization Help English-as-a-Second Language Learners to Read Noisy English Texts? # To What Extent Does Lexical Normalization Help English-as-a-Second Language Learners to Read Noisy English Texts? **Authors:** [Yo Ehara](https://md.research.elamur.ai/free/en/authors/yo-ehara/) · **Year:** 2021 · **Venue:** [Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021)](https://md.research.elamur.ai/free/en/venues/wnut/) ([Workshop on Noisy User-generated Text 2021](https://md.research.elamur.ai/free/en/venues/wnut/2021/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2021.wnut-1.50/) · [PDF](https://aclanthology.org/2021.wnut-1.50.pdf) · [DOI 10.18653/v1/2021.wnut-1.50](https://doi.org/10.18653/v1/2021.wnut-1.50) · [OpenAlex](https://openalex.org/W3212289689) **Citations:** 0 citations (OpenAlex, 2026-10-14) · 2 in our library, of the 149,439 papers whose bibliographies we read ## Summary The paper asks whether noisy social-media English is hard for ESL learners to read, and whether lexical normalization makes it easier. Ehara builds two readability assessors and checks them on OneStopEnglish, a parallel corpus of Guardian articles that teachers rewrote at three levels. The first assessor is supervised: a fine-tuned BERT classifier called spvBERT. The second is unsupervised. It applies item response theory to learners' vocabulary-test answers and extends word difficulty to the whole lexicon using BNC and COCA frequencies. A text's score is then the chance that an average learner knows every word in it. The two assessors largely agree, and both beat classic readability formulae. Applied to the English W-NUT 2021 lexical normalization data, they show that most noisy texts fall at elementary or intermediate level, so intermediate learners can already read them. Normalization still makes texts easier, and this gain is statistically significant for the vocabulary-based assessor. The study gives a first estimate of how much normalization actually matters to human non-native readers. ## Key points - **OneStopEnglish data:** 567 texts, 189 per level (elementary, intermediate, advanced), split into 339 train, 114 validation and 114 test. Unsupervised methods saw only the test split. - **spvBERT setup:** fine-tuned from `bert-large-cased-whole-word-masking` with Adam for 10 epochs at a 1e-5 learning rate. - **Vocabulary-based assessor:** trained on Ehara (2018) data, in which 100 learners took the 100-item Vocabulary Size Test. Word difficulty is a learned weighted sum of log(freq+1) from BNC and COCA. - **OneStopEnglish results:** - Vocabulary-based: Spearman 0.730 / Pearson 0.715, the best unsupervised method. - spvBERT: Spearman 0.866 / Pearson 0.864. - Classic formulae such as Dale-Chall and SMOG reached only about 0.3–0.5. - **Noisy-text results:** 1,325 non-identical noisy/gold-normalized pairs were taken from the 2,361 W-NUT MultiLexNorm English training texts. - spvBERT moved 117 texts from elementary to intermediate and 97 the other way, so its change was not significant. - The mean vocabulary-based score fell from 4.00 to 3.53 (Wilcoxon, p \< 0.01). ## Interesting ideas - Item response theory places learner ability and item difficulty on one shared logistic scale, and a frequency regression extends difficulty to words never tested; this turns a small test into a lexicon-wide graded resource. - A readability score defined as the negative log probability that an average learner knows all the words is fully decomposable, so each word's contribution to difficulty can be inspected. - A strong black-box supervised model and a transparent unsupervised model are trained separately, and their agreement is used as evidence that both are measuring something real. - The evaluation corpus was chosen so that trivial cues like sentence length or topic cannot predict the labels, because the same article appears at every level. - The effect of normalization was measured on paired before/after texts, with identical pairs dropped, a confusion matrix showing changes in both directions, and a paired significance test. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (answer): Student readers answering school-course tests could be modelled with an ability parameter against item or word difficulty, so a wrong answer can be traced to specific unknown words. - **I2** (world): A per-word difficulty value can be stored in compiled lexical knowledge, letting Mova explain which tokens make a passage hard instead of returning an opaque grade. - **I3** (learn): Agreement between an interpretable rule-based witness and a neural witness can serve as a check before accepting silver labels, with disagreements sent for review. - **I4** (learn): Building parallel or minimally different test items works as a built-in negative control, ensuring parsers or readers are not scoring through surface shortcuts. - **I5** (tokens): Any spelling or normalization pre-step in the tokenizer can be evaluated the same way, reporting both improvements and reversals on the downstream tags rather than one average. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Tokyo Gakugei University](https://md.research.elamur.ai/free/en/universities/tokyo-gakugei-university/) — University, Tokyo, Japan · 2 works · 3 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Developmental and Educational Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/developmental-and-educational-psychology/) · 499 works 🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works · [🖍️ Grammatical error correction & learner language](https://md.research.elamur.ai/free/en/topics/grammatical-error-correction/) · 156 works ## Cite Cite as: Yo Ehara. 2021. To What Extent Does Lexical Normalization Help English-as-a-Second Language Learners to Read Noisy English Texts?. Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{ehara2021what, title = {{To What Extent Does Lexical Normalization Help English-as-a-Second Language Learners to Read Noisy English Texts?}}, author = {Yo Ehara}, booktitle = {Proceedings of the Seventh Workshop on Noisy User-generated Text (W-NUT 2021)}, year = {2021}, doi = {10.18653/v1/2021.wnut-1.50}, url = {https://doi.org/10.18653/v1/2021.wnut-1.50} } ``` - [← previous The Utility and Interplay of Gazetteers and Entity Segmentation for Named Entity Recognition in English](https://md.research.elamur.ai/free/en/papers/agarwal-2021-utility-interplay-gazetteers-entity-segmentation/) - [next → Using a New Nonlinear Gradient Method for Solving Large Scale Convex Optimization Problems with an Application on Arabic Medical Text](https://md.research.elamur.ai/free/en/papers/hammoud-2021-using-new-nonlinear-gradient-method/) - [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/8/) ## References 1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Jonathan Anderson. 1983. Lix and rix: Variations on a little-known readability index. Journal of Reading, 26(6):490–496. *Figure 2: Eq. 3 Scores before/after Normalization* [find](https://scholar.google.com/scholar?q=Jonathan+Anderson%2E+1983%2E+Lix+and+rix%3A+Variations+on+a+little%2Dknown+readability+index%2E+Journal+of+Reading%2C+26%286%29%3A490%E2%80%93496%2E+%2AFigure+2%3A+Eq%2E+3+Scores+before%2Fafter+Normalization%2A) *Updated 1 November 2026*