--- title: "Cross-corpus Native Language Identification via Statistical" description: "The paper tackles Native Language Identification (NLI) when training and test texts come from different… 1 reference, cited by 2 papers here." authors: ["Francisco Rangel", "Paolo Rosso", "Julian Brooke", "Alexandra Uitdenbogerd"] year: 2018 venue: "Proceedings of the Second Workshop on Stylistic Variation" source: "https://aclanthology.org/W18-1605/" doi: "10.18653/v1/w18-1605" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/rangel-2018-cross-corpus-native-language-identification/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rangel-2018-cross-corpus-native-language-identification/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2018](https://md.research.elamur.ai/free/en/papers/2018/page/7/) / Cross-corpus Native Language Identification via Statistical Embedding # Cross-corpus Native Language Identification via Statistical Embedding **Authors:** [Francisco Rangel](https://md.research.elamur.ai/free/en/authors/francisco-rangel/), [Paolo Rosso](https://md.research.elamur.ai/free/en/authors/paolo-rosso/), [Julian Brooke](https://md.research.elamur.ai/free/en/authors/julian-brooke/) and Alexandra Uitdenbogerd · **Year:** 2018 · **Venue:** [Proceedings of the Second Workshop on Stylistic Variation](https://md.research.elamur.ai/free/en/venues/workshop-on-stylistic-variation/) ([Workshop on Stylistic Variation 2018](https://md.research.elamur.ai/free/en/venues/workshop-on-stylistic-variation/2018/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/W18-1605/) · [PDF](https://aclanthology.org/W18-1605.pdf) · [DOI 10.18653/v1/w18-1605](https://doi.org/10.18653/v1/w18-1605) · [OpenAlex](https://openalex.org/W2807182733) **Citations:** 1 citations (OpenAlex, 2026-10-14) · 2 in our library, of the 149,439 papers whose bibliographies we read ## Summary The paper tackles Native Language Identification (NLI) when training and test texts come from different corpora. In this realistic setting, earlier systems lost much of their accuracy. The authors focus on three L1s common among students at Australian universities: Arabic, Chinese and Indonesian. They use Lang8, TOEFL and ICNALE, and evaluate by training on some corpora and testing on a corpus left out of training. Their method, Low Dimensionality Statistical Embedding (LDSE), works in three steps: - It computes tf-idf weights for every term. - It scores each term by the share of its total weight that falls in documents of a given L1. - It describes each document by summary statistics of those scores, giving 17 features per class. LDSE needs no external resources, preprocessing or feature engineering, and it discards no vocabulary. Each language is treated as a one-versus-all task. Against the best SVM n-gram baselines, LDSE wins in every train/test pair, with significant gains of about 4–19% in macro-averaged F1. It also stays competitive on a tiny, imbalanced case-study corpus. This suggests that compact distributional descriptors carry over between corpora better than raw n-gram features. ## Key points - **Corpora:** Lang8 and TOEFL are used for Arabic and Chinese, and Lang8 and ICNALE for Indonesian. The data is heavily imbalanced; ICNALE has only 8 Indonesian documents against 74 from other L1s. - **Features:** each class gets 17 features drawn from the distribution of term weights in a document: - mean and median; - min and max; - first and third quartiles; - two Gini-based indexes for skewness and kurtosis; - moments M2 to M10. - **Baseline:** SVM over character or word n-grams, with n from 1 to 5 and the top 100 to 5,000 most frequent terms. The best baseline configurations all used the top 1,000 n-grams. - **Results (macro-F1, LDSE vs. baseline):** - Arabic, Lang8→TOEFL: 65.30 vs. 54.75, the largest relative gain at 19.27%. - Indonesian, Lang8→ICNALE: 86.15 vs. 73.05, the highest absolute score. - Chinese: the smallest gains, 4.39–6.95%. - Training on Lang8 gave the best results for every language, possibly because it is larger and its authors write on freely chosen topics. - Significance was checked with a t-test at 95%. - **Case study:** the Student Writing Task (SWT) corpus has 32 single-topic essays by Computer Science PhD students in Australia, covering 16 L1s. SWT was used for testing only, never for training. ## Interesting ideas - Hold out one entire corpus for testing and train on the rest, so the score shows how a model survives a real change of source rather than a random split. - Turn class-conditional term scores into a short vector of distribution statistics. This keeps rare terms without an explosion in dimensionality, unlike frequency cutoffs, PCA or LSA. - Use one-versus-all binary tasks when label inventories cannot be matched across datasets, since a multi-class setup would need the labels aligned first. - Report macro-averaged F1 on imbalanced data, and treat scores from tiny test sets (like 8 positive documents) as unstable. - Training data with free choice of topics transferred best, which hints that topic diversity matters as much as corpus size. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (learn): Train the parsers with one treebank or genre withheld and test on it, as a regular check that silver-data gains are not tied to one source. - **I2** (learn): The same summary statistics could describe how far an input text drifts from the training treebanks, for example through rare-lemma or parser-confidence profiles, to flag texts where analyses deserve less trust. - **I3** (frames): When speech-act or role datasets use mismatched category sets, train and evaluate one detector per category instead of forcing a shared label scheme. - **I4** (answer): Score school-test and word-problem readers per question type with macro averages, and mark results from small categories as low-confidence. - **I5** (learn): When choosing texts for silver data or LLM-teacher annotation, prefer variety of topics over more text from a narrow domain. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [Universitat Politècnica de València](https://md.research.elamur.ai/free/en/universities/universitat-politecnica-de-valencia/) — University, Valencia, Spain · 34 works · 3,575 citations - [RMIT University](https://md.research.elamur.ai/free/en/universities/rmit-university/) — University, Melbourne, Australia · 17 works · 703 citations Also on the byline: [Thomson Reuters (Canada)](https://ror.org/01r4zz038). 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works 🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 2,908 works ## Cite Cite as: Francisco Rangel, Paolo Rosso, Julian Brooke and Alexandra Uitdenbogerd. 2018. Cross-corpus Native Language Identification via Statistical Embedding. Proceedings of the Second Workshop on Stylistic Variation. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{rangel2018cross, title = {{Cross-corpus Native Language Identification via Statistical Embedding}}, author = {Francisco Rangel, Paolo Rosso, Julian Brooke and Alexandra Uitdenbogerd}, booktitle = {Proceedings of the Second Workshop on Stylistic Variation}, year = {2018}, doi = {10.18653/v1/w18-1605}, url = {https://doi.org/10.18653/v1/w18-1605} } ``` - [← previous Characterizing the Influence of Features on Reading Difficulty Estimation for Non-native Readers](https://md.research.elamur.ai/free/en/papers/huang-2018-characterizing-influence-features-reading-difficulty/) - [next → English Out-of-Vocabulary Lexical Evaluation Task](https://md.research.elamur.ai/free/en/papers/wang-2018-english-out-vocabulary-lexical-evaluation/) - [↑ back to papers of 2018](https://md.research.elamur.ai/free/en/papers/2018/page/7/) ## References 1 references, 0 of them with a card of ours. ↗ leads to the work. 1. Daniel Blanchard, Joel Tetreault, Derrick Higgins, Aoife Cahill, and Martin Chodorow. 2013. Toefl11: A corpus of non-native english. ETS Research Report Series 2013(2). [↗](https://doi.org/10.1002/j.2333-8504.2013.tb02331.x) *Updated 1 November 2026*