---
title: "Corpus-Guided Contrast Sets for Morphosyntactic Feature"
description: "The authors propose CGEdit, a human-in-the-loop way to build contrast sets for training feature… 44 references, cited by 7 papers here."
authors: ["Tessa Masis", "Anissa Neal", "Lisa Green", "Brendan O'Connor"]
year: 2022
venue: "Proceedings of the First Workshop on NLP applications to field linguistics"
source: "https://aclanthology.org/2022.fieldmatters-1.2/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/masis-2022-corpus-guided-contrast-sets-morphosyntactic/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/masis-2022-corpus-guided-contrast-sets-morphosyntactic/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/6/) / Corpus-Guided Contrast Sets for Morphosyntactic Feature Detection in Low-Resource English Varieties

# Corpus-Guided Contrast Sets for Morphosyntactic Feature Detection in Low-Resource English Varieties

**Authors:** [Tessa Masis](https://md.research.elamur.ai/free/en/authors/tessa-masis/), Anissa Neal, [Lisa Green](https://md.research.elamur.ai/free/en/authors/lisa-green/) and [Brendan O'Connor](https://md.research.elamur.ai/free/en/authors/brendan-o-connor/) · **Year:** 2022 · **Venue:** [Proceedings of the First Workshop on NLP applications to field linguistics](https://md.research.elamur.ai/free/en/venues/fieldmatters/) ([Workshop on NLP Applications to Field Linguistics 2022](https://md.research.elamur.ai/free/en/venues/fieldmatters/2022/)) · **Publisher:** [International Committee on Computational Linguistics (ICCL)](https://md.research.elamur.ai/free/en/publishers/international-committee-on-computational-linguistics-iccl/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.fieldmatters-1.2/) · [PDF](https://aclanthology.org/2022.fieldmatters-1.2.pdf) · [OpenAlex](https://openalex.org/W4296415429)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [7](https://md.research.elamur.ai/free/en/papers/masis-2022-corpus-guided-contrast-sets-morphosyntactic/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Sociolinguists often study variation by counting dialect features such as zero copula, habitual *be* or *finna*. Hand annotation is expensive, and automatic detection is hard for informal, low-resource English varieties. The authors propose CGEdit, a human-in-the-loop way to build contrast sets for training feature classifiers.

For each feature, the authors write five positive sentences. Each 3-gram in those sentences is replaced by a similar, frequent n-gram taken from the target corpus, which produces near-miss variants. A person then keeps a few positives and negatives per seed. The resulting sets train multi-head BERT classifiers, one binary head per feature.

The method was tested on Indian English (ICE-India) and African American English (CORAAL and the FWP slave narratives). CGEdit, especially when combined with manually written negatives, outperformed purely manual and fully automatic baselines, by up to 16 Prec@100 points over prior work. Run over all of CORAAL, the models reproduced conclusions from three manually annotated sociolinguistic studies. This shows such detectors can support large-scale variation research. Models and training data are released.

## Key points
- **Data:**
  - ICE-India: 21,759 spontaneous dialogue utterances, with 10 syntactic features annotated by Lange (2012).
  - CORAAL: 152,069 utterances from interviews recorded 1968–2017 at six US sites.
  - FWP: 148,018 utterances from 17 states, used mainly for evaluation because of authenticity concerns.
  - The 17 AAE features were drawn from the feature inventories in Green (2002) and Koenecke et al. (2020). The top 100 predictions per feature were hand-checked.
- **CGEdit procedure:**
  - 5 seed positives per feature.
  - A replacement n-gram (2–4 tokens) may differ from the original 3-gram by at most one subtoken, so edits act as deletions, insertions or reorderings.
  - Each seed yields roughly 10–50 candidates. Filtering takes 30–60 seconds per seed and keeps 2 positives and 3 negatives.
- **Training setup:**
  - bert-base-uncased for IndE, bert-base-cased for AAE.
  - 500 epochs, Adam, batch size 64, learning rate 1e-5, warm-up over 150 epochs.
- **ICE-India results (ROC-AUC / AP / Prec@100):**
  - MANUALGEN + CGEDIT: 88.76 / 29.32 / 35.67.
  - MANUALGEN alone: 86.83 / 25.77 / 31.63.
  - Shuffled n-gram negatives (AUTOGEN): 68.94 / 12.63 / 16.93.
- **AAE Prec@100:**
  - CORAAL: CGEDIT 67.41 vs. 57.88 for random corpus negatives (AUTOID).
  - FWP: the combined model reached 74.35.
- **Replication of prior studies (with Classify & Count):**
  - Confirmed regional and gender patterns from Koenecke et al.: lowest feature use in Rochester, then DC, then Princeville; lower use among men.
  - Quantified the speaker variation within age groups reported by Cukor-Avila and Balcazar.
  - Confirmed Grieser's negative link between feature use and both age and SES. The gender effect depended on which subset of features and data was used.

## Interesting ideas
- Negatives are built to break a feature's syntactic constraint with one small edit, such as reinserting a copula, which sharpens the decision boundary far more than random negatives.
- Edits come from frequent n-grams of the target corpus itself, so perturbed sentences stay plausible for that variety, unlike random edits that tend to produce ungrammatical output.
- A person filters machine-proposed candidates instead of writing examples from scratch, which takes under a minute per seed yet still yields high-quality contrast data.
- The detectors are validated by checking whether large-scale counts reproduce conclusions of earlier hand-annotated studies, not only by test-set scores.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Near-miss sentences that minimally violate a construction could serve as negative controls in evaluation, showing whether a learned repair rule fires only where it should.
- **I2** (tree): Corpus-guided swaps could generate dialect probe sentences (e.g., zero copula, habitual *be*) to test how the perceptron parser and gparse attach predicates when the expected auxiliary is missing.
- **I3** (learn): An LLM teacher tier could play the filtering role, accepting or rejecting generated variants before they enter silver training data.
- **I4** (answer): Readers could be checked by whether aggregate answers over many texts match known reference results, adding an external sanity check beyond per-item accuracy.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Massachusetts Amherst](https://md.research.elamur.ai/free/en/universities/university-of-massachusetts-amherst/) — University, Amherst Center, United States · 106 works · 66,692 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works · [🌱 Low-resource languages](https://md.research.elamur.ai/free/en/topics/low-resource-languages/) · 267 works

## Cite

Cite as: Tessa Masis, Anissa Neal, Lisa Green and Brendan O'Connor. 2022. Corpus-Guided Contrast Sets for Morphosyntactic Feature Detection in Low-Resource English Varieties. Proceedings of the First Workshop on NLP applications to field linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{masis2022corpus,
  title = {{Corpus-Guided Contrast Sets for Morphosyntactic Feature Detection in Low-Resource English Varieties}},
  author = {Tessa Masis, Anissa Neal, Lisa Green and Brendan O'Connor},
  booktitle = {Proceedings of the First Workshop on NLP applications to field linguistics},
  year = {2022},
  url = {https://aclanthology.org/2022.fieldmatters-1.2/}
}
```

- [← previous Contrastive Representation Learning for Cross-Document Coreference Resolution of Events and Entities](https://md.research.elamur.ai/free/en/papers/hsu-2022-contrastive-representation-learning-cross-document/)
- [next → How Much Does Lookahead Matter for Disambiguation? Partial Arabic Diacritization Case Study](https://md.research.elamur.ai/free/en/papers/esmail-2022-how-much-does-lookahead-matter/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/6/)

## References

44 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ahmed Abdelali, Hamdy Mubarak, Younes Samih, Sabit Hassan, and Kareem Darwish. 2021. [QADI: Arabic Dialect Identification in the Wild](https://md.research.elamur.ai/free/en/papers/abdelali-2021-qadi-arabic-dialect-identification-wild/). In Proceedings of the Sixth Arabic Natural Language Processing Workshop, pages 1–10, Kyiv, Ukraine (Virtual). Association for Computational Linguistics.
2. Patricia Cukor-Avila and Ashley Balcazar. 2019. Exploring Grammatical Variation in the Corpus of Regional African American Language. American Speech, 94(1):36–53. [↗](https://doi.org/10.1215/00031283-7321989)
3. Mark Davies. 2016. Corpus del Español: Web/Dialects. [find](https://scholar.google.com/scholar?q=Mark+Davies%2E+2016%2E+Corpus+del+Espa%C3%B1ol%3A+Web%2FDialects%2E)
4. Matteo Alleman, Jonathan Mamou, Miguel A Del Rio, Hanlin Tang, Yoon Kim, and SueYeon Chung. 2021. Syntactic Perturbations Reveal Representational Correlates of Hierarchical Phrase Structure in Pretrained Language Models. In Proceedings of the 6th Workshop on Representation Learning for NLP (RepL4NLP-2021), pages 263–276, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Matteo+Alleman%2C+Jonathan+Mamou%2C+Miguel+A+Del+Rio%2C+Hanlin+Tang%2C+Yoon+Kim%2C+and+SueYeon+Chung%2E+2021%2E+Syntactic+Perturbations+Reveal+Representational+Correlates+of+Hierarchical+Phrase+Structure+in+Pretrai)
5. Dorottya Demszky, Devyani Sharma, Jonathan Clark, Vinodkumar Prabhakaran, and Jacob Eisenstein. 2021. Learning to Recognize Dialect Features. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2315–2338, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.naacl-main.184)
6. Martha Austen. 2017. “Put the Groceries Up”: Comparing Black and White Regional Variation. American Speech, 92(3):298–320. [↗](https://doi.org/10.1215/00031283-4312064)
7. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In NAACL-HLT (1), pages 4171–4186.
8. Jessa Bekker and Jesse Davis. 2020. Learning from Positive and Unlabeled Data: A Survey. Mach. Learn., 109(4):719–760. [↗](https://doi.org/10.1007/s10994-020-05877-5)
9. Jacob Eisenstein. 2015. Systematic patterning in phonologically-motivated orthographic variation. Journal of Sociolinguistics, 19(2):161–188. [↗](https://doi.org/10.1111/josl.12119)
10. Antonio Bella, Cesar Ferri, José Hernández-Orallo, and María José Ramírez-Quintana. 2010. Quantification via Probability Estimators. In 2010 IEEE International Conference on Data Mining, pages 737–742. [find](https://scholar.google.com/scholar?q=Antonio+Bella%2C+Cesar+Ferri%2C+Jos%C3%A9+Hern%C3%A1ndez%2DOrallo%2C+and+Mar%C3%ADa+Jos%C3%A9+Ram%C3%ADrez%2DQuintana%2E+2010%2E+Quantification+via+Probability+Estimators%2E+In+2010+IEEE+International+Conference+on+Data+Mining%2C+pages+737%E2%80%93742)
11. ∀, Wilhelmina Nekoto, Vukosi Marivate, Tshinondiwa Matsila, Timi Fasubaa, Taiwo Fagbohungbe, Solomon Oluwole Akinola, Shamsuddeen Muhammad, Salomon Kabongo Kabenamualu, Salomey Osei, Freshia Sackey, Rubungo Andre Niyongabo, Ricky Macharm, Perez Ogayo, Orevaoghene Ahia, Musie Meressa Berhe, Mofetoluwa Adeyemi, Masabata Mokgesi-Selinga, Lawrence Okegbemi, Laura Martinus, Kolawole Tajudeen, Kevin Degila, Kelechi Ogueji, Kathleen Siminyu, Julia Kreutzer, Jason Webster, Jamiil Toure Ali, Jade Abbott, Iroro Orife, Ignatius Ezeani, Idris Abdulkadir Dangana, Herman Kamper, Hady Elsahar, Goodness Duru, Ghollah Kioko, Murhabazi Espoir, Elan van Biljon, Daniel Whitenack, Christopher Onyefuluchi, Chris Chinenye Emezue, Bonaventure F. P. Dossou, Blessing Sibanda, Blessing Bassey, Ayodele Olabiyi, Arshath Ramkilowan, Alp Öktem, Adewale Akinfaderin, and Abdallah Bashir. 2020. [Participatory Research for Low-resourced Machine Translation: A Case Study in African Languages](https://md.research.elamur.ai/free/en/papers/nekoto-2020-participatory-research-low-resourced-machine/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 2144–2160, Online. Association for Computational Linguistics.
12. Su Lin Blodgett, Lisa Green, and Brendan O’Connor. 2016. [Demographic Dialectal Variation in Social Media: A Case Study of African-American English](https://md.research.elamur.ai/free/en/papers/blodgett-2016-demographic-dialectal-variation-social-media/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1119–1130, Austin, Texas. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/d16-1120)
13. Franck Burlot and François Yvon. 2017. Evaluating the morphological competence of Machine Translation Systems. In Proceedings of the Second Conference on Machine Translation, pages 43–55, Copenhagen, Denmark. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Franck+Burlot+and+Fran%C3%A7ois+Yvon%2E+2017%2E+Evaluating+the+morphological+competence+of+Machine+Translation+Systems%2E+In+Proceedings+of+the+Second+Conference+on+Machine+Translation%2C+pages+43%E2%80%9355%2C+Copenhagen%2C+)
14. Stevie Chancellor, Michael L. Birnbaum, Eric D. Caine, Vincent M. B. Silenzio, and Munmun De Choudhury. 2019. A Taxonomy of Ethical Tensions in Inferring Mental Health States from Social Media. In Proceedings of the Conference on Fairness, Accountability, and Transparency, FAT* ’19, page 79–88, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Stevie+Chancellor%2C+Michael+L%2E+Birnbaum%2C+Eric+D%2E+Caine%2C+Vincent+M%2E+B%2E+Silenzio%2C+and+Munmun+De+Choudhury%2E+2019%2E+A+Taxonomy+of+Ethical+Tensions+in+Inferring+Mental+Health+States+from+Social+Media%2E+In+Pro)
15. Matt Gardner, Yoav Artzi, Victoria Basmov, Jonathan Berant, Ben Bogin, Sihao Chen, Pradeep Dasigi, Dheeru Dua, Yanai Elazar, Ananth Gottumukkala, Nitish Gupta, Hannaneh Hajishirzi, Gabriel Ilharco, Daniel Khashabi, Kevin Lin, Jiangming Liu, Nelson F. Liu, Phoebe Mulcaire, Qiang Ning, Sameer Singh, Noah A. Smith, Sanjay Subramanian, Reut Tsarfaty, Eric Wallace, Ally Zhang, and Ben Zhou. 2020. [Evaluating Models’ Local Decision Boundaries via Contrast Sets](https://md.research.elamur.ai/free/en/papers/gardner-2020-evaluating-models-local-decision-boundaries/). In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1307–1323, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2020.findings-emnlp.117)
16. Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. 2020. [A Simple Framework for Contrastive Learning of Visual Representations](https://md.research.elamur.ai/free/en/papers/chen-2020-simple-framework-contrastive-learning-visual/). In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pages 1597–1607. PMLR. [↗](https://doi.org/10.48550/arxiv.2002.05709)
17. Noam Chomsky. 1957. Syntactic structures. Mouton. [find](https://scholar.google.com/scholar?q=Noam+Chomsky%2E+1957%2E+Syntactic+structures%2E+Mouton%2E)
18. Holly K Craig and Julie A Washington. 2006. Malik goes to school: Examining the language skills of Lisa Green. 2017. Beyond Lists of Differences to Accurate Descriptions. In Data Collection in Sociolinguistics, pages 281–284. Routledge. Systems, pages 33–39, Copenhagen, Denmark. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Holly+K+Craig+and+Julie+A+Washington%2E+2006%2E+Malik+goes+to+school%3A+Examining+the+language+skills+of+Lisa+Green%2E+2017%2E+Beyond+Lists+of+Differences+to+Accurate+Descriptions%2E+In+Data+Collection+in+Socioli)
19. Natalie Maynor. 1988. Written Records of Spoken Language: How Reliable Are They. Methods in Dialectology, pages 109–20. [find](https://scholar.google.com/scholar?q=Natalie+Maynor%2E+1988%2E+Written+Records+of+Spoken+Language%3A+How+Reliable+Are+They%2E+Methods+in+Dialectology%2C+pages+109%E2%80%9320%2E)
20. Lisa J Green. 2002. African American English: A Linguistic Introduction. Cambridge University Press. [↗](https://doi.org/10.1017/cbo9780511800306)
21. Sidney Greenbaum and Gerald Nelson. 1996. The International Corpus of English (ICE) Project. World Englishes, 15(1):3–15. [↗](https://doi.org/10.1111/j.1467-971x.1996.tb00088.x)
22. Zhengjie Miao, Yuliang Li, Xiaolan Wang, and Wang-Chiew Tan. 2020. Snippext: Semi-Supervised Opinion Mining with Augmented Data, page 617–628. Association for Computing Machinery, New York, NY, USA. [find](https://scholar.google.com/scholar?q=Zhengjie+Miao%2C+Yuliang+Li%2C+Xiaolan+Wang%2C+and+Wang%2DChiew+Tan%2E+2020%2E+Snippext%3A+Semi%2DSupervised+Opinion+Mining+with+Augmented+Data%2C+page+617%E2%80%93628%2E+Association+for+Computing+Machinery%2C+New+York%2C+NY%2C+USA%2E)
23. Jessica A. Grieser. 2019. Investigating Topic-Based Style Shifting in the Classic Sociolinguistic Interview. American Speech, 94(1):54–71. [↗](https://doi.org/10.1215/00031283-7322011)
24. Dong Nguyen, A. Seza Doğruöz, Carolyn P. Rosé, and Franciska de Jong. 2016. [Computational Sociolinguistics: A Survey](https://md.research.elamur.ai/free/en/papers/nguyen-2016-computational-sociolinguistics-survey/). Comput. Linguist., 42(3):537–593. [↗](https://doi.org/10.1162/coli_a_00258)
25. Jack Grieve, Douglas Biber, Eric Friginal, and Tatiana Nekrasova. 2011. Variation Among Blogs: A Multi-dimensional Analysis, pages 303–322. Springer Netherlands, Dordrecht. [find](https://scholar.google.com/scholar?q=Jack+Grieve%2C+Douglas+Biber%2C+Eric+Friginal%2C+and+Tatiana+Nekrasova%2E+2011%2E+Variation+Among+Blogs%3A+A+Multi%2Ddimensional+Analysis%2C+pages+303%E2%80%93322%2E+Springer+Netherlands%2C+Dordrecht%2E)
26. Jennifer Renn and J. Michael Terry. 2009. Operationalizing Style: Quantifying the Use of Style Shift in the Speech of African American Adolescents. American Speech, 84(4):367–390. [↗](https://doi.org/10.1215/00031283-2009-030)
27. Kristina Gulordava, Piotr Bojanowski, Edouard Grave, Tal Linzen, and Marco Baroni. 2018. [Colorless Green Recurrent Networks Dream Hierarchically](https://md.research.elamur.ai/free/en/papers/gulordava-2018-colorless-green-recurrent-networks-dream/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1195–1205, New Orleans, Louisiana. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n18-1108)
28. Alexis Ross, Tongshuang Wu, Hao Peng, Matthew E. Peters, and Matt Gardner. 2021. [Tailor: Generating and Perturbing Text with Semantic Controls](https://md.research.elamur.ai/free/en/papers/ross-2022-tailor-generating-perturbing-text-semantic/). CoRR, abs/2107.07150. [↗](https://arxiv.org/abs/2107.07150)
29. Marco Saerens, Patrice Latinne, and Christine Decaestecker. 2002. Adjusting the Outputs of a Classifier to New a Priori Probabilities: A Simple Procedure. Neural Computation, 14(1):21–41. [↗](https://doi.org/10.1162/089976602753284446)
30. Taylor Jones. 2015. Toward a Description of African American Vernacular English Dialect Regions Using “Black Twitter”. American Speech, 90(4):403–440. [find](https://scholar.google.com/scholar?q=Taylor+Jones%2E+2015%2E+Toward+a+Description+of+African+American+Vernacular+English+Dialect+Regions+Using+%E2%80%9CBlack+Twitter%E2%80%9D%2E+American+Speech%2C+90%284%29%3A403%E2%80%93440%2E)
31. Harrison Santiago, Joshua Martin, Sarah Moeller, and Kevin Tang. 2022. Disambiguation of morphosyntactic features of African American English – the case of habitual be. [find](https://scholar.google.com/scholar?q=Harrison+Santiago%2C+Joshua+Martin%2C+Sarah+Moeller%2C+and+Kevin+Tang%2E+2022%2E+Disambiguation+of+morphosyntactic+features+of+African+American+English+%E2%80%93+the+case+of+habitual+be%2E)
32. Tyler Kendall and Charlie Farrington. 2021. The Corpus of Regional African American Language. Eugene, OR: The Online Resources for African American Language Project. [↗](https://doi.org/10.7264/1ad5-6t35)
33. Allison Koenecke, Andrew Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, John R. Rickford, Dan Jurafsky, and Sharad Goel. 2020. Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences, 117(14):7684–7689. [↗](https://doi.org/10.1073/pnas.1915768117)
34. Rico Sennrich. 2017. [How Grammatical is Character-level Neural Machine Translation? Assessing MT Quality with Contrastive Translation Pairs](https://md.research.elamur.ai/free/en/papers/sennrich-2017-how-grammatical-is-character-level/). In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 376–382, Valencia, Spain. Association for Computational Linguistics.
35. Jacob Leon Kröger, Leon Gellrich, Sebastian Pape, Saba Rebecca Brause, and Stefan Ullrich. 2022. Personal information inference from voice recordings: User awareness and privacy concerns. Proceedings on Privacy Enhancing Technologies, 2022(1):6–27. [↗](https://doi.org/10.2478/popets-2022-0002)
36. Devyani Sharma. 2017. Scalar effects of social networks on language variation. Language Variation and Change, 29(3):393–418. [↗](https://doi.org/10.1017/s0954394517000205)
37. Devyani Sharma. 2018. Style dominance: Attention, audience, and the ‘real me’. Language in Society, 47(1):1–31. [↗](https://doi.org/10.1017/s0047404517000835)
38. Claudia Lange. 2012. The Syntax of Spoken Indian English, volume 45. John Benjamins Publishing. [find](https://scholar.google.com/scholar?q=Claudia+Lange%2E+2012%2E+The+Syntax+of+Spoken+Indian+English%2C+volume+45%2E+John+Benjamins+Publishing%2E)
39. Manuscript Division Library of Congress. 2001. Born in Slavery: Slave Narratives from the Federal Writers’ Project, 1936 to 1938. [find](https://scholar.google.com/scholar?q=Manuscript+Division+Library+of+Congress%2E+2001%2E+Born+in+Slavery%3A+Slave+Narratives+from+the+Federal+Writers%E2%80%99+Project%2C+1936+to+1938%2E)
40. Noah A. Smith and Jason Eisner. 2005. [Contrastive Estimation: Training Log-Linear Models on Unlabeled Data](https://md.research.elamur.ai/free/en/papers/smith-2005-contrastive-estimation-training-log-linear/). In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, ACL ’05, page 354–362, USA. Association for Computational Linguistics.
41. Taylor Mahler, Willy Cheung, Micha Elsner, David King, Marie-Catherine de Marneffe, Cory Shain, Symon Stevens-Guille, and Michael White. 2017. [Breaking NLP: Using Morphosyntax, Semantics, Pragmatics and World Knowledge to Fool Sentiment Analysis Systems](https://md.research.elamur.ai/free/en/papers/mahler-2017-breaking-nlp-morphosyntax-semantics-pragmatics/). In Proceedings of the First Workshop on Building Linguistically Generalizable NLP Ieva Staliūnaitė and Ben Bonfil. 2017. Breaking Sentiment Analysis of Movie Reviews. In Proceedings of the First Workshop on Building Linguistically Generalizable NLP Systems, pages 61–64, Copenhagen, Denmark. Association for Computational Linguistics.
42. Alexandre Veronese, Alessandra Silveira, and Amanda Nunes Lopes Espiñeira Lemos. 2019. Artificial intelligence, Digital Single Market and the proposal of a right to fair and reasonable inferences: a legal issue between ethics and techniques. UNIO – EU Law Journal, 5(2):75–91. [↗](https://doi.org/10.21814/unio.5.2.2294)
43. C Wilson and V Mihalicek. 2011. Language Files: Materials for an Introduction to Language and Linguistics. Columbus: Ohio State University Press. [find](https://scholar.google.com/scholar?q=C+Wilson+and+V+Mihalicek%2E+2011%2E+Language+Files%3A+Materials+for+an+Introduction+to+Language+and+Linguistics%2E+Columbus%3A+Ohio+State+University+Press%2E)
44. Walt Wolfram. 1990. Re-Examining Vernacular Black English. Language, 66(1):121–133. [↗](https://doi.org/10.1353/lan.1990.0043)

*Updated 1 November 2026*