---
title: "Analysis of LLM as a grammatical feature tagger for African"
description: "Two morphosyntactic features serve as test cases: Habitual Be, which is rare and grammatically subtle… 39 references, cited by 2 papers here."
authors: ["Rahul Porwal", "Alice Rozet", "Jotsna Gowda", "Pryce Houck", "Kevin Tang", "Sarah Moeller"]
year: 2025
venue: "Findings of the Association for Computational Linguistics: NAACL 2025"
source: "https://aclanthology.org/2025.findings-naacl.431/"
doi: "10.18653/v1/2025.findings-naacl.431"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/porwal-2025-analysis-llm-as-grammatical-feature/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/porwal-2025-analysis-llm-as-grammatical-feature/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/11/) / Analysis of LLM as a grammatical feature tagger for African American English

# Analysis of LLM as a grammatical feature tagger for African American English

**Authors:** Rahul Porwal, [Alice Rozet](https://md.research.elamur.ai/free/en/authors/alice-rozet/), [Jotsna Gowda](https://md.research.elamur.ai/free/en/authors/jotsna-gowda/), Pryce Houck, [Kevin Tang](https://md.research.elamur.ai/free/en/authors/kevin-tang/) and [Sarah Moeller](https://md.research.elamur.ai/free/en/authors/sarah-moeller/) · **Year:** 2025 · **Venue:** [Findings of the Association for Computational Linguistics: NAACL 2025](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2025](https://md.research.elamur.ai/free/en/venues/findings/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.findings-naacl.431/) · [PDF](https://aclanthology.org/2025.findings-naacl.431.pdf) · [DOI 10.18653/v1/2025.findings-naacl.431](https://doi.org/10.18653/v1/2025.findings-naacl.431) · [OpenAlex](https://openalex.org/W4411120504)

**Citations:** 2 citations (OpenAlex, 2026-10-14) · [2](https://md.research.elamur.ai/free/en/papers/porwal-2025-analysis-llm-as-grammatical-feature/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks whether general-purpose LLMs can detect grammatical features of African American English (AAE), a low-resource variety, without any fine-tuning. Two morphosyntactic features serve as test cases: Habitual Be, which is rare and grammatically subtle, and Multiple Negation (negative concord), which is common. All data come from hand-annotated transcribed oral histories in the Joel Buchanan Archive and from CORAAL.

The task is sentence-level binary tagging, and four systems are compared:
- gpt-4o-mini and LLaMA 3-8B-Instruct, each prompted zero-shot and few-shot;
- an earlier Transformer tagger for Habitual Be;
- a hand-written rule-based tagger for Multiple Negation.

The LLMs look promising against the baseline, but they are not reliable. Their answers are skewed by recency effects from the order of examples, and by properties of the text unrelated to the feature, such as formality. The authors therefore question whether LLMs are worth their cost even for simple tagging. They argue that dialect-aware training is needed, because misreading AAE leads to misrepresenting what its speakers mean.

## Key points
- **Data:** sentences from the Joel Buchanan Archive (SPOHP, University of Florida) and CORAAL, labelled by trained, qualified annotators.
  - Every sentence contains the trigger element, either "be" or a negator. Negative examples therefore cannot be separated from positive ones by simple word search.
  - Habitual Be sentences were limited to those with a single "be".
- **Dataset sizes:**
  - Habitual Be has about 7,739 sentences. Roughly 3,500 of these are synthetic positives, added with the Santiago et al. (2022) augmentation method to balance the classes.
  - The Habitual Be baseline was evaluated with 10-fold cross-validation, using 774 sentences per fold.
  - Multiple Negation has only about 198 sentences.
- **Baselines:**
  - The Habitual Be baseline (Previlon et al., 2024) is a Transformer. It uses n-gram windows around "be", POS-window predictions and syntactic context.
  - The Multiple Negation baseline is a rule tagger. It looks for a second negator within the same clause, with clauses bounded by punctuation or conjunctions.
- **LLM setup:** both models were run at temperature 0.7 and top-p 0.9. Prompts were batched and numbered, asked for a one-word label, and came in zero-shot and few-shot variants.
- **Recency analysis:** recency bias was measured by regression. The predictor was the share of the previous 5 predictions that matched the current one. The authors admit this measure can confuse genuine recency bias with simply being correct on runs of identical labels.

## Interesting ideas
- Build every negative example so that it still contains the trigger word ("be", a negator); a classifier then cannot score by spotting the word and must judge the construction.
- Separate negative concord from ordinary double negation by asking whether both negators sit in the same clause, which turns a semantic notion into a cheap structural check.
- Habitual Be is a bare, uninflected "be" that carries habitual aspect without any adverb, so agreement failure is itself a meaningful signal and not a typo.
- Measure LLM recency bias by regressing each prediction on how many of the last few predictions had the same label, and test with both shuffled and grouped orders of examples.
- Constrain the prompt (one-word answer, keep the item numbering) so batched LLM outputs can be parsed without heavy post-processing.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Mova's evaluation sets with negative controls could use minimal-contrast negatives that share the surface trigger with the positives, so a stage that only keys on a lemma is exposed.
- **I2** (frames): polarity handling in prag could treat several negators attached within one clause in the UD tree as one negation, not two that cancel out.
- **I3** (tokens): the en tagger and morphology could add a dialect-aware analysis in which an invariant "be" after a person-marked subject gets a habitual aspect feature, instead of being flagged as an error or normalised away.
- **I4** (learn): when LLM teacher tiers label silver data in batches, the same check would show whether teacher labels follow batch order rather than content.
- **I5** (learn): teacher-tier prompts could require fixed numbered short labels, which keeps silver annotation easy to audit and align back to the source items.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Heinrich Heine University Düsseldorf](https://md.research.elamur.ai/free/en/universities/heinrich-heine-university-dusseldorf/) — University, Düsseldorf, Germany · 33 works · 1,487 citations
- [University of Florida](https://md.research.elamur.ai/free/en/universities/university-of-florida/) — University, Gainesville, United States · 33 works · 364 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works

## Cite

Cite as: Rahul Porwal, Alice Rozet, Jotsna Gowda, Pryce Houck, Kevin Tang and Sarah Moeller. 2025. Analysis of LLM as a grammatical feature tagger for African American English. Findings of the Association for Computational Linguistics: NAACL 2025.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{porwal2025analysis,
  title = {{Analysis of LLM as a grammatical feature tagger for African American English}},
  author = {Rahul Porwal, Alice Rozet, Jotsna Gowda, Pryce Houck, Kevin Tang and Sarah Moeller},
  booktitle = {Findings of the Association for Computational Linguistics: NAACL 2025},
  year = {2025},
  doi = {10.18653/v1/2025.findings-naacl.431},
  url = {https://doi.org/10.18653/v1/2025.findings-naacl.431}
}
```

- [← previous An Icelandic Linguistic Benchmark for Large Language Models](https://md.research.elamur.ai/free/en/papers/armannsson-2025-icelandic-linguistic-benchmark-large-language/)
- [next → Arabic ASR on the SADA Large-Scale Arabic Speech Corpus with Transformer-Based Models](https://md.research.elamur.ai/free/en/papers/gerazov-2025-arabic-asr-sada-large-scale/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/11/)

## References

39 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Soumya Barikeri, Anne Lauscher, Ivan Vulic, and Goran Glavas. 2021. Redditbias: A real-world resource for bias evaluation and debiasing of conversational language models. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Papers), Virtual Event, August 1-6, 2021, pages 1941–1955. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Soumya+Barikeri%2C+Anne+Lauscher%2C+Ivan+Vulic%2C+and+Goran+Glavas%2E+2021%2E+Redditbias%3A+A+real%2Dworld+resource+for+bias+evaluation+and+debiasing+of+conversational+language+models%2E+In+Proceedings+of+the+59th+An)
2. Su Lin Blodgett, Lisa Green, and Brendan T. O’Connor. 2016. [Demographic dialectal variation in social media: A case study of african-american english](https://md.research.elamur.ai/free/en/papers/blodgett-2016-demographic-dialectal-variation-social-media/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, EMNLP 2016, Austin, Texas, USA, November 1-4, 2016, pages 1119–1130. The Association for Computational Linguistics.
3. Su Lin Blodgett, Johnny Wei, and Brendan O’Connor. 2018. [Twitter Universal Dependency parsing for African-American and mainstream American English](https://md.research.elamur.ai/free/en/papers/blodgett-2018-twitter-universal-dependency-parsing-african/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1415–1425, Melbourne, Australia. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/p18-1131)
4. Lu Cheng, Nayoung Kim, and Huan Liu. 2022. Debiasing word embeddings with nonlinear geometry. In Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, October 12-17, 2022, pages 1286–1298. International Committee on Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2208.13899)
5. Christian Clark, Byung-Doh Oh, and William Schuler. 2025. Linear recency bias during training improves transformers’ fit to reading times. In Proceedings of the 31st International Conference on Computational Linguistics, pages 7735–7747, Abu Dhabi, UAE. Association for Computational Linguistics. https://credit.niso.org/ [↗](https://doi.org/10.48550/arxiv.2409.11250)
6. Jamell Dacon. 2022. [Towards a deep multi-layered dialectal language analysis: A case study of African-American English](https://md.research.elamur.ai/free/en/papers/dacon-2022-deep-multi-layered-dialectal-language/). Proceedings of the Second Workshop on Bridging Human–Computer Interaction and Natural Language Processing.
7. Ralph W. Fasold. 1969. Tense and the Form Be in Black English. Language, 45(4):763–776. Publisher: Linguistic Society of America. [↗](https://doi.org/10.2307/412334)
8. Aparna Garimella, Rada Mihalcea, and Akhash Amarnath. 2022. Demographic-aware language model fine-tuning as a bias mitigation technique. In Proceedings of the 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th International Joint Conference on Natural Language Processing, AACL/IJCNLP 2022 - Volume 2: Short Papers, Online only, November 20-23, 2022, pages 311–319. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Aparna+Garimella%2C+Rada+Mihalcea%2C+and+Akhash+Amarnath%2E+2022%2E+Demographic%2Daware+language+model+fine%2Dtuning+as+a+bias+mitigation+technique%2E+In+Proceedings+of+the+2nd+Conference+of+the+Asia%2DPacific+Chapte)
9. Lisa J. Green. 2002. African American English: A Linguistic Introduction. Cambridge University Press. [↗](https://doi.org/10.1017/cbo9780511800306)
10. Xiaobo Guo and Soroush Vosoughi. 2024. Serial position effects of large language models. Preprint, arXiv:2406.15981. [↗](https://arxiv.org/abs/2406.15981)
11. Camille Harris, Matan Halevy, Ayanna Howard, Amy Bruckman, and Diyi Yang. 2022. Exploring the role of grammar and word choice in bias toward African American English (AAE) in hate speech classification. In 2022 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’22, page 789–798, New York, NY, USA. Association for Computing Machinery. [find](https://scholar.google.com/scholar?q=Camille+Harris%2C+Matan+Halevy%2C+Ayanna+Howard%2C+Amy+Bruckman%2C+and+Diyi+Yang%2E+2022%2E+Exploring+the+role+of+grammar+and+word+choice+in+bias+toward+African+American+English+%28AAE%29+in+hate+speech+classificatio)
12. Alyssa Hwang, William R. Frey, and Kathleen McKeown. 2020. [Towards augmenting lexical resources for slang and African American English](https://md.research.elamur.ai/free/en/papers/hwang-2020-augmenting-lexical-resources-slang-african/). In Proceedings of the 7th Workshop on NLP for Similar Languages, Varieties and Dialects, pages 160–172, Barcelona, Spain (Online). International Committee on Computational Linguistics (ICCL).
13. Suramya Jadhav, Abhay Shanbhag, Amogh Thakurdesai, Ridhima Sinare, and Raviraj Joshi. 2024. On limitations of llm as annotator for low resource languages. Preprint, arXiv:2411.17637. [↗](https://arxiv.org/abs/2411.17637)
14. Anna Jørgensen, Dirk Hovy, and Anders Søgaard. 2016. [Learning a POS tagger for aave-like language](https://md.research.elamur.ai/free/en/papers/jorgensen-2016-learning-pos-tagger-aave-like/). In NAACL HLT 2016, The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego California, USA, June 12-17, 2016, pages 1115–1120. The Association for Computational Linguistics.
15. Amel Kallel. 2011. The Loss of Negative Concord in Standard English: A Case of Lexical Reanalysis. Cambridge Scholars Publishing, Newcastle upon Tyne, UK. [find](https://scholar.google.com/scholar?q=Amel+Kallel%2E+2011%2E+The+Loss+of+Negative+Concord+in+Standard+English%3A+A+Case+of+Lexical+Reanalysis%2E+Cambridge+Scholars+Publishing%2C+Newcastle+upon+Tyne%2C+UK%2E)
16. Tyler Kendall and Charlie Farrington. 2021. The Corpus of Regional African American Language. Publisher: Online Resources for African American Language Project. [↗](https://doi.org/10.7264/1ad5-6t35)
17. Svetlana Kiritchenko and Saif M. Mohammad. 2018. [Examining gender and race bias in two hundred sentiment analysis systems](https://md.research.elamur.ai/free/en/papers/kiritchenko-2018-examining-gender-race-bias-two/). In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics, *SEM@NAACL-HLT 2018, New Orleans, Louisiana, USA, June 5-6, 2018, pages 43–53. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1805.04508)
18. Allison Koenecke, Andrew Nam, Emily Lake, Joe Nudell, Minnie Quartey, Zion Mengesha, Connor Toups, John R Rickford, Dan Jurafsky, and Sharad Goel. 2020. Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences, 117(14):7684–7689. [↗](https://doi.org/10.1073/pnas.1915768117)
19. Bernd Kortmann. 2020. Syntactic Variation in English, chapter 16. John Wiley & Sons, Ltd. [find](https://scholar.google.com/scholar?q=Bernd+Kortmann%2E+2020%2E+Syntactic+Variation+in+English%2C+chapter+16%2E+John+Wiley+%26+Sons%2C+Ltd%2E)
20. Ernesto William De Luca and Oliver Streiter. 2003. Example-based NLP for minority languages: Tasks, resources and tools. In Proceedings of the Workshop on Natural Language Processing of Minority Languages with Few Computational Linguistic Resources (TALN 2003), pages 233–242. [find](https://scholar.google.com/scholar?q=Ernesto+William+De+Luca+and+Oliver+Streiter%2E+2003%2E+Example%2Dbased+NLP+for+minority+languages%3A+Tasks%2C+resources+and+tools%2E+In+Proceedings+of+the+Workshop+on+Natural+Language+Processing+of+Minority+Langu)
21. Antonis Maronikolakis, Philip Baader, and Hinrich Schütze. 2022. Analyzing hate speech data along racial, gender and intersectional axes. In Proceedings of the 4th Workshop on Gender Bias in Natural Language Processing (GeBNLP), pages 1–7, Seattle, Washington. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Antonis+Maronikolakis%2C+Philip+Baader%2C+and+Hinrich+Sch%C3%BCtze%2E+2022%2E+Analyzing+hate+speech+data+along+racial%2C+gender+and+intersectional+axes%2E+In+Proceedings+of+the+4th+Workshop+on+Gender+Bias+in+Natural+L)
22. Joshua L. Martin. 2022. Automatic Speech Recognition Systems, Spoken Corpora, and African American Language: An Examination of Linguistic Bias and Morphosyntactic Features. Ph.D. thesis, University of Florida. [find](https://scholar.google.com/scholar?q=Joshua+L%2E+Martin%2E+2022%2E+Automatic+Speech+Recognition+Systems%2C+Spoken+Corpora%2C+and+African+American+Language%3A+An+Examination+of+Linguistic+Bias+and+Morphosyntactic+Features%2E+Ph%2ED%2E+thesis%2C+University+of)
23. Joshua L. Martin and Kevin Tang. 2020. Understanding Racial Disparities in Automatic Speech Recognition: The Case of Habitual “be”. In Proc. Interspeech 2020, pages 626–630. [find](https://scholar.google.com/scholar?q=Joshua+L%2E+Martin+and+Kevin+Tang%2E+2020%2E+Understanding+Racial+Disparities+in+Automatic+Speech+Recognition%3A+The+Case+of+Habitual+%E2%80%9Cbe%E2%80%9D%2E+In+Proc%2E+Interspeech+2020%2C+pages+626%E2%80%93630%2E)
24. Ignacio M. Palacios Martínez. 2003. Multiple negation in modern english. a preliminary corpus-based study. Neuphilologische Mitteilungen, 104(4):477–498. [find](https://scholar.google.com/scholar?q=Ignacio+M%2E+Palacios+Mart%C3%ADnez%2E+2003%2E+Multiple+negation+in+modern+english%2E+a+preliminary+corpus%2Dbased+study%2E+Neuphilologische+Mitteilungen%2C+104%284%29%3A477%E2%80%93498%2E)
25. Tessa Masis, Chloe Eggleston, Lisa J Green, Taylor Jones, Meghan Armstrong, and Brendan O’Connor. 2023. Investigating morphosyntactic variation in African American English on Twitter. Proceedings of the Society for Computation in Linguistics, 6(1):392– 393. [↗](https://doi.org/10.7275/zdg0-0914)
26. Tessa Masis, Anissa Neal, Lisa Green, and Brendan O’Connor. 2022. [Corpus-guided contrast sets for morphosyntactic feature detection in low-resource English varieties](https://md.research.elamur.ai/free/en/papers/masis-2022-corpus-guided-contrast-sets-morphosyntactic/). In Proceedings of the first workshop on NLP applications to field linguistics, pages 11–25, Gyeongju, Republic of Korea. International Conference on Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2209.07611)
27. Sarah Moeller, Alexis Davis, Wilermine Previlon, Michael Bottini, and Kevin Tang. 2024. Compiling a corpus of African American language from oral histories. Research in Corpus Linguistics, 12(2):45– 79. [↗](https://doi.org/10.32714/ricl.12.02.04)
28. Sarah Moeller, Alexis Ciara Davis, Wilermine Previlon, and Kevin Tang. 2025. ALFAH – A Toolkit for Annotating Linguistic Features in African American Oral Histories. [https://osf.io/x9whn/](https://osf.io/x9whn/). Accessed: 2025-02-08. [↗](https://osf.io/x9whn/)
29. Maja Pavlovic and Massimo Poesio. 2024. [The effectiveness of LLMs as annotators: A comparative overview and empirical analysis of direct representation](https://md.research.elamur.ai/free/en/papers/pavlovic-2024-effectiveness-llms-annotators-comparative-overview/). In Proceedings of the 3rd Workshop on Perspectivist Approaches to NLP (NLPerspectives) @ LREC-COLING 2024, pages 100–110, Torino, Italia. ELRA and ICCL.
30. Fabian Pedregosa, Gaël Varoquaux, Alexandre Gramfort, Vincent Michel, Bertrand Thirion, Olivier Grisel, Mathieu Blondel, Peter Prettenhofer, Ron Weiss, Vincent Dubourg, et al. 2011. Scikit-learn: Machine learning in python. Journal of machine learning research, 12(Oct):2825–2830. [↗](https://doi.org/10.48550/arxiv.1201.0490)
31. Alexander Peysakhovich and Adam Lerer. 2023. Attention sorting combats recency bias in long context language models. ArXiv, abs/2310.01427. [↗](https://arxiv.org/abs/2310.01427)
32. Wilermine Previlon, Alice Rozet, Jotsna Gowda, Bill Dyer, Kevin Tang, and Sarah Moeller. 2024. [Leveraging syntactic dependencies in disambiguation: The case of African American English](https://md.research.elamur.ai/free/en/papers/previlon-2024-leveraging-syntactic-dependencies-disambiguation-case/). In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024), pages 10403–10415, Torino, Italia. ELRA and ICCL.
33. Harrison Santiago, Joshua Martin, Sarah Moeller, and Kevin Tang. 2022. Disambiguation of morphosyntactic features of African American English – the case of habitual be. In Proceedings of the Second Workshop on Language Technology for Equality, Diversity and Inclusion, pages 70–75, Dublin, Ireland. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Harrison+Santiago%2C+Joshua+Martin%2C+Sarah+Moeller%2C+and+Kevin+Tang%2E+2022%2E+Disambiguation+of+morphosyntactic+features+of+African+American+English+%E2%80%93+the+case+of+habitual+be%2E+In+Proceedings+of+the+Second+Wo)
34. Maarten Sap, Dallas Card, Saadia Gabriel, Yejin Choi, and Noah A. Smith. 2019. [The risk of racial bias in hate speech detection](https://md.research.elamur.ai/free/en/papers/sap-2019-risk-racial-bias-hate-speech/). In Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy, July 28- August 2, 2019, Volume 1: Long Papers, pages 1668–1678. Association for Computational Linguistics.
35. Jonathan Shaki, Sarit Kraus, and Michael Wooldridge. 2023. Cognitive Effects in Large Language Models. IOS Press. [↗](https://doi.org/10.3233/faia230505)
36. Andrew Silva, Pradyumna Tambwekar, and Matthew Gombolay. 2021. Towards a comprehensive understanding and accurate evaluation of societal biases in pre-trained transformers. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2383–2389, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Andrew+Silva%2C+Pradyumna+Tambwekar%2C+and+Matthew+Gombolay%2E+2021%2E+Towards+a+comprehensive+understanding+and+accurate+evaluation+of+societal+biases+in+pre%2Dtrained+transformers%2E+In+Proceedings+of+the+2021+)
37. Ian Stewart. 2014. [Now we stronger than ever: African-American English syntax in Twitter](https://md.research.elamur.ai/free/en/papers/stewart-2014-now-we-stronger-than-ever/). In Proceedings of the 14th Conference of the European Chapter of the Association for Computational Linguistics, EACL 2014, April 26-30, 2014, Gothenburg, Sweden, pages 31–37. The Association for Computer Linguistics.
38. Samson Tan, Shafiq Joty, Lav Varshney, and Min-Yen Kan. 2020. Mind your inflections! Improving NLP for non-standard Englishes with Base-Inflection Encoding. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 5647–5663, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Samson+Tan%2C+Shafiq+Joty%2C+Lav+Varshney%2C+and+Min%2DYen+Kan%2E+2020%2E+Mind+your+inflections%21+Improving+NLP+for+non%2Dstandard+Englishes+with+Base%2DInflection+Encoding%2E+In+Proceedings+of+the+2020+Conference+on+Em)
39. Caleb Ziems, William Held, Jingfeng Yang, Jwala Dhamala, Rahul Gupta, and Diyi Yang. 2023. [Multi-VALUE: A framework for cross-dialectal English NLP](https://md.research.elamur.ai/free/en/papers/ziems-2023-multi-value-framework-cross-dialectal/). In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 744–768, Toronto, Canada. Association for Computational Linguistics.

*Updated 1 November 2026*