---
title: "Perceptions of Language Technology Failures from South Asian"
description: "The authors study South Asian English (SAsE) speakers living in the US. They ran a Prolific survey… 83 references, cited by 5 papers here."
authors: ["Faye Holt", "William Held", "Diyi Yang"]
year: 2024
venue: "Findings of the Association for Computational Linguistics: ACL 2024"
source: "https://aclanthology.org/2024.findings-acl.241/"
doi: "10.18653/v1/2024.findings-acl.241"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/holt-2024-perceptions-language-technology-failures-south/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/holt-2024-perceptions-language-technology-failures-south/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/10/) / Perceptions of Language Technology Failures from South Asian English Speakers

# Perceptions of Language Technology Failures from South Asian English Speakers

**Authors:** Faye Holt, [William Held](https://md.research.elamur.ai/free/en/authors/william-held/) and [Diyi Yang](https://md.research.elamur.ai/free/en/authors/diyi-yang/) · **Year:** 2024 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2024](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2024](https://md.research.elamur.ai/free/en/venues/findings/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.findings-acl.241/) · [PDF](https://aclanthology.org/2024.findings-acl.241.pdf) · [DOI 10.18653/v1/2024.findings-acl.241](https://doi.org/10.18653/v1/2024.findings-acl.241) · [OpenAlex](https://openalex.org/W4402670462)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [5](https://md.research.elamur.ai/free/en/papers/holt-2024-perceptions-language-technology-failures-south/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
NLP systems are known to perform worse on English dialects other than Standard American English (SAmE). It was less clear whether this gap affects real users. The authors study South Asian English (SAsE) speakers living in the US. They ran a Prolific survey with SAsE speakers and a SAmE control group. Participants were validated through self-reported dialect and shibboleth words. The survey asked closed questions about how often technology fails and open questions about how speakers adapt their language.

SAsE speakers recalled failures more often than the control group, and they singled out written tools more than SAmE speakers did. They reported changing both their vocabulary and their grammar, but saw lexical problems as the main pain point. Guided by these answers, the authors built two benchmarks, each with a SAmE control version:
- a multiple-choice test of SAsE word definitions taken from Wiktionary;
- a minimal-pair test of Indian English syntax.

They evaluated 11 LLM families on both. Open models still show gaps in every category, while GPT-4 comes close to solving the lexical task. The study argues that even monolingual English systems need to account for dialect variation.

## Key points
- Participants: 110 likely SAsE and 150 likely SAmE speakers completed the survey. After screening, 78 SAsE and 97 SAmE remained, and 46 SAsE participants gave open-ended answers. Shibboleth pairs included eggplant/brinjal, lentils/daal and elevator/lift.
- SAsE speakers recall language-technology failures about 14% more often than SAmE speakers. Spoken technology causes problems for both groups, but written technology fails SAsE users disproportionately.
- Lexical benchmark: 317 loanwords and 724 standalone SAsE terms from Wiktionary.
- Syntactic benchmark: the 110 Indian English minimal pairs of Demszky et al. (2021), plus synthetic negative sentences built from syntax not attested in Indian or Pakistani English.
- Syntax results: every model is significantly worse on the SAsE syntax set, and the best is LLaMA 65B at 89%.
- Lexical results:
  - 14 of 15 open models scoring above 60% on the control are significantly worse on SAsE words.
  - Indian English term accuracy tracks control accuracy closely (ρ = 0.98).
  - GPT-4 and GPT-4-Turbo exceed 90%. Of GPT-4's 57 errors, 16 are terms used mostly in historical documents.

## Interesting ideas
- The control and dialect syntax tests share the same negative sentences, so any accuracy drop can only come from the attested dialect features.
- Screening relies on a few shibboleth words that cheaply separate two varieties, and these are checked against self-reports.
- Benchmark targets come from what users say hurts most (here, standalone dialect words), not from what researchers assume matters.
- A definition-matching multiple-choice test with a standard-English control serves as a probe of whether a system knows dialect vocabulary.
- The residual errors of the strongest model are sorted into categories: historical usage, slurs, rare transliterations, and agriculture and legal terms.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Evaluation could pair every dialect test set with a standard-English twin that uses identical negatives, so a parser's score drop is attributed cleanly to the dialect phenomenon.
- **I2** (tree): A small shibboleth lexicon could flag likely SAsE input before parsing and route it through UD dialect conversion or dialect-aware handling.
- **I3** (tokens): The en lexicon and lemmatizer could be prioritized with Wiktionary-sourced SAsE terms and loanwords, so words like brinjal or lift get correct lemmas and POS instead of being treated as unknown.
- **I4** (world): Dialect terms could be linked to WordNet/Wikidata concepts (daal to lentils), then tested by asking Mova to pick the right gloss, which checks the ontology mapping explicitly.
- **I5** (learn): Errors could be bucketed by such categories when inducing repair rules, which shows whether failures are lexicon gaps, transliteration issues or domain coverage problems.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Stanford University](https://md.research.elamur.ai/free/en/universities/stanford-university/) — University, Stanford, United States · 434 works · 314,419 citations
- [Georgia Institute of Technology](https://md.research.elamur.ai/free/en/universities/georgia-institute-of-technology/) — University, Atlanta, United States · 133 works · 24,497 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works

## Cite

Cite as: Faye Holt, William Held and Diyi Yang. 2024. Perceptions of Language Technology Failures from South Asian English Speakers. Findings of the Association for Computational Linguistics: ACL 2024.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{holt2024perceptions,
  title = {{Perceptions of Language Technology Failures from South Asian English Speakers}},
  author = {Faye Holt, William Held and Diyi Yang},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2024},
  year = {2024},
  doi = {10.18653/v1/2024.findings-acl.241},
  url = {https://doi.org/10.18653/v1/2024.findings-acl.241}
}
```

- [← previous MSciNLI: A Diverse Benchmark for Scientific Natural Language Inference](https://md.research.elamur.ai/free/en/papers/sadat-2024-mscinli-diverse-benchmark-scientific-natural/)
- [next → Role-Play Paradox in Large Language Models: Reasoning Performance Gains and Ethical Dilemmas](https://md.research.elamur.ai/free/en/papers/zhao-2024-role-play-paradox-large-language/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/10/)

## References

83 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Anmol Agarwal, Jigar Gupta, Rahul Goel, Shyam Upadhyay, Pankaj Joshi, and Rengarajan Aravamudhan. 2023. CST5: Data augmentation for code-switched semantic parsing. In Proceedings of the 1st Workshop on Taming Large Language Models: Controllability in the era of Interactive Assistants!, pages 1–10, Prague, Czech Republic. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2211.07514)
2. Ashley Amaya, Ruben Bach, Florian Keusch, and Frauke Kreuter. 2021. New data sources in social science research: Things to know before working with reddit data. Social science computer review, 39(5):943–960. [find](https://scholar.google.com/scholar?q=Ashley+Amaya%2C+Ruben+Bach%2C+Florian+Keusch%2C+and+Frauke+Kreuter%2E+2021%2E+New+data+sources+in+social+science+research%3A+Things+to+know+before+working+with+reddit+data%2E+Social+science+computer+review%2C+39%285%29%3A9)
3. GA Barnard. 1947. Significance tests for 2× 2 tables. Biometrika, 34(1/2):123–138. [↗](https://doi.org/10.1093/biomet/34.1-2.123)
4. Tobias Bernaisch and Christopher Koch. 2016. Attitudes towards englishes in india. World Englishes, 35(1):118–132. [↗](https://doi.org/10.1111/weng.12174)
5. Terra Blevins, Robert Kwiatkowski, Jamie MacBeth, Kathleen McKeown, Desmond Patton, and Owen Rambow. 2016. Automatically processing tweets from gang-involved youth: Towards detecting loss and aggression. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pages 2196–2206, Osaka, Japan. [find](https://scholar.google.com/scholar?q=Terra+Blevins%2C+Robert+Kwiatkowski%2C+Jamie+MacBeth%2C+Kathleen+McKeown%2C+Desmond+Patton%2C+and+Owen+Rambow%2E+2016%2E+Automatically+processing+tweets+from+gang%2Dinvolved+youth%3A+Towards+detecting+loss+and+aggressi)
6. The COLING 2016 Organizing Committee. [find](https://scholar.google.com/scholar?q=The+COLING+2016+Organizing+Committee%2E)
7. Su Lin Blodgett, Solon Barocas, Hal Daumé III, and Hanna Wallach. 2020. [Language (technology) is power: A critical survey of" bias" in nlp](https://md.research.elamur.ai/free/en/papers/blodgett-2020-language-technology-is-power-critical/). arXiv preprint arXiv:2005.14050. [↗](https://arxiv.org/abs/2005.14050)
8. Su Lin Blodgett, Lisa Green, and Brendan O’Connor. 2016. [Demographic dialectal variation in social media: A case study of African-American English](https://md.research.elamur.ai/free/en/papers/blodgett-2016-demographic-dialectal-variation-social-media/). In Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, pages 1119–1130, Austin, Texas. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/d16-1120)
9. Su Lin Blodgett, Johnny Wei, and Brendan O’Connor. 2018. [Twitter Universal Dependency parsing for African-American and mainstream American English](https://md.research.elamur.ai/free/en/papers/blodgett-2018-twitter-universal-dependency-parsing-african/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1415–1425, Melbourne, Australia. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/p18-1131)
10. Sarah Buschfeld and Alexander Kautzsch. 2017. Towards an integrated approach to postcolonial and non-postcolonial englishes. World Englishes, 36(1):104– 126. [↗](https://doi.org/10.1111/weng.12203)
11. Dorottya Demszky, Devyani Sharma, Jonathan Clark, Vinodkumar Prabhakaran, and Jacob Eisenstein. 2021. Learning to recognize dialect features. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2315–2338, Online. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2021.naacl-main.184)
12. A. Seza Doğruöz, Sunayana Sitaram, Barbara E. Bullock, and Almeida Jacqueline Toribio. 2021. [A survey of code-switching: Linguistic and social perspectives for language technologies](https://md.research.elamur.ai/free/en/papers/dogruoz-2021-survey-code-switching-linguistic-social/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1654–1666, Online. Association for Computational Linguistics.
13. Benjamin D Douglas, Patrick J Ewell, and Markus Brauer. 2023. Data quality in online human-subjects research: Comparisons between mturk, prolific, cloudresearch, qualtrics, and sona. Plos one, 18(3):e0279720. [↗](https://doi.org/10.1371/journal.pone.0279720)
14. Glendon Frank Drake. 1980. The social role of slang. In Language, pages 63–70. Elsevier. [↗](https://doi.org/10.1016/b978-0-08-024696-3.50015-6)
15. Jacob Eisenstein, Vinodkumar Prabhakaran, Clara Rivera, Dorottya Demszky, and Devyani Sharma. 2023. MD3: The Multi-Dialect Dataset of Dialogues. In Proc. INTERSPEECH 2023, pages 4059–4063. [find](https://scholar.google.com/scholar?q=Jacob+Eisenstein%2C+Vinodkumar+Prabhakaran%2C+Clara+Rivera%2C+Dorottya+Demszky%2C+and+Devyani+Sharma%2E+2023%2E+MD3%3A+The+Multi%2DDialect+Dataset+of+Dialogues%2E+In+Proc%2E+INTERSPEECH+2023%2C+pages+4059%E2%80%934063%2E)
16. Peer Eyal, Rothschild David, Gordon Andrew, Evernden Zak, and Damer Ekaterina. 2021. Data quality of platforms and panels for online behavioral research. Behavior Research Methods, pages 1–20. [↗](https://doi.org/10.3758/s13428-021-01694-3)
17. Fahim Faisal, Sharlina Keshava, Md Mahfuz Ibn Alam, and Antonios Anastasopoulos. 2021. Sd-qa: Spoken dialectal question answering for the real world. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 3296–3315. [find](https://scholar.google.com/scholar?q=Fahim+Faisal%2C+Sharlina+Keshava%2C+Md+Mahfuz+Ibn+Alam%2C+and+Antonios+Anastasopoulos%2E+2021%2E+Sd%2Dqa%3A+Spoken+dialectal+question+answering+for+the+real+world%2E+In+Findings+of+the+Association+for+Computational+L)
18. Ravinder Gargesh. 2019. South asian englishes. The handbook of world Englishes, pages 105–134. [↗](https://doi.org/10.1002/9781119147282.ch7)
19. Lyle Campbell and Verónica Grondona. 2008. Ethnologue: Languages of the world. Language, 84(3):636–641. [find](https://scholar.google.com/scholar?q=Lyle+Campbell+and+Ver%C3%B3nica+Grondona%2E+2008%2E+Ethnologue%3A+Languages+of+the+world%2E+Language%2C+84%283%29%3A636%E2%80%93641%2E)
20. Sandra Goetz. 2017. Non-canonical syntax in south asian varieties of english: A corpus-based pilot study on fronting. Zeitschrift für Anglistik und Amerikanistik, 65(3):265–281. [find](https://scholar.google.com/scholar?q=Sandra+Goetz%2E+2017%2E+Non%2Dcanonical+syntax+in+south+asian+varieties+of+english%3A+A+corpus%2Dbased+pilot+study+on+fronting%2E+Zeitschrift+f%C3%BCr+Anglistik+und+Amerikanistik%2C+65%283%29%3A265%E2%80%93281%2E)
21. Francesco Cavallaro and Ng Bee Chin. 2009. Between status and solidarity in singapore. World Englishes, 28(2):143–159. [↗](https://doi.org/10.1111/j.1467-971x.2009.01580.x)
22. Anthea Fraser Gupta. 2008. Singapore english. The Handbook of World Englishes, 7:351–368. [find](https://scholar.google.com/scholar?q=Anthea+Fraser+Gupta%2E+2008%2E+Singapore+english%2E+The+Handbook+of+World+Englishes%2C+7%3A351%E2%80%93368%2E)
23. Eble Chapel. 1998. Slang and sociability: In-group language among college students. by connie. Journal of English Linguistics, 26(3):247–265. [find](https://scholar.google.com/scholar?q=Eble+Chapel%2E+1998%2E+Slang+and+sociability%3A+In%2Dgroup+language+among+college+students%2E+by+connie%2E+Journal+of+English+Linguistics%2C+26%283%29%3A247%E2%80%93265%2E)
24. Anthea Fraser Gupta. 2010. Indian english. The Handbook of World Englishes, 7:203–222. [find](https://scholar.google.com/scholar?q=Anthea+Fraser+Gupta%2E+2010%2E+Indian+english%2E+The+Handbook+of+World+Englishes%2C+7%3A203%E2%80%93222%2E)
25. CIA. 2021. Median age - the world factbook. [find](https://scholar.google.com/scholar?q=CIA%2E+2021%2E+Median+age+%2D+the+world+factbook%2E)
26. William Held, Camille Harris, Michael Best, and Diyi Yang. 2023a. A material lens on coloniality in nlp. arXiv preprint arXiv:2311.08391. [↗](https://arxiv.org/abs/2311.08391)
27. Thomas Davidson, Debasmita Bhattacharya, and Ingmar Weber. 2019. [Racial bias in hate speech and abusive language detection datasets](https://md.research.elamur.ai/free/en/papers/davidson-2019-racial-bias-hate-speech-abusive/). arXiv preprint arXiv:1905.12516. [↗](https://arxiv.org/abs/1905.12516)
28. William Held, Christopher Hidey, Fei Liu, Eric Zhu, Rahul Goel, Diyi Yang, and Rushin Shah. 2023b. DAMP: Doubly aligned multilingual parser for task-oriented dialogue. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=William+Held%2C+Christopher+Hidey%2C+Fei+Liu%2C+Eric+Zhu%2C+Rahul+Goel%2C+Diyi+Yang%2C+and+Rushin+Shah%2E+2023b%2E+DAMP%3A+Doubly+aligned+multilingual+parser+for+task%2Doriented+dialogue%2E+In+Proceedings+of+the+61st+Annua)
29. Annika Hohenthal. 2003. English in india: Loyalty and attitudes. Language in India, 3(5):1–107. [find](https://scholar.google.com/scholar?q=Annika+Hohenthal%2E+2003%2E+English+in+india%3A+Loyalty+and+attitudes%2E+Language+in+India%2C+3%285%29%3A1%E2%80%93107%2E)
30. Dirk Hovy and Shannon L Spruit. 2016. [The social impact of natural language processing](https://md.research.elamur.ai/free/en/papers/hovy-2016-social-impact-natural-language-processing/). In Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 591–598. [↗](https://doi.org/10.5281/zenodo.1000924)
31. Ann Irvine, Jonathan Weese, and Chris Callison-Burch. 2012. Processing informal, romanized pakistani text messages. In Proceedings of the Second Workshop on Language in Social Media, pages 75–78. [find](https://scholar.google.com/scholar?q=Ann+Irvine%2C+Jonathan+Weese%2C+and+Chris+Callison%2DBurch%2E+2012%2E+Processing+informal%2C+romanized+pakistani+text+messages%2E+In+Proceedings+of+the+Second+Workshop+on+Language+in+Social+Media%2C+pages+75%E2%80%9378%2E)
32. Tahir Javed, Sakshi Joshi, Vignesh Nagarajan, Sai Sundaresan, Janki Nawale, Abhigyan Raman, Kaushal Bhogale, Pratyush Kumar, and Mitesh M. Khapra. 2023. [Svarah: Evaluating english asr systems on indian accents](https://md.research.elamur.ai/free/en/papers/javed-2023-svarah-evaluating-english-asr-systems/).
33. Anna Jørgensen, Dirk Hovy, and Anders Søgaard. 2016a. [Learning a POS tagger for AAVE-like language](https://md.research.elamur.ai/free/en/papers/jorgensen-2016-learning-pos-tagger-aave-like/). In Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 1115–1120, San Diego, California. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n16-1130)
34. Anna Jørgensen, Dirk Hovy, Anders Søgaard, et al. 2016b. [Learning a pos tagger for aave-like language](https://md.research.elamur.ai/free/en/papers/jorgensen-2016-learning-pos-tagger-aave-like/). In The 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Proceedings of the conference. Association for Computational Linguistics.
35. David Jurgens, Yulia Tsvetkov, and Dan Jurafsky. 2017. Incorporating dialectal variability for socially equitable language identification. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers). [find](https://scholar.google.com/scholar?q=David+Jurgens%2C+Yulia+Tsvetkov%2C+and+Dan+Jurafsky%2E+2017%2E+Incorporating+dialectal+variability+for+socially+equitable+language+identification%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+)
36. B Kachru. 1994. English in south asia. The Cambridge. [↗](https://doi.org/10.1017/chol9780521264785.011)
37. Braj B Kachru. 1965. The indianness in indian english. Word, 21(3):391–410. [↗](https://doi.org/10.1080/00437956.1965.11435436)
38. Lucie-Aimée Kaffee, Arnav Arora, Zeerak Talat, and Isabelle Augenstein. 2023. Thorny roses: Investigating the dual use dilemma in natural language processing. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 13977–13998, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Lucie%2DAim%C3%A9e+Kaffee%2C+Arnav+Arora%2C+Zeerak+Talat%2C+and+Isabelle+Augenstein%2E+2023%2E+Thorny+roses%3A+Investigating+the+dual+use+dilemma+in+natural+language+processing%2E+In+Findings+of+the+Association+for+Comput)
39. Simran Khanuja, Sandipan Dandapat, Anirudh Srinivasan, Sunayana Sitaram, and Monojit Choudhury. 2020. [GLUECoS: An evaluation benchmark for code-switched NLP](https://md.research.elamur.ai/free/en/papers/khanuja-2020-gluecos-evaluation-benchmark-code-switched/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 3575–3585, Online. Association for Computational Linguistics.
40. Svetlana Kiritchenko and Saif M Mohammad. 2018. [Examining gender and race bias in two hundred sentiment analysis systems](https://md.research.elamur.ai/free/en/papers/kiritchenko-2018-examining-gender-race-bias-two/). arXiv preprint arXiv:1805.04508. [↗](https://arxiv.org/abs/1805.04508)
41. Bernd Kortmann, Kerstin Lunkenheimer, and Katharina Ehret, editors. 2020. eWAVE. [find](https://scholar.google.com/scholar?q=Bernd+Kortmann%2C+Kerstin+Lunkenheimer%2C+and+Katharina+Ehret%2C+editors%2E+2020%2E+eWAVE%2E)
42. Jon A Krosnick. 2018. Questionnaire design. The Palgrave handbook of survey research, pages 439–455. [find](https://scholar.google.com/scholar?q=Jon+A+Krosnick%2E+2018%2E+Questionnaire+design%2E+The+Palgrave+handbook+of+survey+research%2C+pages+439%E2%80%93455%2E)
43. Claudia Lange. 2012. The syntax of spoken indian english. The Syntax of Spoken Indian English, pages 1–281. [↗](https://doi.org/10.1075/veaw.g45)
44. Junjie Li, Jieyu Wu, and Richard Socher. 2021. Understanding code-switching in language models. arXiv preprint arXiv:2109.04278. [↗](https://arxiv.org/abs/2109.04278)
45. Li Lucy, Suchin Gururangan, Luca Soldaini, Emma Strubell, David Bamman, Lauren Klein, and Jesse Dodge. 2024. [Aboutme: Using self-descriptions in webpages to document the effects of english pretraining data filters](https://md.research.elamur.ai/free/en/papers/lucy-2024-aboutme-self-descriptions-webpages-document/). arXiv preprint arXiv:2401.06408. [↗](https://arxiv.org/abs/2401.06408)
46. Yash Madhani, Sushane Parthan, Priyanka Bedekar, Gokul Nc, Ruchi Khapra, Anoop Kunchukuttan, Pratyush Kumar, and Mitesh Khapra. 2023. Aksharantar: Open Indic-language transliteration datasets and models for the next billion users. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 40–57, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Yash+Madhani%2C+Sushane+Parthan%2C+Priyanka+Bedekar%2C+Gokul+Nc%2C+Ruchi+Khapra%2C+Anoop+Kunchukuttan%2C+Pratyush+Kumar%2C+and+Mitesh+Khapra%2E+2023%2E+Aksharantar%3A+Open+Indic%2Dlanguage+transliteration+datasets+and+mode)
47. Braj B Kachru. 1976. Models of english for the third world: White man’s linguistic burden or language pragmatics? Tesol Quarterly, pages 221–239. [↗](https://doi.org/10.2307/3585643)
48. Ahmar Mahboob, Nadra Huma Ahmar, and Edgar W Schneider. 2008. Pakistani english: Phonology. Varieties of English, 4:244–258. [↗](https://doi.org/10.1515/9783110208429.1.244)
49. Braj B Kachru. 1986. The indianization of english. English Today, 2(2):31–33. [↗](https://doi.org/10.1017/s026607840000198x)
50. Colin P Masica. 2005. Defining a linguistic area: South Asia. Orient Blackswan. [find](https://scholar.google.com/scholar?q=Colin+P+Masica%2E+2005%2E+Defining+a+linguistic+area%3A+South+Asia%2E+Orient+Blackswan%2E)
51. Braj B Kachru. 1992. World englishes: Approaches, issues and resources. Language teaching, 25(1):1– 14. [↗](https://doi.org/10.1017/s0261444800006583)
52. Tessa Masis, Anissa Neal, Lisa Green, and Brendan O’Connor. 2022. [Corpus-guided contrast sets for morphosyntactic feature detection in low-resource English varieties](https://md.research.elamur.ai/free/en/papers/masis-2022-corpus-guided-contrast-sets-morphosyntactic/). In Proceedings of the first workshop on NLP applications to field linguistics, pages 11–25, Gyeongju, Republic of Korea. International Conference on Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2209.07611)
53. Yamuna Kachru and Cecil L Nelson. 2006. World Englishes in Asian Contexts, volume 1. Hong Kong University Press. [find](https://scholar.google.com/scholar?q=Yamuna+Kachru+and+Cecil+L+Nelson%2E+2006%2E+World+Englishes+in+Asian+Contexts%2C+volume+1%2E+Hong+Kong+University+Press%2E)
54. Zion Mengesha, Courtney Heldreth, Michal Lahav, Juliana Sublewski, and Elyse Tuennerman. 2021. “i don’t think these devices are very culturally sensitive.”—impact of automated speech recognition errors on african americans. Frontiers in Artificial Intelligence, 4:169. [↗](https://doi.org/10.3389/frai.2021.725911)
55. Koustav Rudra, Ashish Sharma, Kalika Bali, Monojit Choudhury, and Niloy Ganguly. 2019. Identifying and analyzing different aspects of english-hindi code-switching in twitter. ACM Transactions on Asian and Low-Resource Language Information Processing (TALLIP), 18(3):1–28. [↗](https://doi.org/10.1145/3314935)
56. Rajend Mesthrie and Rakesh M Bhatt. 2008. World Englishes: The study of new linguistic varieties. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Rajend+Mesthrie+and+Rakesh+M+Bhatt%2E+2008%2E+World+Englishes%3A+The+study+of+new+linguistic+varieties%2E+Cambridge+University+Press%2E)
57. Anju Sahgal. 1991. Patterns of language use in a bilingual setting in india. English around the world: Sociolinguistic perspectives, pages 299–307. [find](https://scholar.google.com/scholar?q=Anju+Sahgal%2E+1991%2E+Patterns+of+language+use+in+a+bilingual+setting+in+india%2E+English+around+the+world%3A+Sociolinguistic+perspectives%2C+pages+299%E2%80%93307%2E)
58. Christian M Meyer and Iryna Gurevych. 2012. [Wiktionary: A new rival for expert-built lexicons? Exploring the possibilities of collaborative lexicography](https://md.research.elamur.ai/free/en/papers/meyer-2012-wiktionary-new-rival/).
59. Rupak Sarkar, Sayantan Mahinder, and Ashiqur KhudaBukhsh. 2020. The non-native speaker aspect: Indian English in social media. In Proceedings of the Sixth Workshop on Noisy User-generated Text (W-NUT 2020), pages 61–70, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Rupak+Sarkar%2C+Sayantan+Mahinder%2C+and+Ashiqur+KhudaBukhsh%2E+2020%2E+The+non%2Dnative+speaker+aspect%3A+Indian+English+in+social+media%2E+In+Proceedings+of+the+Sixth+Workshop+on+Noisy+User%2Dgenerated+Text+%28W%2DNUT+)
60. John Nerbonne and Wilbert Heeringa. 2010. Measuring dialect differences. Language and Space: Theories and Methods. Berlin: Mouton De Gruyter, pages 550–566. [find](https://scholar.google.com/scholar?q=John+Nerbonne+and+Wilbert+Heeringa%2E+2010%2E+Measuring+dialect+differences%2E+Language+and+Space%3A+Theories+and+Methods%2E+Berlin%3A+Mouton+De+Gruyter%2C+pages+550%E2%80%93566%2E)
61. Temitayo Olatoye. 2022. Attitudes of educated nigerians towards varieties of english. Language Matters: Studies in the Languages of Southern Africa, 53(1):81–102. [↗](https://doi.org/10.1080/10228195.2022.2067216)
62. Isabel Papadimitriou, Richard Futrell, and Kyle Mahowald. 2022. When classifying grammatical role, BERT doesn’t care about word order... except when it matters. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 636–643, Dublin, Ireland. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Isabel+Papadimitriou%2C+Richard+Futrell%2C+and+Kyle+Mahowald%2E+2022%2E+When+classifying+grammatical+role%2C+BERT+doesn%E2%80%99t+care+about+word+order%2E%2E%2E+except+when+it+matters%2E+In+Proceedings+of+the+60th+Annual+Meeti)
63. Thang Pham, Trung Bui, Long Mai, and Anh Nguyen. 2021. [Out of order: How important is the sequential order of words in a sentence in natural language understanding tasks?](https://md.research.elamur.ai/free/en/papers/pham-2021-out-order-how-important-is/) In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1145–1160, Online. Association for Computational Linguistics.
64. Edgar W Schneider. 2003. The dynamics of new englishes: From identity construction to dialect birth. Language, 79(2):233–281. [↗](https://doi.org/10.1353/lan.2003.0136)
65. Edgar W Schneider. 2007. Postcolonial English: Varieties around the world. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Edgar+W+Schneider%2E+2007%2E+Postcolonial+English%3A+Varieties+around+the+world%2E+Cambridge+University+Press%2E)
66. Alexander Shan, John Bauer, Riley Carlson, and Christopher Manning. 2023. [Do “English” named entity recognizers work well on global englishes?](https://md.research.elamur.ai/free/en/papers/shan-2023-do-english-named-entity-recognizers/) In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 11778–11791, Singapore. Association for Computational Linguistics.
67. Devyani Sharma. 2005a. Dialect stabilization and speaker awareness in non-native varieties of english 1. Journal of Sociolinguistics, 9(2):194–224. [find](https://scholar.google.com/scholar?q=Devyani+Sharma%2E+2005a%2E+Dialect+stabilization+and+speaker+awareness+in+non%2Dnative+varieties+of+english+1%2E+Journal+of+Sociolinguistics%2C+9%282%29%3A194%E2%80%93224%2E)
68. Devyani Sharma. 2005b. Language transfer and discourse universals in indian english article use. Studies in Second Language Acquisition, 27(4):535–566. [↗](https://doi.org/10.1017/s0272263105050242)
69. Devyani Sharma. 2012. Indian english. The Mouton world atlas of variation in English, pages 523–30. [find](https://scholar.google.com/scholar?q=Devyani+Sharma%2E+2012%2E+Indian+english%2E+The+Mouton+world+atlas+of+variation+in+English%2C+pages+523%E2%80%9330%2E)
70. John Platt. 1989. The nature of indigenized englishes: Interference — creativity — universals. Language Sciences, 11(4):395–407. [↗](https://doi.org/10.1016/0388-0001(89)90028-4)
71. Devyani Sharma. 2023. From Deficit to Dialect: The Evolution of English in India and Singapore. Oxford University Press. [find](https://scholar.google.com/scholar?q=Devyani+Sharma%2E+2023%2E+From+Deficit+to+Dialect%3A+The+Evolution+of+English+in+India+and+Singapore%2E+Oxford+University+Press%2E)
72. John Talbot Platt, Heidi Weber, and Mian Lian Ho. 1984. The new englishes. [find](https://scholar.google.com/scholar?q=John+Talbot+Platt%2C+Heidi+Weber%2C+and+Mian+Lian+Ho%2E+1984%2E+The+new+englishes%2E)
73. Devyani Sharma, Alexander Bergs, and Laurel J Brinton. 2017. English in india. The history of English, 5:311– 29. [find](https://scholar.google.com/scholar?q=Devyani+Sharma%2C+Alexander+Bergs%2C+and+Laurel+J+Brinton%2E+2017%2E+English+in+india%2E+The+history+of+English%2C+5%3A311%E2%80%93+29%2E)
74. Jelena Prokić, Çağrı Çöltekin, and John Nerbonne. 2012. Detecting shibboleths. In Proceedings of the EACL 2012 Joint Workshop of LINGVIS & UNCLH, pages 72–80. [find](https://scholar.google.com/scholar?q=Jelena+Prokic%CC%81%2C+%C3%87ag%CC%86r%C4%B1+%C3%87%C3%B6ltekin%2C+and+John+Nerbonne%2E+2012%2E+Detecting+shibboleths%2E+In+Proceedings+of+the+EACL+2012+Joint+Workshop+of+LINGVIS+%26+UNCLH%2C+pages+72%E2%80%9380%2E)
75. Itamar Shatz. 2017. Fast, free, and targeted: Reddit as a source for recruiting participants online. Social Science Computer Review, 35(4):537–549. [find](https://scholar.google.com/scholar?q=Itamar+Shatz%2E+2017%2E+Fast%2C+free%2C+and+targeted%3A+Reddit+as+a+source+for+recruiting+participants+online%2E+Social+Science+Computer+Review%2C+35%284%29%3A537%E2%80%93549%2E)
76. John R Rickford. 1996. Regional and social variation. Sociolinguistics and language teaching, pages 151– 194. [↗](https://doi.org/10.1017/cbo9780511551185.009)
77. Brian Roark, Lawrence Wolf-Sonkin, Christo Kirov, Sabrina J. Mielke, Cibu Johny, Isin Demirsahin, and Keith Hall. 2020. Processing South Asian languages written in the Latin script: the Dakshina dataset. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 2413–2423, Marseille, France. European Language Resources Association. [find](https://scholar.google.com/scholar?q=Brian+Roark%2C+Lawrence+Wolf%2DSonkin%2C+Christo+Kirov%2C+Sabrina+J%2E+Mielke%2C+Cibu+Johny%2C+Isin+Demirsahin%2C+and+Keith+Hall%2E+2020%2E+Processing+South+Asian+languages+written+in+the+Latin+script%3A+the+Dakshina+datas)
78. Jiao Sun, Thibault Sellam, Elizabeth Clark, Tu Vu, Timothy Dozat, Dan Garrette, Aditya Siddhant, Jacob Eisenstein, and Sebastian Gehrmann. 2023. Dialect-robust evaluation of generated text. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 6010–6028, Toronto, Canada. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.2211.00922)
79. Peter KW Tan and Daniel KH Tan. 2008. Attitudes towards non-standard english in singapore 1. World Englishes, 27(3-4):465–479. [find](https://scholar.google.com/scholar?q=Peter+KW+Tan+and+Daniel+KH+Tan%2E+2008%2E+Attitudes+towards+non%2Dstandard+english+in+singapore+1%2E+World+Englishes%2C+27%283%2D4%29%3A465%E2%80%93479%2E)
80. Alex Warstadt, Alicia Parrish, Haokun Liu, Anhad Mohananey, Wei Peng, Sheng-Fu Wang, and Samuel R Bowman. 2020. [Blimp: The benchmark of linguistic minimal pairs for english](https://md.research.elamur.ai/free/en/papers/warstadt-2020-blimp-benchmark-linguistic-minimal-pairs/). Transactions of the Association for Computational Linguistics, 8:377–392. [↗](https://doi.org/10.1162/tacl_a_00321)
81. Tatu Ylonen. 2022. Wiktextract: Wiktionary as machine-readable structured data. In Proceedings of the 13th Conference on Language Resources and Evaluation (LREC). European Language Resources Association (ELRA). [find](https://scholar.google.com/scholar?q=Tatu+Ylonen%2E+2022%2E+Wiktextract%3A+Wiktionary+as+machine%2Dreadable+structured+data%2E+In+Proceedings+of+the+13th+Conference+on+Language+Resources+and+Evaluation+%28LREC%29%2E+European+Language+Resources+Associati)
82. Meng Zhang, Yang Liu, Huanbo Luan, and Maosong Sun. 2017. Earth mover’s distance minimization for unsupervised bilingual lexicon induction. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1934–1945, Copenhagen, Denmark. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Meng+Zhang%2C+Yang+Liu%2C+Huanbo+Luan%2C+and+Maosong+Sun%2E+2017%2E+Earth+mover%E2%80%99s+distance+minimization+for+unsupervised+bilingual+lexicon+induction%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+i)
83. Yanchun Zhou and Yanhong Fan. 2013. A sociolinguistic study of american slang. Theory & Practice in Language Studies, 3(12). A A pilot study was performed on Reddit, to allow for targeted outreach potential (Shatz, 2017). We select participants from subreddits r/SampleSize and r/India based on prior usage in Reddit surveying (Amaya et al., 2021) and relevant population. Based on feedback from the pilot, we revised the initial survey with a focus on gathering specific instances of adaptive behavior from SAsE speakers and more opt-in open-ended prompts for respondents to provide perspectives. Furthermore, due to the difficulty of confirming background information from Reddit we moved to Prolific for the final survey. [↗](https://doi.org/10.4304/tpls.3.12.2209-2213)

*Updated 1 November 2026*