---
title: "Artificial Intelligence Bias on English Language Learners in"
description: "The authors used Grade 8 constructed-response items from the California Science Test (CAST), each… 29 references, cited by 1 paper here."
authors: ["Shuchen Guo", "Yun Wang", "Jichao Yu", "Xuansheng Wu", "Bilgehan Ayik", "Field M. Watts", "Ehsan Latif", "Ninghao Liu", "Lei Liu", "Xiaoming Zhai"]
year: 2025
venue: "arXiv:2505.10643"
source: "https://arxiv.org/abs/2505.10643"
doi: "10.48550/arxiv.2505.10643"
arxiv: "2505.10643"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/guo-2025-artificial-intelligence-bias-english-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/guo-2025-artificial-intelligence-bias-english-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/) / Artificial Intelligence Bias on English Language Learners in Automatic Scoring

# Artificial Intelligence Bias on English Language Learners in Automatic Scoring

**Authors:** Shuchen Guo, [Yun Wang](https://md.research.elamur.ai/free/en/authors/yun-wang/), Jichao Yu, [Xuansheng Wu](https://md.research.elamur.ai/free/en/authors/xuansheng-wu/), Bilgehan Ayik, Field M. Watts, Ehsan Latif, [Ninghao Liu](https://md.research.elamur.ai/free/en/authors/ninghao-liu/), [Lei Liu](https://md.research.elamur.ai/free/en/authors/lei-liu/) and [Xiaoming Zhai](https://md.research.elamur.ai/free/en/authors/xiaoming-zhai/) · **Year:** 2025 · **Venue:** [arXiv:2505.10643](https://md.research.elamur.ai/free/en/venues/lecture-notes-in-computer-science/) ([Lecture notes in computer science 2025](https://md.research.elamur.ai/free/en/venues/lecture-notes-in-computer-science/2025/)) · **Publisher:** [Springer Science+Business Media](https://md.research.elamur.ai/free/en/publishers/springer-science-business-media/) · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2505.10643) · [PDF](https://arxiv.org/pdf/2505.10643) · [DOI 10.48550/arxiv.2505.10643](https://doi.org/10.48550/arxiv.2505.10643) · [arXiv 2505.10643](https://arxiv.org/abs/2505.10643) · [OpenAlex](https://openalex.org/W4412496583)

**Citations:** 14 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/guo-2025-artificial-intelligence-bias-english-language/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks whether automatic scoring of short written science answers treats English Language Learners (ELLs) unfairly, and in particular whether ELLs being underrepresented in training data is the cause. The authors used Grade 8 constructed-response items from the California Science Test (CAST), each scored 0–2. For every item they fine-tuned BERT four times: on ELL answers only, on non-ELL answers only, on a mix with the natural (unbalanced) ratio, and on a balanced mix. They downsampled so that the four variants could be compared fairly. Bias was measured as a difference in scoring accuracy, tested with Friedman tests. Disparity was measured by whether the model's mean score gap between ELLs and non-ELLs matched the gap in human scores. With plenty of ELL data (about 30,000 or about 1,000 per item) they found no bias and no distorted gaps. With only about 200 ELL answers, problems may appear. Their advice is to train on sufficiently large data that keeps real-world group proportions.

## Key points
- Data: 21 CAST Grade 8 items. 10 items have about 30,000 ELL responses, 5 have about 1,000, and 6 have about 200. ELLs are roughly 15% of all students.
- Grouping: EL and ADEL were merged into "ELL". EO, RFEP and IFEP were merged into "non-ELL". The TBD and EPU categories were dropped as ambiguous.
- There are four BERT training variants per item (ELL, Non-ELL, Unbalanced Mixed, Balanced Mixed), downsampled to the ELL sample size.
- Two separate measures are used:
  - Accuracy (Acc) compared with Friedman tests, for bias.
  - The difference between human and AI Mean Score Gaps (MSGs), for disparity.
- Training with the natural ELL/non-ELL ratio kept the gaps undistorted. Adding more ELL data narrowed the gap, but the authors could not tell whether this was fairer scoring or just lower non-ELL scores.

## Interesting ideas
- The paper keeps two fairness questions apart: does the model score one group less accurately, and does it keep the gap between groups that human raters see. These two can fail independently, so checking both catches more problems.
- The ablation builds training sets from different group compositions (only A, only B, natural mix, balanced mix) but holds total size fixed. This isolates the effect of composition from the effect of quantity.
- The authors note that a shrinking gap might come from scores dropping for the majority group rather than real improvement. So a better-looking metric always needs a check of absolute per-group levels.
- Results are stratified by data scale (about 30,000 / 1,000 / 200 per group). Bias only showed up in the smallest tier, so averages over all items would have hidden it.
- The paper argues that short science answers are judged on content, not language polish, which reduces the effect of English proficiency on scoring.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (answer): When grading school-course test answers, report per-group accuracy and also whether Mova's score gap between, e.g., learner-style and fluent answers matches human graders.
- **I2** (learn): Apply the same design to silver data from different sources or UD dialects, so a source's real effect is not confused with having more data.
- **I3** (learn): Add per-group absolute scores as a negative control in evaluation, so a "fairer" result caused by lowering everyone is flagged.
- **I4** (tree): Report parser LAS separately for small-data genres or converted treebanks, where weaknesses tend to hide behind large-corpus averages.
- **I5** (answer): Score student short answers by comparing meaning graphs rather than surface wording, so nonstandard but correct phrasing is not penalized.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Educational Testing Service](https://md.research.elamur.ai/free/en/ngos/educational-testing-service/) — Nonprofit, Princeton, United States · 27 works · 2,212 citations
- [University of Georgia](https://md.research.elamur.ai/free/en/universities/university-of-georgia/) — University, Athens, United States · 17 works · 26,331 citations
- [George Mason University](https://md.research.elamur.ai/free/en/universities/george-mason-university/) — University, Fairfax, United States · 24 works · 1,288 citations
- [Nanjing Normal University](https://md.research.elamur.ai/free/en/universities/nanjing-normal-university/) — University, Nanjing, China · 3 works · 195 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,730 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 626 works

🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,379 works · [🖍️ Grammatical error correction & learner language](https://md.research.elamur.ai/free/en/topics/grammatical-error-correction/) · 162 works

💡 Concepts (1): [💡 Artificial intelligence](https://md.research.elamur.ai/free/en/concepts/artificial-intelligence/) · 49 works

## Cite

Cite as: Shuchen Guo, Yun Wang, Jichao Yu, Xuansheng Wu, Bilgehan Ayik, Field M. Watts, Ehsan Latif, Ninghao Liu, Lei Liu and Xiaoming Zhai. 2025. Artificial Intelligence Bias on English Language Learners in Automatic Scoring. arXiv:2505.10643.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{guo2025artificial,
  title = {{Artificial Intelligence Bias on English Language Learners in Automatic Scoring}},
  author = {Shuchen Guo, Yun Wang, Jichao Yu, Xuansheng Wu, Bilgehan Ayik, Field M. Watts, Ehsan Latif, Ninghao Liu, Lei Liu and Xiaoming Zhai},
  year = {2025},
  doi = {10.48550/arxiv.2505.10643},
  eprint = {2505.10643},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2505.10643}
}
```

- [← previous Analyzing and Adapting Large Language Models for Few-Shot Multilingual NLU: Are We There Yet?](https://md.research.elamur.ai/free/en/papers/razumovskaia-2025-analyzing-adapting-large-language-models/)
- [next → BasqBBQ: A QA Benchmark for Assessing Social Biases in LLMs for Basque, a Low-Resource Language](https://md.research.elamur.ai/free/en/papers/zulaika-2025-basqbbq-qa-benchmark-assessing-social/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/)

## References

29 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bablu, T.A., et al.: Machine learning in automated assessment: Enhancing objectivity and efficiency in educational evaluations. Journal of Advanced Computing Systems 4(7) (2024) [find](https://scholar.google.com/scholar?q=Bablu%2C+T%2EA%2E%2C+et+al%2E%3A+Machine+learning+in+automated+assessment%3A+Enhancing+objectivity+and+efficiency+in+educational+evaluations%2E+Journal+of+Advanced+Computing+Systems+4%287%29+%282024%29)
2. Cheuk, T.: Can AI be racist? Color-evasiveness in the application of machine learning to science assessments. Science Education 105(5), 825–836 (2021) [find](https://scholar.google.com/scholar?q=Cheuk%2C+T%2E%3A+Can+AI+be+racist%3F+Color%2Devasiveness+in+the+application+of+machine+learning+to+science+assessments%2E+Science+Education+105%285%29%2C+825%E2%80%93836+%282021%29)
3. Devlin, J., Chang, M., Lee, K., Toutanova, K.: [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). CoRR abs/1810.04805 (2018), http://arxiv.org/abs/1810.04805 [↗](https://arxiv.org/abs/1810.04805)
4. Friedman, M.: The use of ranks to avoid the assumption of normality implicit in the analysis of variance. Journal of the American Statistical Association 32(200), 675–701 (1937). https://doi.org/10.1080/01621459.1937.10503522 [↗](https://doi.org/10.1080/01621459.1937.10503522)
5. Gerard, L.F., Linn, M.C.: Using automated scores of student essays to support teacher guidance in classroom inquiry. Journal of Science Teacher Education 27(1), 111–129 (2016) [find](https://scholar.google.com/scholar?q=Gerard%2C+L%2EF%2E%2C+Linn%2C+M%2EC%2E%3A+Using+automated+scores+of+student+essays+to+support+teacher+guidance+in+classroom+inquiry%2E+Journal+of+Science+Teacher+Education+27%281%29%2C+111%E2%80%93129+%282016%29)
6. Grapin, S.E.: Multimodal Assessment of English Learners in Science: Expanding What "Counts" as Evidence of Content Learning. Ph.D. thesis, New York University, Steinhardt School of Culture, Education, and Human Development (2020), [https://www.proquest.com/dissertations-theses/multimodal-assessment-english-learners-science/docview/2411140462/se-2](https://www.proquest.com/dissertations-theses/multimodal-assessment-english-learners-science/docview/2411140462/se-2) [↗](https://www.proquest.com/dissertations-theses/multimodal-assessment-english-learners-science/docview/2411140462/se-2)
7. Grapin, S.E., Llosa, L.: Multimodal tasks to assess english learners and their peers in science. Educational Assessment 27(1), 46–70 (2022). https://doi.org/10.1080/10627197.2022.2028139, https://doi.org/10.1080/10627197.2022.2028139, published online: January 17, 2022 [↗](https://doi.org/10.1080/10627197.2022.2028139)
8. Guo, S., Latif, E., Zhou, Y., Huang, X., Zhai, X.: Using generative ai and multi-agents to provide automatic feedback. arXiv preprint arXiv:2411.07407 (2024) [↗](https://arxiv.org/abs/2411.07407)
9. Guo, S., Zheng, Y., Zhai, X.: Artificial intelligence in education research during 2013–2023: A review based on bibliometric analysis. Education and Information Technologies pp. 1–23 (2024) [find](https://scholar.google.com/scholar?q=Guo%2C+S%2E%2C+Zheng%2C+Y%2E%2C+Zhai%2C+X%2E%3A+Artificial+intelligence+in+education+research+during+2013%E2%80%932023%3A+A+review+based+on+bibliometric+analysis%2E+Education+and+Information+Technologies+pp%2E+1%E2%80%9323+%282024%29)
10. Ifenthaler, D.: Automated essay scoring systems. In: Handbook of open, distance and digital education, pp. 1–15. Springer (2022) [find](https://scholar.google.com/scholar?q=Ifenthaler%2C+D%2E%3A+Automated+essay+scoring+systems%2E+In%3A+Handbook+of+open%2C+distance+and+digital+education%2C+pp%2E+1%E2%80%9315%2E+Springer+%282022%29)
11. Jeon, J., Lee, S., Coronel-Molina, S.M.: Rethinking ai: bias in speech-recognition chatbots for elt. ELT Journal 78(4), 435–445 (2024) [find](https://scholar.google.com/scholar?q=Jeon%2C+J%2E%2C+Lee%2C+S%2E%2C+Coronel%2DMolina%2C+S%2EM%2E%3A+Rethinking+ai%3A+bias+in+speech%2Drecognition+chatbots+for+elt%2E+ELT+Journal+78%284%29%2C+435%E2%80%93445+%282024%29)
12. Jourdain, L., Sharma, S.V.: Language challenges in mathematics education for english language learners: A literature review. Waikato Journal of Education 21(2) (2016) [find](https://scholar.google.com/scholar?q=Jourdain%2C+L%2E%2C+Sharma%2C+S%2EV%2E%3A+Language+challenges+in+mathematics+education+for+english+language+learners%3A+A+literature+review%2E+Waikato+Journal+of+Education+21%282%29+%282016%29)
13. Latif, E., Zhai, X., Liu, L.: AI gender bias, disparities, and fairness: Does training data matter? arXiv preprint arXiv:2312.10833 (2023), https://arxiv.org/abs/2312.10833 [↗](https://arxiv.org/abs/2312.10833)
14. LeClair, C., Doll, B., Osborn, A., Jones, K.: English language learners’ and non– english language learners’ perceptions of the classroom environment. Psychology in the Schools 46(6), 568–577 (2009) [find](https://scholar.google.com/scholar?q=LeClair%2C+C%2E%2C+Doll%2C+B%2E%2C+Osborn%2C+A%2E%2C+Jones%2C+K%2E%3A+English+language+learners%E2%80%99+and+non%E2%80%93+english+language+learners%E2%80%99+perceptions+of+the+classroom+environment%2E+Psychology+in+the+Schools+46%286%29%2C+568%E2%80%93577+%282009%29)
15. Lee, G.G., Latif, E., Wu, X., Liu, N., Zhai, X.: Applying large language models and chain-of-thought for automatic scoring. Computers and Education: Artificial Intelligence 6, 100213 (2024) [find](https://scholar.google.com/scholar?q=Lee%2C+G%2EG%2E%2C+Latif%2C+E%2E%2C+Wu%2C+X%2E%2C+Liu%2C+N%2E%2C+Zhai%2C+X%2E%3A+Applying+large+language+models+and+chain%2Dof%2Dthought+for+automatic+scoring%2E+Computers+and+Education%3A+Artificial+Intelligence+6%2C+100213+%282024%29)
16. Lee, H.S., Pallant, A., Pryputniewicz, S., Lord, T., Mulholland, M., Liu, O.L.: Automated text scoring and real-time adjustable feedback: Supporting revision of scientific arguments involving uncertainty. Science Education 103(3), 590–622 (2019) S. Guo et al. [find](https://scholar.google.com/scholar?q=Lee%2C+H%2ES%2E%2C+Pallant%2C+A%2E%2C+Pryputniewicz%2C+S%2E%2C+Lord%2C+T%2E%2C+Mulholland%2C+M%2E%2C+Liu%2C+O%2EL%2E%3A+Automated+text+scoring+and+real%2Dtime+adjustable+feedback%3A+Supporting+revision+of+scientific+arguments+involving+uncertai)
17. Li, C., Xing, W., Leite, W.: Using fair AI to predict students’ math learning outcomes in an online platform. Interactive Learning Environments 32(3), 1117–1136 (2024) [find](https://scholar.google.com/scholar?q=Li%2C+C%2E%2C+Xing%2C+W%2E%2C+Leite%2C+W%2E%3A+Using+fair+AI+to+predict+students%E2%80%99+math+learning+outcomes+in+an+online+platform%2E+Interactive+Learning+Environments+32%283%29%2C+1117%E2%80%931136+%282024%29)
18. Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., Zou, J.: [GPT detectors are biased against non-native English writers](https://md.research.elamur.ai/free/en/papers/liang-2023-gpt-detectors-biased-against-non/). Patterns 4(7) (2023)
19. Nesbitt, J., Quesen, S.: Machines imitating humans: Scoring systems and the risk of bias (December 2023), [https://www.wested.org/blog/machines-imitating-humans-scoring-systems-and-the-risk-of-bias/](https://www.wested.org/blog/machines-imitating-humans-scoring-systems-and-the-risk-of-bias/), accessed: February 19, 2025 [↗](https://www.wested.org/blog/machines-imitating-humans-scoring-systems-and-the-risk-of-bias/)
20. Reed, D.K., Mercer, S.H.: Potential scoring and predictive bias in interim and summative writing assessments. School Psychology 38(4), 215 (2023) [find](https://scholar.google.com/scholar?q=Reed%2C+D%2EK%2E%2C+Mercer%2C+S%2EH%2E%3A+Potential+scoring+and+predictive+bias+in+interim+and+summative+writing+assessments%2E+School+Psychology+38%284%29%2C+215+%282023%29)
21. Schwartz, R., Vassilev, A., Greene, K., Perine, L., Burt, A., Hall, P.: Towards a standard for identifying and managing bias in artificial intelligence, vol. 3. US Department of Commerce, National Institute of Standards and Technology (2022) [find](https://scholar.google.com/scholar?q=Schwartz%2C+R%2E%2C+Vassilev%2C+A%2E%2C+Greene%2C+K%2E%2C+Perine%2C+L%2E%2C+Burt%2C+A%2E%2C+Hall%2C+P%2E%3A+Towards+a+standard+for+identifying+and+managing+bias+in+artificial+intelligence%2C+vol%2E+3%2E+US+Department+of+Commerce%2C+National+Ins)
22. Weigle, S.C.: English language learners and automated scoring of essays: Critical considerations. Assessing Writing 18(1), 85–99 (2013) [find](https://scholar.google.com/scholar?q=Weigle%2C+S%2EC%2E%3A+English+language+learners+and+automated+scoring+of+essays%3A+Critical+considerations%2E+Assessing+Writing+18%281%29%2C+85%E2%80%9399+%282013%29)
23. WIDA Consortium: WIDA English Language Development Standards Framework, 2020 Edition. Tech. rep., Board of Regents of the University of Wisconsin-Madison (2020), [https://wida.wisc.edu/resources/wida-english-language-development-standards-framework-2020-edition](https://wida.wisc.edu/resources/wida-english-language-development-standards-framework-2020-edition) [↗](https://wida.wisc.edu/resources/wida-english-language-development-standards-framework-2020-edition)
24. Wilson, J., Huang, Y.: Validity of automated essay scores for elementary-age English language learners: Evidence of bias? Assessing Writing 60, 100815 (2024) [find](https://scholar.google.com/scholar?q=Wilson%2C+J%2E%2C+Huang%2C+Y%2E%3A+Validity+of+automated+essay+scores+for+elementary%2Dage+English+language+learners%3A+Evidence+of+bias%3F+Assessing+Writing+60%2C+100815+%282024%29)
25. Zhai, X.: AI and machine learning for next generation science assessments. In: Machine Learning, Natural Language Processing, and Psychometrics. p. 201. IAP (2024) [find](https://scholar.google.com/scholar?q=Zhai%2C+X%2E%3A+AI+and+machine+learning+for+next+generation+science+assessments%2E+In%3A+Machine+Learning%2C+Natural+Language+Processing%2C+and+Psychometrics%2E+p%2E+201%2E+IAP+%282024%29)
26. Zhai, X., He, P., Krajcik, J.: Applying machine learning to automatically assess scientific models. Journal of Research in Science Teaching 59(10), 1765–1794 (2022) [find](https://scholar.google.com/scholar?q=Zhai%2C+X%2E%2C+He%2C+P%2E%2C+Krajcik%2C+J%2E%3A+Applying+machine+learning+to+automatically+assess+scientific+models%2E+Journal+of+Research+in+Science+Teaching+59%2810%29%2C+1765%E2%80%931794+%282022%29)
27. Zhai, X., Krajcik, J.: Pseudo artificial intelligence bias p. 568–578 (2024), https://doi.org/10.1093/oso/9780198882077.003.0025 [↗](https://doi.org/10.1093/oso/9780198882077.003.0025)
28. Zhai, X., Nehm, R.H.: AI and formative assessment: The train has left the station. Journal of Research in Science Teaching 60(6), 1390–1398 (2023) [find](https://scholar.google.com/scholar?q=Zhai%2C+X%2E%2C+Nehm%2C+R%2EH%2E%3A+AI+and+formative+assessment%3A+The+train+has+left+the+station%2E+Journal+of+Research+in+Science+Teaching+60%286%29%2C+1390%E2%80%931398+%282023%29)
29. Zhai, X., Yin, Y., Pellegrino, J.W., Haudek, K.C., Shi, L.: Applying machine learning in science assessment: a systematic review. Studies in Science Education 56(1), 111–151 (2020) [find](https://scholar.google.com/scholar?q=Zhai%2C+X%2E%2C+Yin%2C+Y%2E%2C+Pellegrino%2C+J%2EW%2E%2C+Haudek%2C+K%2EC%2E%2C+Shi%2C+L%2E%3A+Applying+machine+learning+in+science+assessment%3A+a+systematic+review%2E+Studies+in+Science+Education+56%281%29%2C+111%E2%80%93151+%282020%29)

*Updated 31 October 2026*