---
title: "Can Large Language Models Reliably Correct Errors in"
description: "The paper asks whether LLMs can fix the output of a speech recognizer for West Frisian, a language with about 400,000… 21 references."
authors: ["Yun Hao", "Reihaneh Amooie", "Wietse de Vries", "Rik van Noord", "Martijn Wieling"]
year: 2026
venue: "arXiv:2605.19711"
source: "https://arxiv.org/abs/2605.19711"
doi: "10.48550/arxiv.2605.19711"
arxiv: "2605.19711"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hao-2026-large-language-models-reliably-correct/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hao-2026-large-language-models-reliably-correct/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/24/) / Can Large Language Models Reliably Correct Errors in Low-Resource ASR? A Contamination-Aware Case Study on West Frisian

# Can Large Language Models Reliably Correct Errors in Low-Resource ASR? A Contamination-Aware Case Study on West Frisian

**Authors:** [Yun Hao](https://md.research.elamur.ai/free/en/authors/yun-hao/), [Reihaneh Amooie](https://md.research.elamur.ai/free/en/authors/reihaneh-amooie/), [Wietse de Vries](https://md.research.elamur.ai/free/en/authors/wietse-de-vries/), [Rik van Noord](https://md.research.elamur.ai/free/en/authors/rik-van-noord/) and [Martijn Wieling](https://md.research.elamur.ai/free/en/authors/martijn-wieling/) · **Year:** 2026 · **Venue:** arXiv:2605.19711 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2605.19711) · [PDF](https://arxiv.org/pdf/2605.19711) · [DOI 10.48550/arxiv.2605.19711](https://doi.org/10.48550/arxiv.2605.19711) · [arXiv 2605.19711](https://arxiv.org/abs/2605.19711) · [OpenAlex](https://openalex.org/W7161798603)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper asks whether LLMs can fix the output of a speech recognizer for West Frisian, a language with about 400,000 speakers and little training data. It also asks whether such gains are genuine or come from the model having seen the test sentences during pretraining. The authors fine-tune XLS-R 1B on the Common Voice 17.0 Frisian training split. For each utterance they decode a five-best list and give it to an LLM, which either picks one hypothesis or writes a new corrected transcript, in zero-shot or few-shot mode. The LLMs are GPT-4o-mini, GPT-5.1, and Qwen3-8B, both untouched and LoRA-tuned. To control for contamination, they recorded a new Frisian Offline Dataset whose texts were never published online. GPT-5.1 used generatively gives the largest gains and beats the five-best oracle on both test sets. Its relative improvement on the private set is similar to the one on Common Voice, which suggests real correction rather than memorization. This matters because many low-resource benchmarks reuse a few public text sources.

## Key points
- **Data.** Common Voice 17.0 Frisian uses the official splits: train 3,921 utterances (5.5 h), validation 4.6 h, test 3,171 utterances from 904 speakers (4.7 h). The Frisian Offline Dataset has 811 utterances (1.5 h) read by four male native speakers in a lab. Its texts are:
  - 103 sentences from a storybook with no digital copy online;
  - 100 sentences newly written by a native speaker.
- **ASR setup.** XLS-R 1B was fine-tuned with CTC for 2,000 steps. Five-best lists come from beam search with beam width 50. Qwen3-8B was LoRA-tuned (r=16, α=32) on five-best lists paired with reference transcripts.
- **Common Voice WER.**
  - XLS-R baseline 13.5%, trigram LM 12.1%, five-best oracle 9.6%.
  - GPT-5.1 generative correction reaches 8.9% (3- and 5-shot), against 12.1% when it only selects a hypothesis.
  - GPT-4o-mini sits around 12.2–12.8%.
- **Offline WER.**
  - Baseline 21.1%, trigram 19.2%, oracle 18.0%.
  - GPT-5.1 drops to 13.8% (5- and 10-shot). Both sets thus show roughly a one-third relative reduction.
- **Qwen3-8B.** The base model makes Common Voice output worse than the baseline (13.8–14.7%). Fine-tuning brings it only to about baseline level on both sets.

## Interesting ideas
- Build a contamination-proof test set from text that cannot be on the web, such as a book with no digital edition plus sentences freshly written by a native speaker, and compare it against a public benchmark.
- Judge a contamination check by whether the relative gain holds on both sets, not by absolute scores, since the private set is harder for the base system anyway.
- Report every corrector against two reference lines, the uncorrected baseline and the oracle best of the N candidates, so it is clear whether the corrector only selects well or truly adds new information.
- Contrast "pick one of the N hypotheses" with "write a new answer seeing all N"; free generation gained much more because the right output was often absent from the list.
- A weaker or small model asked to correct can make output worse than doing nothing, so over-correction should be measured directly.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (answer): Add a small set of newly written fables and questions to the public tale QA set, so LLM-teacher or pretrained components cannot score well by remembering famous stories.
- **I2** (learn): When validating LLM teacher tiers, compare their relative improvement over the Mova baseline on public and private material, which works as a negative control against memorized benchmarks.
- **I3** (tree): Compute an oracle LAS over the k-best trees from the witness parsers, then check whether a repair step beats that ceiling or just picks among candidates.
- **I4** (learn): An LLM teacher could receive disagreeing parses from several witness parsers and propose a repaired tree rather than vote, with the result checked explicitly before it becomes silver data.
- **I5** (learn): Self-induced repair rules should always be tested against the no-repair baseline, and any rule whose application raises errors should be rejected automatically.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Groningen](https://md.research.elamur.ai/free/en/universities/university-of-groningen/) — University, Groningen, Netherlands · 70 works · 5,142 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🔊 Signal Processing](https://md.research.elamur.ai/free/en/shelves/computer-science/signal-processing/) · 452 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [🎙️ Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) · 480 works · [🌱 Low-resource languages](https://md.research.elamur.ai/free/en/topics/low-resource-languages/) · 267 works

## Cite

Cite as: Yun Hao, Reihaneh Amooie, Wietse de Vries, Rik van Noord and Martijn Wieling. 2026. Can Large Language Models Reliably Correct Errors in Low-Resource ASR? A Contamination-Aware Case Study on West Frisian. arXiv:2605.19711.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{hao2026can,
  title = {{Can Large Language Models Reliably Correct Errors in Low-Resource ASR? A Contamination-Aware Case Study on West Frisian}},
  author = {Yun Hao, Reihaneh Amooie, Wietse {de Vries}, Rik {van Noord} and Martijn Wieling},
  year = {2026},
  doi = {10.48550/arxiv.2605.19711},
  eprint = {2605.19711},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2605.19711}
}
```

- [← previous Can Large Language Models Infer Human Actions and Motives? Evaluation in Social Prediction and Inspection Games](https://md.research.elamur.ai/free/en/papers/shrestha-2026-can-large-language-models-infer/)
- [next → Can Large Language Models Replace Statistical Software?](https://md.research.elamur.ai/free/en/papers/staudt-2026-can-large-language-models-replace/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/24/)

## References

21 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. A. Babu, C. Wang, A. Tjandra, K. Lakhotia, Q. Xu, N. Goyal, K. Singh, P. von Platen, Y. Saraf, J. Pino, A. Baevski, A. Conneau, and M. Auli, “[XLS-R: Self-supervised Cross-lingual Speech Representation Learning at Scale](https://md.research.elamur.ai/free/en/papers/babu-2021-xls-r-self-supervised-cross/),” in Interspeech 2022, 2022, pp. 2278–2282.
2. A. Radford, J. W. Kim, T. Xu, G. Brockman, C. McLeavey, and I. Sutskever, “[Robust speech recognition via large-scale weak supervision](https://md.research.elamur.ai/free/en/papers/radford-2022-robust-speech-recognition-large-scale/),” in International conference on machine learning. PMLR, 2023, pp. 28 492–28 518. [↗](https://doi.org/10.48550/arxiv.2212.04356)
3. M. Bartelds, N. San, B. McDonnell, D. Jurafsky, and M. Wieling, “Making more of little data: Improving low-resource automatic speech recognition using data augmentation,” in Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), 2023, pp. 715–729. [find](https://scholar.google.com/scholar?q=M%2E+Bartelds%2C+N%2E+San%2C+B%2E+McDonnell%2C+D%2E+Jurafsky%2C+and+M%2E+Wieling%2C+%E2%80%9CMaking+more+of+little+data%3A+Improving+low%2Dresource+automatic+speech+recognition+using+data+augmentation%2C%E2%80%9D+in+Proceedings+of+the+61st+An)
4. Z. X. Yong, V. Pratap, M. Auli, and J. Maillard, “Effects of Speaker Count, Duration, and Accent Diversity on Zero-Shot Accent Robustness in Low-Resource ASR,” in Interspeech 2025, 2025, pp. 1148–1152. [find](https://scholar.google.com/scholar?q=Z%2E+X%2E+Yong%2C+V%2E+Pratap%2C+M%2E+Auli%2C+and+J%2E+Maillard%2C+%E2%80%9CEffects+of+Speaker+Count%2C+Duration%2C+and+Accent+Diversity+on+Zero%2DShot+Accent+Robustness+in+Low%2DResource+ASR%2C%E2%80%9D+in+Interspeech+2025%2C+2025%2C+pp%2E+1148%E2%80%931152)
5. R. Ma, M. Qian, P. Manakul, M. Gales, and K. Knill, “Can generative large language models perform ASR error correction?” arXiv preprint arXiv:2307.04172, 2023. [↗](https://arxiv.org/abs/2307.04172)
6. C.-H. H. Yang, Y. Gu, Y.-C. Liu, S. Ghosh, I. Bulyko, and A. Stolcke, “Generative speech recognition error correction with large language models and task-activating prompting,” in 2023 IEEE Automatic Speech Recognition and Understanding Workshop (ASRU). IEEE, 2023, pp. 1–8. [find](https://scholar.google.com/scholar?q=C%2E%2DH%2E+H%2E+Yang%2C+Y%2E+Gu%2C+Y%2E%2DC%2E+Liu%2C+S%2E+Ghosh%2C+I%2E+Bulyko%2C+and+A%2E+Stolcke%2C+%E2%80%9CGenerative+speech+recognition+error+correction+with+large+language+models+and+task%2Dactivating+prompting%2C%E2%80%9D+in+2023+IEEE+Automatic+)
7. C. Chen, Y. Hu, C.-H. H. Yang, S. M. Siniscalchi, P.-Y. Chen, and E.-S. Chng, “Hyporadise: An open baseline for generative speech recognition with large language models,” Advances in Neural Information Processing Systems, vol. 36, pp. 31 665–31 688, 2023. [find](https://scholar.google.com/scholar?q=C%2E+Chen%2C+Y%2E+Hu%2C+C%2E%2DH%2E+H%2E+Yang%2C+S%2E+M%2E+Siniscalchi%2C+P%2E%2DY%2E+Chen%2C+and+E%2E%2DS%2E+Chng%2C+%E2%80%9CHyporadise%3A+An+open+baseline+for+generative+speech+recognition+with+large+language+models%2C%E2%80%9D+Advances+in+Neural+Informatio)
8. N. Yamashita, M. Yamamoto, H. Kokubo, and Y. Kawaguchi, “LLM-based Generative Error Correction for Rare Words with Synthetic Data and Phonetic Context,” in Interspeech 2025, 2025, pp. 3653–3657. [↗](https://doi.org/10.48550/arxiv.2505.17410)
9. R. Ardila, M. Branson, K. Davis, M. Kohler, J. Meyer, M. Henretty, R. Morais, L. Saunders, F. Tyers, and G. Weber, “[Common voice: A massively-multilingual speech corpus](https://md.research.elamur.ai/free/en/papers/ardila-2019-common-voice-massively-multilingual-speech/),” in Proceedings of the twelfth language resources and evaluation conference, 2020, pp. 4218–4222. [↗](https://doi.org/10.48550/arxiv.1912.06670)
10. M. Naderi, E. Hermann, A. Nanchen, S. Hovsepyan, and M. Magimai.-Doss, “Towards interfacing large language models with ASR systems using confidence measures and prompting,” in Interspeech 2024, 2024, pp. 2980–2984. [find](https://scholar.google.com/scholar?q=M%2E+Naderi%2C+E%2E+Hermann%2C+A%2E+Nanchen%2C+S%2E+Hovsepyan%2C+and+M%2E+Magimai%2E%2DDoss%2C+%E2%80%9CTowards+interfacing+large+language+models+with+ASR+systems+using+confidence+measures+and+prompting%2C%E2%80%9D+in+Interspeech+2024%2C+2024%2C+)
11. S. Li, C. Chen, C. Y. Kwok, C. Chu, E. S. Chng, and H. Kawai, “Investigating ASR Error Correction with Large Language Model and Multilingual 1-best Hypotheses,” in Interspeech 2024, 2024, pp. 1315–1319. [find](https://scholar.google.com/scholar?q=S%2E+Li%2C+C%2E+Chen%2C+C%2E+Y%2E+Kwok%2C+C%2E+Chu%2C+E%2E+S%2E+Chng%2C+and+H%2E+Kawai%2C+%E2%80%9CInvestigating+ASR+Error+Correction+with+Large+Language+Model+and+Multilingual+1%2Dbest+Hypotheses%2C%E2%80%9D+in+Interspeech+2024%2C+2024%2C+pp%2E+1315%E2%80%93131)
12. Z. Yang, Z. Wan, S. Li, C.-H. H. Yang, and C. Chu, “CoVoGER: A multilingual multitask benchmark for speech-to-text generative error correction with large language models,” in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, pp. 6313–6325. [find](https://scholar.google.com/scholar?q=Z%2E+Yang%2C+Z%2E+Wan%2C+S%2E+Li%2C+C%2E%2DH%2E+H%2E+Yang%2C+and+C%2E+Chu%2C+%E2%80%9CCoVoGER%3A+A+multilingual+multitask+benchmark+for+speech%2Dto%2Dtext+generative+error+correction+with+large+language+models%2C%E2%80%9D+in+Proceedings+of+the+2025+C)
13. A. Xu, T. Feng, S. H. Kim, S. Bishop, C. Lord, and S. Narayanan, “Large Language Models based ASR Error Correction for Child Conversations,” in Interspeech 2025, 2025, pp. 2840–2844. [find](https://scholar.google.com/scholar?q=A%2E+Xu%2C+T%2E+Feng%2C+S%2E+H%2E+Kim%2C+S%2E+Bishop%2C+C%2E+Lord%2C+and+S%2E+Narayanan%2C+%E2%80%9CLarge+Language+Models+based+ASR+Error+Correction+for+Child+Conversations%2C%E2%80%9D+in+Interspeech+2025%2C+2025%2C+pp%2E+2840%E2%80%932844%2E)
14. O. Sainz, J. Campos, I. Garcı́a-Ferrero, J. Etxaniz, O. L. de Lacalle, and E. Agirre, “[NLP evaluation in trouble: On the need to measure LLM data contamination for each benchmark](https://md.research.elamur.ai/free/en/papers/sainz-2023-nlp-evaluation-trouble-need-measure/),” in Findings of the Association for Computational Linguistics: EMNLP 2023, H. Bouamor, J. Pino, and K. Bali, Eds. Singapore: Association for Computational Linguistics, Dec. 2023, pp. 10 776–10 787. [Online]. Available: https://aclanthology.org/2023.findings-emnlp.722/ [↗](https://aclanthology.org/2023.findings-emnlp.722/)
15. Y. Li, Y. Guo, F. Guerin, and C. Lin, “An open-source data contamination report for large language models,” in Findings of the Association for Computational Linguistics: EMNLP 2024, 2024, pp. 528–541. [find](https://scholar.google.com/scholar?q=Y%2E+Li%2C+Y%2E+Guo%2C+F%2E+Guerin%2C+and+C%2E+Lin%2C+%E2%80%9CAn+open%2Dsource+data+contamination+report+for+large+language+models%2C%E2%80%9D+in+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2024%2C+2024%2C+pp%2E+528%E2%80%93541%2E)
16. C. Deng, Y. Zhao, X. Tang, M. Gerstein, and A. Cohan, “[Investigating data contamination in modern benchmarks for large language models](https://md.research.elamur.ai/free/en/papers/deng-2024-investigating-data-contamination-modern-benchmarks/),” in Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), 2024, pp. 8706–8719.
17. S. Chen, Y. Chen, Z. Li, Y. Jiang, Z. Wan, Y. He, D. Ran, T. Gu, H. Li, T. Xie et al., “Benchmarking large language models under data contamination: A survey from static to dynamic evaluation,” in Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing, 2025, pp. 10 091–10 109. [find](https://scholar.google.com/scholar?q=S%2E+Chen%2C+Y%2E+Chen%2C+Z%2E+Li%2C+Y%2E+Jiang%2C+Z%2E+Wan%2C+Y%2E+He%2C+D%2E+Ran%2C+T%2E+Gu%2C+H%2E+Li%2C+T%2E+Xie+et+al%2E%2C+%E2%80%9CBenchmarking+large+language+models+under+data+contamination%3A+A+survey+from+static+to+dynamic+evaluation%2C%E2%80%9D+in+Pro)
18. R. Ma, M. Qian, M. Gales, and K. Knill, “ASR error correction using large language models,” IEEE Transactions on Audio, Speech and Language Processing, 2025. [↗](https://doi.org/10.1109/taslpro.2025.3551083)
19. S. Balloccu, P. Schmidtová, M. Lango, and O. Dušek, “[Leak, cheat, repeat: Data contamination and evaluation malpractices in closed-source LLMs](https://md.research.elamur.ai/free/en/papers/balloccu-2024-leak-cheat-repeat-data-contamination/),” in Proceedings of the 18th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), 2024, pp. 67–93. [↗](https://doi.org/10.48550/arxiv.2402.03927)
20. C. Xu, S. Guan, D. Greene, M. Kechadi et al., “Benchmark data contamination of [large language models: A survey](https://md.research.elamur.ai/free/en/papers/minaee-2024-large-language-models-survey/),” arXiv preprint arXiv:2406.04244, 2024. [↗](https://arxiv.org/abs/2406.04244)
21. A. Baevski, Y. Zhou, A. Mohamed, and M. Auli, “[wav2vec 2.0: A framework for self-supervised learning of speech representations](https://md.research.elamur.ai/free/en/papers/baevski-2020-wav2vec-2-0-framework-self/),” Advances in neural information processing systems, vol. 33, pp. 12 449–12 460, 2020.

*Updated 1 November 2026*