--- title: "Spoken English Intelligibility Remediation with PocketSphinx" description: "Standard computer-aided pronunciation teaching (CAPT) imitates expert judges or uses goodness of… 82% agreement. 1 reference." authors: ["Yuan Gao; Brij Mohan Lal Srivastava;", "James Salsman"] year: 2017 venue: "arXiv:1709.01713 · IEEE Advanced Information Management, Communications, Electronic and Automation Control Conference 2 (2018)" source: "https://arxiv.org/abs/1709.01713" doi: "10.1109/imcec.2018.8469649" arxiv: "1709.01713" licence: "CC BY-SA 4.0 (arXiv)" licence_url: "http://creativecommons.org/licenses/by-sa/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/gao-2017-spoken-english-intelligibility-remediation-pocketsphinx/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/gao-2017-spoken-english-intelligibility-remediation-pocketsphinx/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2017](https://md.research.elamur.ai/free/en/papers/2017/page/4/) / Spoken English Intelligibility Remediation with PocketSphinx Alignment and Feature Extraction Improves Substantially over the State of the Art # Spoken English Intelligibility Remediation with PocketSphinx Alignment and Feature Extraction Improves Substantially over the State of the Art **Authors:** Yuan Gao; Brij Mohan Lal Srivastava; and James Salsman · **Year:** 2017 · **Venue:** arXiv:1709.01713 · IEEE Advanced Information Management, Communications, Electronic and Automation Control Conference 2 (2018) · **Licence:** [CC BY-SA 4.0 (arXiv)](http://creativecommons.org/licenses/by-sa/4.0/) **Links:** [Publisher's page](https://doi.org/10.1109/imcec.2018.8469649) · [PDF](https://arxiv.org/pdf/1709.01713) · [DOI 10.1109/imcec.2018.8469649](https://doi.org/10.1109/imcec.2018.8469649) · [arXiv 1709.01713](https://arxiv.org/abs/1709.01713) · [OpenAlex](https://openalex.org/W2753797950) **Citations:** 5 citations (OpenAlex, 2026-10-14) ## Summary Standard computer-aided pronunciation teaching (CAPT) imitates expert judges or uses goodness of pronunciation (GOP) scores. Neither tracks well whether a listener can actually understand the learner. Prior work cited by the authors found that obvious mispronunciations explain only about 16% of word-level intelligibility variation. The authors instead predict "authentic intelligibility": whether crowdworkers on Amazon Mechanical Turk transcribe a learner's word correctly. They adapt PocketSphinx in two steps: - It first aligns the expected phonemes. - It then re-runs recognition on short aligned windows with tightly constrained grammars. This yields per-phoneme scores for substitution and for insertion/deletion, along with duration, alignment acoustic score and physiological articulation features. An RBF-kernel SVM in scikit-learn reaches 82% agreement with transcription correctness on roughly 700 basic English words. A logistic regression baseline reaches 75%, the level reported in earlier studies. The extractor, featex.c, is open-source and also runs in browsers via pocketsphinx.js. The system is being deployed for 17zuoye.com's K-6 learners and in Wiktionary, so the approach gives practical, explainable pronunciation feedback at scale. ## Key points - Each phoneme gets nine features: duration, log acoustic score (the GOP numerator), a [0,1] substitution score, a [0,1] insertion/deletion score, and five articulation features (place, closedness, roundedness, voicing, neighbour confusability). - The data covers about 700 words, with about 30 recordings per word and 4 MTurk transcripts per recording. The SVM scores 82%, against 75% for logistic regression on four features. The authors report that more recordings and more transcripts per recording clearly help. - The PocketSphinx settings are non-default: 65 frames/s, -topn 64, very wide beams, and -fsgusefiller off. Despite this, feature extraction runs faster than real time on a laptop and also runs in-browser. - When transcribers disagree about an utterance, accuracy is reported as a share of the best achievable score for it. - MTurk responses came faster at $0.03 per transcript than at $0.15. The authors are also exploring using learners themselves as transcribers. - Code is released under the MIT license: featex.c, pocketsphinx.js, the iremedy demo and the proneval-service. ## Interesting ideas - Train against the real downstream outcome (did a listener understand?) rather than against expert labels of "correctness", which may track the outcome poorly. - Normalise accuracy by the ceiling implied by annotator disagreement, so items with split labels do not unfairly penalise a model. - Probe one slot at a time: hold the neighbours fixed, open the middle position to every alternative, and use the expected item's rank in the n-best list as a graded confidence score. - Find what to fix by perturbing each unit's features in the trained model and seeing which change raises the overall success probability most. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (answer): Judge parser or frame changes partly by whether school-test and fable-QA answers get better, not only by tree-label agreement. - **I2** (learn): When witness parsers or LLM teacher tiers disagree on silver data, score models against the maximum attainable agreement per sentence. - **I3** (tree): Pin all other arcs, re-score every candidate head or label for one token, and use the gold candidate's rank as an explainable per-arc confidence feature. - **I4** (learn): Counterfactually flip single tags or attachments to see which one changes a wrong answer, and use that to seed self-induced repair rules. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [International Institute of Information Technology, Hyderabad](https://md.research.elamur.ai/free/en/universities/international-institute-of-information-technology-hyderabad/) — University, Hyderabad, India · 24 works · 1,719 citations - [Beijing University of Technology](https://md.research.elamur.ai/free/en/universities/beijing-university-of-technology/) — University, Beijing, China · 4 works · 1,819 citations 📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 798 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 463 works · [🔊 Signal Processing](https://md.research.elamur.ai/free/en/shelves/computer-science/signal-processing/) · 213 works 🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 1,373 works ## Cite Cite as: Yuan Gao; Brij Mohan Lal Srivastava; and James Salsman. 2017. Spoken English Intelligibility Remediation with PocketSphinx Alignment and Feature Extraction Improves Substantially over the State of the Art. arXiv:1709.01713 · IEEE Advanced Information Management, Communications, Electronic and Automation Control Conference 2 (2018). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @misc{srivastava2017spoken, title = {{Spoken English Intelligibility Remediation with PocketSphinx Alignment and Feature Extraction Improves Substantially over the State of the Art}}, author = {Yuan Gao; Brij Mohan Lal Srivastava; and James Salsman}, year = {2017}, doi = {10.1109/imcec.2018.8469649}, eprint = {1709.01713}, archivePrefix = {arXiv}, url = {https://doi.org/10.1109/imcec.2018.8469649} } ``` - [← previous Postcolonial Literature and World Englishes: A Corpus-Based Approach of Modes of Representation of the Non-Standard in Writing](https://md.research.elamur.ai/free/en/papers/percillier-2017-postcolonial-literature-world-englishes-corpus/) - [next → The theory and syntactic representation of control structures: An analysis from Amharic](https://md.research.elamur.ai/free/en/papers/leung-2017-theory-syntactic-representation-control-structures/) - [↑ back to papers of 2017](https://md.research.elamur.ai/free/en/papers/2017/page/4/) ## References 1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. [1] Hiroshi Kibishi and Seiichi Nakagawa, “New feature parameters for pronunciation evaluation in english presentations at international conferences,” in Twelfth Annual Conference of the International Speech Communication Association, 2011. *Fig. 3. Determining feedback: Adjusting the feature scores for each phoneme changes the probability of intelligibility of the whole word. The adjustments which make the best changes signal which phoneme(s) need improvement the most.* [find](https://scholar.google.com/scholar?q=%5B1%5D+Hiroshi+Kibishi+and+Seiichi+Nakagawa%2C+%E2%80%9CNew+feature+parameters+for+pronunciation+evaluation+in+english+presentations+at+international+conferences%2C%E2%80%9D+in+Twelfth+Annual+Conference+of+the+International) *Updated 24 October 2026*