---
title: "AISPEECH-SJTU accent identification system for the Accented"
description: "The paper presents the top-ranked system for the accent-classification task of the… 83.63% accuracy. 26 references, cited by 1 paper here."
authors: ["Houjun Huang", "Xu Xiang", "Yexin Yang", "Rao Ma", "Yanmin Qian"]
year: 2021
venue: "arXiv:2102.09828"
source: "https://arxiv.org/abs/2102.09828"
doi: "10.48550/arxiv.2102.09828"
arxiv: "2102.09828"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/huang-2021-aispeech-sjtu-accent-identification-system/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/huang-2021-aispeech-sjtu-accent-identification-system/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/7/) / AISPEECH-SJTU accent identification system for the Accented English Speech Recognition Challenge

# AISPEECH-SJTU accent identification system for the Accented English Speech Recognition Challenge

**Authors:** Houjun Huang, Xu Xiang, Yexin Yang, Rao Ma and [Yanmin Qian](https://md.research.elamur.ai/free/en/authors/yanmin-qian/) · **Year:** 2021 · **Venue:** arXiv:2102.09828 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2102.09828) · [PDF](https://arxiv.org/pdf/2102.09828) · [DOI 10.48550/arxiv.2102.09828](https://doi.org/10.48550/arxiv.2102.09828) · [arXiv 2102.09828](https://arxiv.org/abs/2102.09828) · [OpenAlex](https://openalex.org/W3133187441)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [1](https://md.research.elamur.ai/free/en/papers/huang-2021-aispeech-sjtu-accent-identification-system/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper presents the top-ranked system for the accent-classification task of the Accented English Speech Recognition Challenge (AESRC), held at Interspeech 2020. The task is to label English speech with one of 8 national accents. Only about 160 hours of accented training data were available, plus the unlabeled-for-accent Librispeech corpus. The authors make three changes to a standard pipeline:

- **PPG features.** Instead of raw FBANK features, the classifier reads phone posteriorgrams (PPG). These come from a speaker-independent ASR model trained on all available speech.
- **TTS data augmentation.** A FastSpeech + LPCNet synthesizer, fine-tuned separately for each accent, produces new accented speech in the voices of real training speakers.
- **Test-time augmentation and fusion.** Tempo-perturbed copies of each test utterance are added at inference. The final classifier combines embeddings from two networks (TDNN and RES2SETDNN).

The system averaged 83.63% accuracy on the evaluation set and won the track by more than 10 points absolute. The work shows that features from a related, data-rich task, together with synthetic data, can make up for very small labeled sets.

## Key points
- **Data:** 160 h of accented English, about 20 h per accent, from US, UK, CHN, IND, JPN, KR, PT and RU. Librispeech was the auxiliary corpus.
- **Effect of PPG:** on the development set, switching from FBANK to PPG raised a TDNN from 60.24% to 83.74% average accuracy. Adding conventional augmentation (MUSAN noise, RIR reverberation, SoX tempo) and then TTS augmentation brought it to 89.74%.
- **TTS setup:**
  - A TDNN x-vector model supplies speaker identity.
  - The FastSpeech input vocabulary has 41 symbols: phonemes, a pause token and a sentence-boundary token.
  - There are 8 accent-specific fine-tuned FastSpeech models and two LPCNet vocoders, one male and one female.
  - Speaker statistics are computed from groups of 30 utterances.
- **Fusion model:** each sub-model is first trained alone with an additive angular margin softmax loss. Its embedding extractor is then frozen, and a linear classifier is trained on the concatenated embeddings.
- **Overall gain:** the final system reached 91.13% average accuracy on the development set, against 76.1% for the challenge baseline.

## Interesting ideas
- Instead of learning from low-level signals, the classifier takes the soft output distributions of a model trained on a richer neighbouring task. This filters out nuisance variation and lets unlabeled data contribute indirectly.
- Synthetic data is generated by holding one attribute fixed (the speaker's style) and swapping the target attribute (the accent). This gives targeted coverage of under-represented classes rather than generic noise.
- At inference, perturbed versions of the input are attached to the original and scored together, so robustness improves without retraining.
- Fusion is done progressively: strong sub-models are trained separately, then frozen, and only a simple linear layer learns to combine them. This keeps the combiner small and inspectable.
- Results are reported per class, not just as an average. This exposes where each change actually helps; for example, TTS augmentation mainly lifted CHN and RU.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (tokens): tagger posterior distributions over UPOS/morph features could serve as compact, explainable input features for downstream classifiers, such as register or dialect detection, instead of raw word forms.
- **I2** (learn): the text → graph → text path could rephrase sentences while keeping their meaning graph fixed, producing silver examples for rare constructions that witness parsers then verify.
- **I3** (answer): a reader could answer several meaning-preserving rewordings of a test question and keep the answer they agree on, flagging disagreements as low confidence.
- **I4** (learn): outputs of the perceptron parser and gparse could feed a small linear arbiter whose weights show which witness is trusted for which relation.
- **I5** (learn): evaluation tables could break scores down by deprel or construction type, so that a gain in one area cannot hide a regression in another.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Shanghai Jiao Tong University](https://md.research.elamur.ai/free/en/universities/shanghai-jiao-tong-university/) — University, Shanghai, China · 149 works · 16,161 citations

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Experimental and Cognitive Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/experimental-and-cognitive-psychology/) · 870 works · [🔊 Signal Processing](https://md.research.elamur.ai/free/en/shelves/computer-science/signal-processing/) · 452 works

🧭 Topics (2): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works · [🎙️ Speech & spoken language](https://md.research.elamur.ai/free/en/topics/speech/) · 480 works

## Cite

Cite as: Houjun Huang, Xu Xiang, Yexin Yang, Rao Ma and Yanmin Qian. 2021. AISPEECH-SJTU accent identification system for the Accented English Speech Recognition Challenge. arXiv:2102.09828.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{huang2021aispeech,
  title = {{AISPEECH-SJTU accent identification system for the Accented English Speech Recognition Challenge}},
  author = {Houjun Huang, Xu Xiang, Yexin Yang, Rao Ma and Yanmin Qian},
  year = {2021},
  doi = {10.48550/arxiv.2102.09828},
  eprint = {2102.09828},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2102.09828}
}
```

- [← previous A Corpus-based Syntactic Analysis of Two-termed Unlike Coordination](https://md.research.elamur.ai/free/en/papers/kallini-2021-corpus-based-syntactic-analysis-two/)
- [next → An alternatives account of 'most’ and 'more than half’](https://md.research.elamur.ai/free/en/papers/carcassi-2021-alternatives-account-most-more-than/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/7/)

## References

26 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Marina Piat, Dominique Fohr, and Irina Illina, “Foreign accent identification based on prosodic parameters,” in Ninth Annual Conference of the International Speech Communication Association, 2008. [find](https://scholar.google.com/scholar?q=Marina+Piat%2C+Dominique+Fohr%2C+and+Irina+Illina%2C+%E2%80%9CForeign+accent+identification+based+on+prosodic+parameters%2C%E2%80%9D+in+Ninth+Annual+Conference+of+the+International+Speech+Communication+Association%2C+2008%2E)
2. Kay Berkling, Marc A Zissman, Julie Vonwiller, and Christopher Cleirigh, “Improving accent identification through knowledge of english syllable structure,” in Fifth International Conference on Spoken Language Processing, 1998. [find](https://scholar.google.com/scholar?q=Kay+Berkling%2C+Marc+A+Zissman%2C+Julie+Vonwiller%2C+and+Christopher+Cleirigh%2C+%E2%80%9CImproving+accent+identification+through+knowledge+of+english+syllable+structure%2C%E2%80%9D+in+Fifth+International+Conference+on+Spoken+)
3. Too Chen, Chao Huang, Eric Chang, and Jingehan Wang, “Automatic accent identification using gaussian mixture models,” in IEEE Workshop on Automatic Speech Recognition and Understanding, 2001. ASRU’01. IEEE, 2001, pp. 343–346. [find](https://scholar.google.com/scholar?q=Too+Chen%2C+Chao+Huang%2C+Eric+Chang%2C+and+Jingehan+Wang%2C+%E2%80%9CAutomatic+accent+identification+using+gaussian+mixture+models%2C%E2%80%9D+in+IEEE+Workshop+on+Automatic+Speech+Recognition+and+Understanding%2C+2001%2E+ASRU%E2%80%9901%2E)
4. Felix Weninger, Yang Sun, Junho Park, Daniel Willett, and Puming Zhan, “Deep learning based mandarin accent identification for accent robust asr.,” in INTERSPEECH, 2019, pp. 510–514. [find](https://scholar.google.com/scholar?q=Felix+Weninger%2C+Yang+Sun%2C+Junho+Park%2C+Daniel+Willett%2C+and+Puming+Zhan%2C+%E2%80%9CDeep+learning+based+mandarin+accent+identification+for+accent+robust+asr%2E%2C%E2%80%9D+in+INTERSPEECH%2C+2019%2C+pp%2E+510%E2%80%93514%2E)
5. Yishan Jiao, Ming Tu, Visar Berisha, and Julie M Liss, “Accent identification by combining deep neural networks and recurrent neural networks trained on long and short term features.,” in Interspeech, 2016, pp. 2388–2392. [find](https://scholar.google.com/scholar?q=Yishan+Jiao%2C+Ming+Tu%2C+Visar+Berisha%2C+and+Julie+M+Liss%2C+%E2%80%9CAccent+identification+by+combining+deep+neural+networks+and+recurrent+neural+networks+trained+on+long+and+short+term+features%2E%2C%E2%80%9D+in+Interspeech%2C)
6. Xian Shi, Fan Yu, Yizhou Lu, Qiangze Feng, Daliang Wang, Yanmin Qian, and Lei Xie, “[The accented english speech recognition challenge 2020: open datasets, tracks, baselines, results and methods](https://md.research.elamur.ai/free/en/papers/shi-2021-accented-english-speech-recognition-challenge/),” 2020. [↗](https://doi.org/10.48550/arxiv.2102.10233)
7. Lifa Sun, Hao Wang, Shiyin Kang, Kun Li, and Helen M Meng, “Personalized, cross-lingual tts using phonetic posteriorgrams,” in INTERSPEECH, 2016, pp. 322–326. [find](https://scholar.google.com/scholar?q=Lifa+Sun%2C+Hao+Wang%2C+Shiyin+Kang%2C+Kun+Li%2C+and+Helen+M+Meng%2C+%E2%80%9CPersonalized%2C+cross%2Dlingual+tts+using+phonetic+posteriorgrams%2C%E2%80%9D+in+INTERSPEECH%2C+2016%2C+pp%2E+322%E2%80%93326%2E)
8. Guanlong Zhao, Shaojin Ding, and Ricardo Gutierrez-Osuna, “Foreign accent conversion by synthesizing speech from phonetic posteriorgrams,” in INTERSPEECH, 2019, pp. 2843– 2847. [find](https://scholar.google.com/scholar?q=Guanlong+Zhao%2C+Shaojin+Ding%2C+and+Ricardo+Gutierrez%2DOsuna%2C+%E2%80%9CForeign+accent+conversion+by+synthesizing+speech+from+phonetic+posteriorgrams%2C%E2%80%9D+in+INTERSPEECH%2C+2019%2C+pp%2E+2843%E2%80%93+2847%2E)
9. David Snyder, Guoguo Chen, and Daniel Povey, “Musan: A music, speech, and noise corpus,” arXiv preprint arXiv:1510.08484, 2015. [↗](https://arxiv.org/abs/1510.08484)
10. Daniel Povey, Arnab Ghoshal, Gilles Boulianne, Lukas Burget, Ondrej Glembek, Nagendra Goel, Mirko Hannemann, Petr Motlicek, Yanmin Qian, Petr Schwarz, et al., “The kaldi speech recognition toolkit,” in IEEE 2011 workshop on automatic speech recognition and understanding. IEEE Signal Processing Society, 2011, number CONF. [find](https://scholar.google.com/scholar?q=Daniel+Povey%2C+Arnab+Ghoshal%2C+Gilles+Boulianne%2C+Lukas+Burget%2C+Ondrej+Glembek%2C+Nagendra+Goel%2C+Mirko+Hannemann%2C+Petr+Motlicek%2C+Yanmin+Qian%2C+Petr+Schwarz%2C+et+al%2E%2C+%E2%80%9CThe+kaldi+speech+recognition+toolkit%2C%E2%80%9D+i)
11. Guangzhi Sun, Yu Zhang, Ron J Weiss, Yuan Cao, Heiga Zen, Andrew Rosenberg, Bhuvana Ramabhadran, and Yonghui Wu, “Generating diverse and natural text-to-speech samples using a quantized fine-grained vae and autoregressive prosody prior,” in ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2020, pp. 6699–6703. [find](https://scholar.google.com/scholar?q=Guangzhi+Sun%2C+Yu+Zhang%2C+Ron+J+Weiss%2C+Yuan+Cao%2C+Heiga+Zen%2C+Andrew+Rosenberg%2C+Bhuvana+Ramabhadran%2C+and+Yonghui+Wu%2C+%E2%80%9CGenerating+diverse+and+natural+text%2Dto%2Dspeech+samples+using+a+quantized+fine%2Dgrained+v)
12. Yi Ren, Yangjun Ruan, Xu Tan, Tao Qin, Sheng Zhao, Zhou Zhao, and Tie-Yan Liu, “Fastspeech: Fast, robust and controllable text to speech,” in Advances in Neural Information Processing Systems, 2019, pp. 3171–3180. [↗](https://doi.org/10.48550/arxiv.1905.09263)
13. Jean-Marc Valin and Jan Skoglund, “Lpcnet: Improving neural speech synthesis through linear prediction,” in ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2019, pp. 5891–5895. [find](https://scholar.google.com/scholar?q=Jean%2DMarc+Valin+and+Jan+Skoglund%2C+%E2%80%9CLpcnet%3A+Improving+neural+speech+synthesis+through+linear+prediction%2C%E2%80%9D+in+ICASSP+2019%2D2019+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+Processing+%28I)
14. Shinji Watanabe, Takaaki Hori, Shigeki Karita, Tomoki Hayashi, Jiro Nishitoba, Yuya Unno, Nelson Enrique Yalta Soplin, Jahn Heymann, Matthew Wiesner, Nanxin Chen, et al., “Espnet: End-to-end speech processing toolkit,” arXiv preprint arXiv:1804.00015, 2018. [↗](https://arxiv.org/abs/1804.00015)
15. Jonathan Shen, Ruoming Pang, Ron J Weiss, Mike Schuster, Navdeep Jaitly, Zongheng Yang, Zhifeng Chen, Yu Zhang, Yuxuan Wang, Rj Skerrv-Ryan, et al., “Natural tts synthesis by conditioning wavenet on mel spectrogram predictions,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018, pp. 4779–4783. [find](https://scholar.google.com/scholar?q=Jonathan+Shen%2C+Ruoming+Pang%2C+Ron+J+Weiss%2C+Mike+Schuster%2C+Navdeep+Jaitly%2C+Zongheng+Yang%2C+Zhifeng+Chen%2C+Yu+Zhang%2C+Yuxuan+Wang%2C+Rj+Skerrv%2DRyan%2C+et+al%2E%2C+%E2%80%9CNatural+tts+synthesis+by+conditioning+wavenet+on+m)
16. David Snyder, Daniel Garcia-Romero, Gregory Sell, Daniel Povey, and Sanjeev Khudanpur, “X-vectors: Robust dnn embeddings for speaker recognition,” in 2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). IEEE, 2018, pp. 5329–5333. [find](https://scholar.google.com/scholar?q=David+Snyder%2C+Daniel+Garcia%2DRomero%2C+Gregory+Sell%2C+Daniel+Povey%2C+and+Sanjeev+Khudanpur%2C+%E2%80%9CX%2Dvectors%3A+Robust+dnn+embeddings+for+speaker+recognition%2C%E2%80%9D+in+2018+IEEE+International+Conference+on+Acoustics%2C+S)
17. Brecht Desplanques, Jenthe Thienpondt, and Kris Demuynck, “Ecapa-tdnn: Emphasized channel attention, propagation and aggregation in tdnn based speaker verification,” arXiv preprint arXiv:2005.07143, 2020. [↗](https://arxiv.org/abs/2005.07143)
18. Shanghua Gao, Ming-Ming Cheng, Kai Zhao, Xin-Yu Zhang, Ming-Hsuan Yang, and Philip HS Torr, “Res2net: A new multi-scale backbone architecture,” IEEE transactions on pattern analysis and machine intelligence, 2019. [↗](https://doi.org/10.1109/tpami.2019.2938758)
19. Jie Hu, Li Shen, and Gang Sun, “Squeeze-and-excitation networks,” in Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, pp. 7132–7141. [↗](https://doi.org/10.1109/tpami.2019.2913372)
20. Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou, “Arcface: Additive angular margin loss for deep face recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2019, pp. 4690–4699. [find](https://scholar.google.com/scholar?q=Jiankang+Deng%2C+Jia+Guo%2C+Niannan+Xue%2C+and+Stefanos+Zafeiriou%2C+%E2%80%9CArcface%3A+Additive+angular+margin+loss+for+deep+face+recognition%2C%E2%80%9D+in+Proceedings+of+the+IEEE+Conference+on+Computer+Vision+and+Pattern+Rec)
21. Xu Xiang, Shuai Wang, Houjun Huang, Yanmin Qian, and Kai Yu, “Margin matters: Towards more discriminative deep neural network embeddings for speaker recognition,” in 2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). IEEE, 2019, pp. 1652– 1656. [find](https://scholar.google.com/scholar?q=Xu+Xiang%2C+Shuai+Wang%2C+Houjun+Huang%2C+Yanmin+Qian%2C+and+Kai+Yu%2C+%E2%80%9CMargin+matters%3A+Towards+more+discriminative+deep+neural+network+embeddings+for+speaker+recognition%2C%E2%80%9D+in+2019+Asia%2DPacific+Signal+and+Infor)
22. Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton, “[Imagenet classification with deep convolutional neural networks](https://md.research.elamur.ai/free/en/papers/krizhevsky-2012-imagenet-classification-deep-convolutional/),” in Advances in neural information processing systems, 2012, pp. 1097–1105.
23. Karen Simonyan and Andrew Zisserman, “[Very deep convolutional networks for large-scale image recognition](https://md.research.elamur.ai/free/en/papers/simonyan-2014-very-deep-convolutional-networks-large/),” arXiv preprint arXiv:1409.1556, 2014. [↗](https://arxiv.org/abs/1409.1556)
24. Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al., “[Pytorch: An imperative style, high-performance deep learning library](https://md.research.elamur.ai/free/en/papers/paszke-2019-pytorch-imperative-style-high-performance/),” in Advances in neural information processing systems, 2019, pp. 8026–8037. [↗](https://doi.org/10.48550/arxiv.1912.01703)
25. Tan Tian, Lu Yizhou, Ma Rao, Zhu Sen, Guo Jiaqi, and Yanmin Qian, “Aispeech-sjtu asr system for the accented english speech recognition challenge,” IEEE, 2020. [find](https://scholar.google.com/scholar?q=Tan+Tian%2C+Lu+Yizhou%2C+Ma+Rao%2C+Zhu+Sen%2C+Guo+Jiaqi%2C+and+Yanmin+Qian%2C+%E2%80%9CAispeech%2Dsjtu+asr+system+for+the+accented+english+speech+recognition+challenge%2C%E2%80%9D+IEEE%2C+2020%2E)
26. G. E. Hinton, “Visualizing high-dimensional data using t-sne,” Journal of Machine Learning Research, vol. 9, no. 2, pp. 2579– 2605, 2008. [find](https://scholar.google.com/scholar?q=G%2E+E%2E+Hinton%2C+%E2%80%9CVisualizing+high%2Ddimensional+data+using+t%2Dsne%2C%E2%80%9D+Journal+of+Machine+Learning+Research%2C+vol%2E+9%2C+no%2E+2%2C+pp%2E+2579%E2%80%93+2605%2C+2008%2E)

*Updated 1 November 2026*