--- title: "Disfluency and Out-of-vocabulary Word Processing in Arabic" description: "The authors build a robust, partial-analysis pipeline for SARF, a voice server for Tunisian railway information. 79.23% precision. 11 references." authors: ["Younès Bahou", "Lamia Hadrich Belguith", "Abdelmajid Ben Hamadou"] year: 2009 venue: "Proceedings of the Third Workshop on Computational Approaches to Arabic-Script-based Languages (CAASL3)" source: "https://aclanthology.org/2009.mtsummit-caasl.3/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/bahou-2009-disfluency-out-vocabulary-word-processing/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bahou-2009-disfluency-out-vocabulary-word-processing/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2009](https://md.research.elamur.ai/free/en/papers/2009/page/2/) / Disfluency and Out-of-vocabulary Word Processing in Arabic Speech Understanding # Disfluency and Out-of-vocabulary Word Processing in Arabic Speech Understanding **Authors:** Younès Bahou, [Lamia Hadrich Belguith](https://md.research.elamur.ai/free/en/authors/lamia-hadrich-belguith/) and Abdelmajid Ben Hamadou · **Year:** 2009 · **Venue:** [Proceedings of the Third Workshop on Computational Approaches to Arabic-Script-based Languages (CAASL3)](https://md.research.elamur.ai/free/en/venues/mtsummit/) ([Machine Translation Summit (MT Summit) 2009](https://md.research.elamur.ai/free/en/venues/mtsummit/2009/)) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/2009.mtsummit-caasl.3/) · [PDF](https://aclanthology.org/2009.mtsummit-caasl.3.pdf) · [OpenAlex](https://openalex.org/W2098229396) **Citations:** 2 citations (OpenAlex, 2026-10-14) ## Summary The paper addresses spoken Modern Standard Arabic in a narrow-domain dialogue system. Automatic transcripts of spontaneous requests often contain hesitations, repeated or self-corrected phrases, and broken words. They also contain out-of-vocabulary (OOV) tokens: words the recognizer got wrong, words missing from the lexicon, or words only partly recognized. The authors build a robust, partial-analysis pipeline for SARF, a voice server for Tunisian railway information. The pipeline has three steps: - A semantically tagged utterance is cut into conceptual segments: illocutionary, referential and filler segments, plus a special Disfluent segment. - Disfluent segments are repaired with reparandum/editing-term/alteration patterns. - Words that do not fit the word class expected by their segment are flagged as OOV and reassigned or corrected. On a Wizard-of-Oz evaluation corpus, the module reduced semantic-frame filling failures from 18.54% to 12.63%. This shows that shallow, meaning-driven repair can make a speech understanding system much more tolerant of noisy input, even for an under-resourced language. ## Key points - In the study corpus, 25.24% of utterances contained disfluencies and 38.22% contained OOV words. - The evaluation corpus has 2823 spontaneous, automatically transcribed requests (34,726 words): 1291 of type A, 1434 of type D and 98 of type X. - Detection and correction rates: | | Correctly detected | Well corrected | |---|---|---| | OOV words | 78.04% | 87.36% | | Disfluencies | 61.29% | 81.07% | - Overall results were recall 79.23%, precision 74.09% and F-Measure 76.57%. Processing a 12-word utterance took about 0.394 s on average. - The module is implemented in Java (JBuilder 2007) and outputs semantic frames as XML for the dialogue manager. - Most errors trace back to wrong conceptual segmentation. Complex disfluencies that mix repetition and self-correction remain unhandled. ## Interesting ideas - A word can be in the lexicon and still be misrecognized: the system catches this when the word's class clashes with the type the slot expects (a month name where a destination city should be). Type expectations thus act as an error detector, not just a filter. - A repair is written as reparandum, optional editing term, interruption point and alteration, with each pair of words labelled as identical, same-role substitute, or neutral insertion. Only the alteration is kept, and the cleaned segment is then re-classified from scratch. - Suspected disfluencies are first quarantined in a dedicated Disfluent segment, so later stages handle them explicitly instead of letting them corrupt ordinary analysis. - A generic Number tag is refined into specific readings such as Fare, Hour or Minute using its position and neighbouring words. - When an OOV word sits equally close to two candidates, the system declines to guess and hands the ambiguity to the dialogue manager. Detection and correction are also scored separately. ## For Mova How these ideas could serve Mova, our small open language model — its files, stages and tests: - **I1** (frames): VerbNet selectional restrictions on roles could flag fillers whose type clashes with the role, marking a likely tagging or parse error instead of silently accepting it. - **I2** (tokens): a small explainable pattern layer could strip self-corrections such as "to Boston, uh, no, to Denver" from transcribed or chat-style English before tagging, and log which pattern fired. - **I3** (tree): material that is probably broken could be wrapped as a flagged span for the parser, giving traceable treatment rather than spurious dependency arcs. - **I4** (world): bare numerals could be typed into measures, values or times from local context before entering the world model, which helps maths word problems. - **I5** (answer): readers could abstain or name the competing candidates when evidence ties, and evaluation could report "found the problem" and "fixed it" as separate numbers. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works 🧭 Topics (1): [Semitic languages](https://md.research.elamur.ai/free/en/topics/semitic-languages/) · 1,485 works ## Cite Cite as: Younès Bahou, Lamia Hadrich Belguith and Abdelmajid Ben Hamadou. 2009. Disfluency and Out-of-vocabulary Word Processing in Arabic Speech Understanding. Proceedings of the Third Workshop on Computational Approaches to Arabic-Script-based Languages (CAASL3). The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{bahou2009disfluency, title = {{Disfluency and Out-of-vocabulary Word Processing in Arabic Speech Understanding}}, author = {Younès Bahou, Lamia Hadrich Belguith and Abdelmajid Ben Hamadou}, booktitle = {Proceedings of the Third Workshop on Computational Approaches to Arabic-Script-based Languages (CAASL3)}, year = {2009}, url = {https://aclanthology.org/2009.mtsummit-caasl.3/} } ``` - [← previous How to Establish a Verbal Paradigm on the Basis of Ancient Syriac Manuscripts](https://md.research.elamur.ai/free/en/papers/peursen-2009-how-establish-verbal-paradigm-basis/) - [next → Theorizing in the Absence of a Theory:The Case of the Aramaic Targums to the Pentateuch](https://md.research.elamur.ai/free/en/papers/lasair-2009-theorizing-absence-theory-case-aramaic/) - [↑ back to papers of 2009](https://md.research.elamur.ai/free/en/papers/2009/page/2/) ## References 11 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Bazzi I. and Glass J. 2001. Learning Units for Domain-Independent Out-Of-Vocabulary Words Modeling. European Conference on Speech Communication and Technology, pp. 61--64. Aalborg, Denmark. segment. Note that, for the example (9), our understanding module did not generate an error in filling semantic frame because it retains the last value of semantic cases. [find](https://scholar.google.com/scholar?q=Bazzi+I%2E+and+Glass+J%2E+2001%2E+Learning+Units+for+Domain%2DIndependent+Out%2DOf%2DVocabulary+Words+Modeling%2E+European+Conference+on+Speech+Communication+and+Technology%2C+pp%2E+61%2D%2D64%2E+Aalborg%2C+Denmark%2E+segment%2E+N) 2. Bear J., Dowding J. and Shriberg E. 1992. Integrating Multiple Knowledge Sources for Detection and Correction of Repairs in Human-Computer Dialog. In Proceedings of the 30th Annual Meeting of the Association for Computational Linguistics, Newark, Deleware, USA. [find](https://scholar.google.com/scholar?q=Bear+J%2E%2C+Dowding+J%2E+and+Shriberg+E%2E+1992%2E+Integrating+Multiple+Knowledge+Sources+for+Detection+and+Correction+of+Repairs+in+Human%2DComputer+Dialog%2E+In+Proceedings+of+the+30th+Annual+Meeting+of+the+Asso) 3. Bousquet-Vernhettes C. 2002. Compréhension Robuste de la Parole Spontanée dans le Dialogue Oral Homme-Machine – Décodage Conceptuel Stochastique. PHD Thesis, University of Toulouse III–Paul SABATIER, France. [find](https://scholar.google.com/scholar?q=Bousquet%2DVernhettes+C%2E+2002%2E+Compr%C3%A9hension+Robuste+de+la+Parole+Spontan%C3%A9e+dans+le+Dialogue+Oral+Homme%2DMachine+%E2%80%93+D%C3%A9codage+Conceptuel+Stochastique%2E+PHD+Thesis%2C+University+of+Toulouse+III%E2%80%93Paul+SABATIER%2C+) 4. Bove R. 2008. A Tagged Corpus-Based Study for Repeats and Self-Repairs Detection in French Transcribed Speech. Proceedings of the 11th International Conference on Text, Speech and Dialogue, pp. 269--276. Brno, Czech Republic. [find](https://scholar.google.com/scholar?q=Bove+R%2E+2008%2E+A+Tagged+Corpus%2DBased+Study+for+Repeats+and+Self%2DRepairs+Detection+in+French+Transcribed+Speech%2E+Proceedings+of+the+11th+International+Conference+on+Text%2C+Speech+and+Dialogue%2C+pp%2E+269%2D%2D2) 5. Bruce B. 1975. Cases Systems for Natural Languages. Artificial Intelligence, vol. 6, pp. 327—360. [find](https://scholar.google.com/scholar?q=Bruce+B%2E+1975%2E+Cases+Systems+for+Natural+Languages%2E+Artificial+Intelligence%2C+vol%2E+6%2C+pp%2E+327%E2%80%94360%2E) 6. Core M. and Schubert L. 1999. A Model of Speech Repairs and Other Disruptions. Presented at AAAI Fall Symposium on Psychological Models of Communication in Collaborative Systems, Cape Cod, MA, USA. [find](https://scholar.google.com/scholar?q=Core+M%2E+and+Schubert+L%2E+1999%2E+A+Model+of+Speech+Repairs+and+Other+Disruptions%2E+Presented+at+AAAI+Fall+Symposium+on+Psychological+Models+of+Communication+in+Collaborative+Systems%2C+Cape+Cod%2C+MA%2C+USA%2E) 7. Hazen T.J., Seneff S. and Polifroni J. 2002. Recognition Confidence Scoring and Its Use in Speech Understanding Systems. Computer Speech and Language, pp. 49--67. Academic Press, vol. 16, N° 1. [↗](https://doi.org/10.1006/csla.2001.0183) 8. Heeman P.A. and Allen J.F. 1996. Combining the Detection and Correction of Speech Repairs. In Proceedings of International Conference of Spoken Language Processing, pp. 363--365. Philadelphia, PA, USA. [find](https://scholar.google.com/scholar?q=Heeman+P%2EA%2E+and+Allen+J%2EF%2E+1996%2E+Combining+the+Detection+and+Correction+of+Speech+Repairs%2E+In+Proceedings+of+International+Conference+of+Spoken+Language+Processing%2C+pp%2E+363%2D%2D365%2E+Philadelphia%2C+PA%2C+USA) 9. Minker W. and Bennacef S. 2005. Speech and Human-Machine Dialog. Computational Linguistics. Vol. 31, pp. 157—158. [↗](https://doi.org/10.1162/0891201053630309) 10. Schaaf T. 2001. Detection of OOV Words Using Generalized Word Models and a Semantic Class Language Model. European Conference on Speech Communication and Technology, pp. 2581—2584. Aalborg, Denmark. [find](https://scholar.google.com/scholar?q=Schaaf+T%2E+2001%2E+Detection+of+OOV+Words+Using+Generalized+Word+Models+and+a+Semantic+Class+Language+Model%2E+European+Conference+on+Speech+Communication+and+Technology%2C+pp%2E+2581%E2%80%942584%2E+Aalborg%2C+Denmark%2E) 11. Zouaghi A., Zrigui M. and Antoniadis, G. 2008. Compréhension Automatique de la Parole Arabe Spontanée : Une Modélisation Numérique. TAL, vol. 49, N°1, pp. 141—166. [find](https://scholar.google.com/scholar?q=Zouaghi+A%2E%2C+Zrigui+M%2E+and+Antoniadis%2C+G%2E+2008%2E+Compr%C3%A9hension+Automatique+de+la+Parole+Arabe+Spontan%C3%A9e+%3A+Une+Mod%C3%A9lisation+Num%C3%A9rique%2E+TAL%2C+vol%2E+49%2C+N%C2%B01%2C+pp%2E+141%E2%80%94166%2E) *Updated 24 October 2026*