---
title: "A TAG-based noisy-channel model of speech repairs (2004)"
description: "The paper tackles speech repairs in spontaneous speech transcripts. 8 references, cited by 76 papers here."
authors: ["Mark Johnson", "Eugene Charniak"]
year: 2004
venue: "Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)"
source: "https://aclanthology.org/P04-1005/"
doi: "10.3115/1218955.1218960"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/johnson-2004-tag-based-noisy-channel-model/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/johnson-2004-tag-based-noisy-channel-model/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2004](https://md.research.elamur.ai/free/en/papers/2004/) / A TAG-based noisy-channel model of speech repairs

# A TAG-based noisy-channel model of speech repairs

**Authors:** [Mark Johnson](https://md.research.elamur.ai/free/en/authors/mark-johnson/) and [Eugene Charniak](https://md.research.elamur.ai/free/en/authors/eugene-charniak/) · **Year:** 2004 · **Venue:** [Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2004](https://md.research.elamur.ai/free/en/venues/acl/2004/)) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/P04-1005/) · [PDF](https://aclanthology.org/P04-1005.pdf) · [DOI 10.3115/1218955.1218960](https://doi.org/10.3115/1218955.1218960) · [OpenAlex](https://openalex.org/W2169058332)

**Citations:** 128 citations (OpenAlex, 2026-10-14) · [76](https://md.research.elamur.ai/free/en/papers/johnson-2004-tag-based-noisy-channel-model/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper tackles speech repairs in spontaneous speech transcripts. In a repair, a speaker abandons a stretch of words (the reparandum), optionally inserts a filler or editing phrase (the interregnum), and then produces a corrected version (the repair). Johnson and Charniak treat repair removal as noisy-channel decoding. A language model generates a clean, repair-free sentence, and a channel model adds repairs to it. The channel model is a stochastic transducer built on Tree Adjoining Grammar (TAG). TAG was chosen because the repair usually echoes the reparandum word by word, which creates crossed dependencies that context-free or finite-state models cannot express.

Search uses a bigram language model to produce 20 candidate analyses. These candidates are then rescored with the syntactic parser-based language model of Charniak (2001). Training and testing use the disfluency-annotated Switchboard section of Penn Treebank III, with partial words and punctuation removed. With parser rescoring, word-level reparandum detection reaches an F-score of 0.797. This beats the bigram and trigram variants and the earlier Charniak and Johnson (2001) classifier, showing that syntax-aware language models help disfluency detection.

## Key points
- **Data:**
  - Training used the disfluency-tagged Switchboard files (dysfl/dps/swbd) on the LDC Penn Treebank III CD-ROM.
  - The split follows Charniak and Johnson (2001): sw[23]*.dps for main training, sw4[5-9]*.dps as held-out data, and sw4[0-1]*.mrg from the parsed version for testing.
- **Copying evidence:** In the training data, more than 60% of the words in a reparandum reappear identically in the corresponding repair. The channel model is designed to assign high probability to such copies.
- **Search pipeline:**
  - TAG dynamic programming with a bigram language model finds the 20 best analyses.
  - The parser-based language model then reranks them.
  - Because parsing costs n^5, analysis is restricted to substrings of at most 12 words.
- **Results (precision / recall / F-score):**

  | Model | Precision | Recall | F-score |
  |---|---|---|---|
  | CJ01 classifier | 0.951 | 0.631 | 0.759 |
  | Bigram | 0.776 | 0.736 | 0.756 |
  | Trigram | 0.774 | 0.763 | 0.768 |
  | Parser | 0.820 | 0.778 | 0.797 |

- **Negative result:** Replacing word bigram dependencies with POS bi-tag dependencies gave nearly identical results.

## Interesting ideas
- **Cheap search, rich rescoring:** A finite-state language model keeps the combined grammar tractable for dynamic programming, and a stronger global model is applied only to a short n-best list. The same split works whenever a good scorer cannot be intersected with the search space.
- **Transducer as a grammar over symbol pairs:** Each terminal is an observed:source pair, with an empty symbol marking deletions. Any grammar formalism then becomes a transducer whose two sides are read off by projection, a simple trick for alignment-style tasks.
- **Mildly context-sensitive formalisms for "rough copy" phenomena:** Using TAG to model a second string that echoes the first with crossed links is a principled option for other near-duplicate structures. The authors draw a parallel to genome modelling.
- **Local repair odds instead of one global parse:** Each bounded substring gets a score: the probability of generating it as a repair versus as non-repair material. The scored substrings are merged afterwards. This is faster, and it can represent a repair that is itself repaired again, which a single global analysis cannot.
- **Realistic evaluation and complementary systems:** - Removing punctuation and partial words imitates speech-recognizer output, a useful habit for any speech-text pipeline. - The authors observe that windowed classifiers handle short repairs well while the generative model handles long, similar ones. This suggests combining the two approaches.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Brown University](https://md.research.elamur.ai/free/en/universities/brown-university/) — University, Providence, United States · 58 works · 25,310 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: Mark Johnson and Eugene Charniak. 2004. A TAG-based noisy-channel model of speech repairs. Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{johnson2004tag,
  title = {{A TAG-based noisy-channel model of speech repairs}},
  author = {Mark Johnson and Eugene Charniak},
  booktitle = {Proceedings of the 42nd Annual Meeting of the Association for Computational Linguistics (ACL-04)},
  year = {2004},
  doi = {10.3115/1218955.1218960},
  url = {https://doi.org/10.3115/1218955.1218960}
}
```

- [← previous The Importance of Supertagging for Wide-Coverage CCG Parsing](https://md.research.elamur.ai/free/en/papers/clark-2004-importance-supertagging-wide-coverage-ccg/)
- [next → LinGO Redwoods: A Rich and Dynamic Treebank for HPSG](https://md.research.elamur.ai/free/en/papers/oepen-2004-lingo-redwoods-rich-dynamic-treebank/)
- [↑ back to papers of 2004](https://md.research.elamur.ai/free/en/papers/2004/)

## References

8 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Eugene Charniak and Mark Johnson. 2001. [Edit detection and parsing for transcribed speech](https://md.research.elamur.ai/free/en/papers/charniak-2001-edit-detection-parsing-transcribed-speech/). In Proceedings of the 2nd Meeting of the North American Chapter of the Association for Computational Linguistics, pages 118–126. The Association for Computational Linguistics.
2. Eugene Charniak. 2001. [Immediate-head parsing for language models](https://md.research.elamur.ai/free/en/papers/charniak-2001-immediate-head-parsing-language-models/). In Proceedings of the 39th Annual Meeting of the Association for Computational Linguistics. The Association for Computational Linguistics.
3. Stanley F. Chen and Joshua Goodman. 1998. [An empirical study of smoothing techniques for language modeling](https://md.research.elamur.ai/free/en/papers/chen-1996-empirical-study-smoothing-techniques-language/). Technical Report TR-10-98, Center for Research in Computing Technology, Harvard University.
4. Peter A. Heeman and James F. Allen. 1999. Speech repairs, intonational phrases, and discourse markers: Modeling speaker’s utterances in spoken dialogue. Computational Linguistics, 25(4):527–571. [find](https://scholar.google.com/scholar?q=Peter+A%2E+Heeman+and+James+F%2E+Allen%2E+1999%2E+Speech+repairs%2C+intonational+phrases%2C+and+discourse+markers%3A+Modeling+speaker%E2%80%99s+utterances+in+spoken+dialogue%2E+Computational+Linguistics%2C+25%284%29%3A527%E2%80%93571%2E)
5. Stuart M. Shieber and Yves Schabes. 1990. [Synchronous tree-adjoining grammars](https://md.research.elamur.ai/free/en/papers/shieber-1990-synchronous-tree-adjoining-grammars/). In Proceedings of the 13th International Conference on Computational Linguistics (COLING 1990), pages 253–258.
6. Stuart M. Shieber. 1985. Evidence against the Context-Freeness of natural language. Linguistics and Philosophy, 8(3):333–344. [find](https://scholar.google.com/scholar?q=Stuart+M%2E+Shieber%2E+1985%2E+Evidence+against+the+Context%2DFreeness+of+natural+language%2E+Linguistics+and+Philosophy%2C+8%283%29%3A333%E2%80%93344%2E)
7. Elizabeth Shriberg and Andreas Stolcke. 1998. How far do speakers back up in repairs? a quantitative model. In Proceedings of the International Conference on Spoken Language Processing, volume 5, pages 2183–2186, Sydney, Australia. [find](https://scholar.google.com/scholar?q=Elizabeth+Shriberg+and+Andreas+Stolcke%2E+1998%2E+How+far+do+speakers+back+up+in+repairs%3F+a+quantitative+model%2E+In+Proceedings+of+the+International+Conference+on+Spoken+Language+Processing%2C+volume+5%2C+page)
8. Elizabeth Shriberg. 1994. Preliminaries to a Theory of Speech Disfluencies. Ph.D. thesis, University of California, Berkeley. [find](https://scholar.google.com/scholar?q=Elizabeth+Shriberg%2E+1994%2E+Preliminaries+to+a+Theory+of+Speech+Disfluencies%2E+Ph%2ED%2E+thesis%2C+University+of+California%2C+Berkeley%2E)

*Updated 24 October 2026*