---
title: "Adaptive Multilingual Sentence Boundary Disambiguation (1997)"
description: "Palmer and Hearst introduce Satz, a trainable disambiguator. 6.3% error rate. 34 references, cited by 37 papers here."
authors: ["David D. Palmer", "Marti A. Hearst"]
year: 1997
venue: "Computational Linguistics"
source: "https://aclanthology.org/J97-2002/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/palmer-1997-adaptive-multilingual-sentence-boundary-disambiguation/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/palmer-1997-adaptive-multilingual-sentence-boundary-disambiguation/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [1997](https://md.research.elamur.ai/free/en/papers/1997/) / Adaptive Multilingual Sentence Boundary Disambiguation

# Adaptive Multilingual Sentence Boundary Disambiguation

**Authors:** [David D. Palmer](https://md.research.elamur.ai/free/en/authors/david-d-palmer/) and [Marti A. Hearst](https://md.research.elamur.ai/free/en/authors/marti-a-hearst/) · **Year:** 1997 · **Venue:** [Computational Linguistics](https://md.research.elamur.ai/free/en/venues/cl/) ([Computational Linguistics 1997](https://md.research.elamur.ai/free/en/venues/cl/1997/)) · **Publisher:** [The MIT Press](https://md.research.elamur.ai/free/en/publishers/the-mit-press/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/J97-2002/) · [PDF](https://aclanthology.org/J97-2002.pdf) · [OpenAlex](https://openalex.org/W1490831147)

**Citations:** 150 citations (OpenAlex, 2026-10-14) · [37](https://md.research.elamur.ai/free/en/papers/palmer-1997-adaptive-multilingual-sentence-boundary-disambiguation/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Splitting text into sentences is a prerequisite for taggers, parsers and sentence aligners. However, periods, question marks and exclamation points are ambiguous: a period can end a sentence, mark an abbreviation or decimal, or do both at once. Rule-based tokenizers built from regular expressions and abbreviation lists can work well, but they need months of manual tuning per corpus. They also fail on all-caps or OCR-damaged text.

Palmer and Hearst introduce Satz, a trainable disambiguator. For each candidate mark, it represents the neighbouring tokens by rough part-of-speech estimates from a small lexicon instead of brittle cues like capitalization. A learned classifier then labels the mark as a boundary or not. Satz trains in under a minute and segments about 10,000 sentences per minute. Its error rate stayed below 1.5% across corpora, and it was ported easily from English to French and German. It offers a cheap, portable alternative to hand-written rules and can also repair difficult cases left by a rule-based system.

## Key points
- Satz needs only a lexicon that can stay under 5,000 words and a training corpus of roughly 300–500 sentences.
- The "lower bound" baseline labels every candidate mark as a sentence end, and its score depends heavily on genre. It is about 53% for the Wall Street Journal corpus, where 47% of periods are abbreviations, versus about 90% for the Brown corpus.
- The UNIX STYLE program, which relies on a 48-entry abbreviation list, served as a comparison baseline. Its developers reported a 6.3% error rate (204 errors out of 3,287 candidate marks).
- Hand-built rule systems required heavy effort:
  - Alembic used 75 abbreviations and over 100 rules developed over more than six staff months, reaching 0.9% error on WSJ.
  - Wasson's finite-automaton recognizer took nine staff months to build.
- The system was evaluated on English, French and German, with error consistently below 1.5%.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Palo Alto Research Center](https://md.research.elamur.ai/free/en/labs/palo-alto-research-center/) — Research institute, Palo Alto, United States · 14 works · 8,123 citations
- [Mitre](https://md.research.elamur.ai/free/en/companies/mitre/) — Company, Bedford, United States · 10 works · 2,631 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🌍 Multilingual & cross-lingual NLP](https://md.research.elamur.ai/free/en/topics/multilingual-nlp/) · 828 works · [✂️ Tokenization & segmentation](https://md.research.elamur.ai/free/en/topics/tokenization/) · 119 works

## Cite

Cite as: David D. Palmer and Marti A. Hearst. 1997. Adaptive Multilingual Sentence Boundary Disambiguation. Computational Linguistics.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{palmer1997adaptive,
  title = {{Adaptive Multilingual Sentence Boundary Disambiguation}},
  author = {David D. Palmer and Marti A. Hearst},
  journal = {Computational Linguistics},
  year = {1997},
  doi = {10.5555/972695.972697},
  url = {https://doi.org/10.5555/972695.972697}
}
```

- [← previous IGTree: Using Trees for Compression and Classification in Lazy Learning Algorithms](https://md.research.elamur.ai/free/en/papers/daelemans-1997-igtree-trees-compression-classification-lazy/)
- [next → A Perspective on Word Sense Disambiguation Methods and Their Evaluation](https://md.research.elamur.ai/free/en/papers/resnik-1997-perspective-word-sense-disambiguation-methods/)
- [↑ back to papers of 1997](https://md.research.elamur.ai/free/en/papers/1997/)

## References

34 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Aberdeen, John, John Burger, David Day, Lynette Hirschman, Patricia Robinson, and Marc Vilain. 1995. MITRE: Description of the Alembic system used for MUC-6. In Proceedings of the Sixth Message Understanding Conference (MUC-6), Columbia, MD, November. [find](https://scholar.google.com/scholar?q=Aberdeen%2C+John%2C+John+Burger%2C+David+Day%2C+Lynette+Hirschman%2C+Patricia+Robinson%2C+and+Marc+Vilain%2E+1995%2E+MITRE%3A+Description+of+the+Alembic+system+used+for+MUC%2D6%2E+In+Proceedings+of+the+Sixth+Message+Unders)
2. Bahl, L. R., P. E Brown, P. V. deSouza, and R. L. Mercer. 1989. A tree-based statistical language model for natural language speech recognition. IEEE Transactions on Acoustics, Speech, and Signal Processing, 36(7):1001-1008. [find](https://scholar.google.com/scholar?q=Bahl%2C+L%2E+R%2E%2C+P%2E+E+Brown%2C+P%2E+V%2E+deSouza%2C+and+R%2E+L%2E+Mercer%2E+1989%2E+A+tree%2Dbased+statistical+language+model+for+natural+language+speech+recognition%2E+IEEE+Transactions+on+Acoustics%2C+Speech%2C+and+Signal+Proc)
3. Bourland, Herv~ and Nelson Morgan. 1994. Connectionist Speech Recognition: A Hybrid Approach. Kluwer Academic Publishers, Norwell, MA. [find](https://scholar.google.com/scholar?q=Bourland%2C+Herv%7E+and+Nelson+Morgan%2E+1994%2E+Connectionist+Speech+Recognition%3A+A+Hybrid+Approach%2E+Kluwer+Academic+Publishers%2C+Norwell%2C+MA%2E)
4. Breiman, Leo, Jerome H. Friedman, Richard Olshen, and Charles J. Stone. 1984. Classification and Regression Trees. Wadsworth International Group, Belmont, CA. [find](https://scholar.google.com/scholar?q=Breiman%2C+Leo%2C+Jerome+H%2E+Friedman%2C+Richard+Olshen%2C+and+Charles+J%2E+Stone%2E+1984%2E+Classification+and+Regression+Trees%2E+Wadsworth+International+Group%2C+Belmont%2C+CA%2E)
5. Cherry, L. L. and W. Vesterman. 1991. Writing tools--the STYLE and DICTION programs. In 4.3BSD UNIX System Documentation. University of California, Berkeley. [find](https://scholar.google.com/scholar?q=Cherry%2C+L%2E+L%2E+and+W%2E+Vesterman%2E+1991%2E+Writing+tools%2D%2Dthe+STYLE+and+DICTION+programs%2E+In+4%2E3BSD+UNIX+System+Documentation%2E+University+of+California%2C+Berkeley%2E)
6. Church, Kenneth W. 1988. [A stochastic parts program and noun phrase parser for unrestricted text](https://md.research.elamur.ai/free/en/papers/church-1988-stochastic-parts-program-noun-phrase/). In Second Conference on Applied Natural Language Processing, pages 136-143, Austin, TX.
7. Church, Kenneth W. and Mark Y. Liberman. 1991. A status report on the ACL/DCI. In Proceedings of the 7th Annual Conference of the UW Centre for the New OED and Text Research: Using Corpora, pages 84-91, Oxford. [find](https://scholar.google.com/scholar?q=Church%2C+Kenneth+W%2E+and+Mark+Y%2E+Liberman%2E+1991%2E+A+status+report+on+the+ACL%2FDCI%2E+In+Proceedings+of+the+7th+Annual+Conference+of+the+UW+Centre+for+the+New+OED+and+Text+Research%3A+Using+Corpora%2C+pages+84%2D9)
8. Cutting, Doug, Julian Kupiec, Jan Pedersen, and Penelope Sibun. 1991. [A practical part-of-speech tagger](https://md.research.elamur.ai/free/en/papers/cutting-1992-practical-part-speech-tagger/). In The 3rd Conference on Applied Natural Language Processing, Trento, Italy.
9. Francis, W. Nelson and Henry Kucera. 1982. Frequency Analysis of English Usage. Houghton Mifflin Co., New York. [find](https://scholar.google.com/scholar?q=Francis%2C+W%2E+Nelson+and+Henry+Kucera%2E+1982%2E+Frequency+Analysis+of+English+Usage%2E+Houghton+Mifflin+Co%2E%2C+New+York%2E)
10. Fung, Pascale and Kenneth Ward Church. 1994. K-vec: A new approach to aligning parallel texts. In Proceedings of COLING-94. [find](https://scholar.google.com/scholar?q=Fung%2C+Pascale+and+Kenneth+Ward+Church%2E+1994%2E+K%2Dvec%3A+A+new+approach+to+aligning+parallel+texts%2E+In+Proceedings+of+COLING%2D94%2E)
11. Fung, Pascale and Kathleen McKeown. 1994. Aligning noisy parallel corpora across language groups: Word pair feature matching by dynamic time warping. In Proceedings of the First Conference of the Association for Machine Translation in the Americas AMTA-94, pages 81-88, Columbia, MD. [find](https://scholar.google.com/scholar?q=Fung%2C+Pascale+and+Kathleen+McKeown%2E+1994%2E+Aligning+noisy+parallel+corpora+across+language+groups%3A+Word+pair+feature+matching+by+dynamic+time+warping%2E+In+Proceedings+of+the+First+Conference+of+the+Asso)
12. Gale, William A. and Kenneth W. Church. 1993. [A program for aligning sentences in bilingual corpora](https://md.research.elamur.ai/free/en/papers/gale-1993-program-aligning-sentences-bilingual-corpora/). Computational Linguistics, 19(1):75-102.
13. Hertz, John, Anders Krogh, and Richard G. Palmer. 1991. Introduction to the Theory of Neural Computation. Santa Fe Institute Studies in the Sciences of Complexity. Addison-Wesley Publishing Co., Redwood City, CA. [find](https://scholar.google.com/scholar?q=Hertz%2C+John%2C+Anders+Krogh%2C+and+Richard+G%2E+Palmer%2E+1991%2E+Introduction+to+the+Theory+of+Neural+Computation%2E+Santa+Fe+Institute+Studies+in+the+Sciences+of+Complexity%2E+Addison%2DWesley+Publishing+Co%2E%2C+Redwo)
14. Hoffmann, Christiane. 1994. Automatische Disambiguierung von Satzgrenzen in einem maschinenlesbaren deutschen Korpus. Manuscript, University of Trier, Germany. [find](https://scholar.google.com/scholar?q=Hoffmann%2C+Christiane%2E+1994%2E+Automatische+Disambiguierung+von+Satzgrenzen+in+einem+maschinenlesbaren+deutschen+Korpus%2E+Manuscript%2C+University+of+Trier%2C+Germany%2E)
15. Humphrey, T. L. and E-q. Zhou. 1989. Period disambiguation using a neural network. In IJCNN: International Joint Conference on Neural Networks, page 606, Washington, D.C. [find](https://scholar.google.com/scholar?q=Humphrey%2C+T%2E+L%2E+and+E%2Dq%2E+Zhou%2E+1989%2E+Period+disambiguation+using+a+neural+network%2E+In+IJCNN%3A+International+Joint+Conference+on+Neural+Networks%2C+page+606%2C+Washington%2C+D%2EC%2E)
16. Kay, Martin and Martin ROscheisen. 1993. [Text-translation alignment](https://md.research.elamur.ai/free/en/papers/kay-1993-text-translation-alignment/). Computational Linguistics, 19(1):121-142.
17. Lesk, M. E. and E. Schmidt. 1975. Lex--a lexical analyzer generator. Computing Science Technical Report 39, AT&T Bell Laboratories, Murray Hill, NJ. [find](https://scholar.google.com/scholar?q=Lesk%2C+M%2E+E%2E+and+E%2E+Schmidt%2E+1975%2E+Lex%2D%2Da+lexical+analyzer+generator%2E+Computing+Science+Technical+Report+39%2C+AT%26T+Bell+Laboratories%2C+Murray+Hill%2C+NJ%2E)
18. Liberman, Mark Y. and Kenneth W. Church. 1992. Text analysis and word pronunciation in text-to-speech synthesis. In Sadaoki Furui and Man Mohan Sondhi, editors, Advances in Speech Signal Processing. Marcel Dekker, Inc., pages 791-831. [find](https://scholar.google.com/scholar?q=Liberman%2C+Mark+Y%2E+and+Kenneth+W%2E+Church%2E+1992%2E+Text+analysis+and+word+pronunciation+in+text%2Dto%2Dspeech+synthesis%2E+In+Sadaoki+Furui+and+Man+Mohan+Sondhi%2C+editors%2C+Advances+in+Speech+Signal+Processing%2E+M)
19. Lippmann, R. P. 1989. Review of neural networks for speech recognition. Neural Computation, 1:1-38. [find](https://scholar.google.com/scholar?q=Lippmann%2C+R%2E+P%2E+1989%2E+Review+of+neural+networks+for+speech+recognition%2E+Neural+Computation%2C+1%3A1%2D38%2E)
20. Magerman, David M. 1995. [Statistical decision-tree models for parsing](https://md.research.elamur.ai/free/en/papers/magerman-1995-statistical-decision-tree-models-parsing/). In Proceedings of the 33rd Annual Meeting. Association for Computational Linguistics.
21. Mtiller, Hans, V. Amerl, and G. Natalis. 1980. Worterkennungsverfahren als Grundlage einer Universalmethode zur automatischen Segmentierung von Texten in S~itze. Ein Verfahren zur maschinellen Satzgrenzenbestimmung im Englischen. Sprache und Datenverarbeitung, 1. [find](https://scholar.google.com/scholar?q=Mtiller%2C+Hans%2C+V%2E+Amerl%2C+and+G%2E+Natalis%2E+1980%2E+Worterkennungsverfahren+als+Grundlage+einer+Universalmethode+zur+automatischen+Segmentierung+von+Texten+in+S%7Eitze%2E+Ein+Verfahren+zur+maschinellen+Satzgre)
22. Nakamura, Masami, Katsuteru Maruyama, Takeshi Kawabata, and Kiyohiro Shikano. 1990. [Neural network approach to word category prediction for English texts](https://md.research.elamur.ai/free/en/papers/nakamura-1990-neural-network-approach-word-category/). In Proceedings of COLING-90, volume 3, pages 213-218.
23. Nicol, G. T. 1993. Flex--The Lexical Scanner Generator. The Free Software Foundation, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Nicol%2C+G%2E+T%2E+1993%2E+Flex%2D%2DThe+Lexical+Scanner+Generator%2E+The+Free+Software+Foundation%2C+Cambridge%2C+MA%2E)
24. Nunberg, Geoffrey. 1990. The Linguistics of Punctuation. C.S.L.I. Lecture Notes, Number 18. Center for the Study of Language and Information, Stanford, CA. [find](https://scholar.google.com/scholar?q=Nunberg%2C+Geoffrey%2E+1990%2E+The+Linguistics+of+Punctuation%2E+C%2ES%2EL%2EI%2E+Lecture+Notes%2C+Number+18%2E+Center+for+the+Study+of+Language+and+Information%2C+Stanford%2C+CA%2E)
25. Palmer, David D. 1995. Experiments in multilingual sentence boundary recognition. In Proceedings of Recent Advances in Natural Language Processing, Velingrad, Bulgaria, September 1995. [find](https://scholar.google.com/scholar?q=Palmer%2C+David+D%2E+1995%2E+Experiments+in+multilingual+sentence+boundary+recognition%2E+In+Proceedings+of+Recent+Advances+in+Natural+Language+Processing%2C+Velingrad%2C+Bulgaria%2C+September+1995%2E)
26. Palmer, David D. and Marti A. Hearst. 1995. [Adaptive sentence boundary disambiguation](https://md.research.elamur.ai/free/en/papers/palmer-1994-adaptive-sentence-boundary-disambiguation/). In Proceedings of the Fourth ACL Conference on Applied Natural Language Processing, pages 78-83.
27. Morgan Kaufmann, October 1994. [find](https://scholar.google.com/scholar?q=Morgan+Kaufmann%2C+October+1994%2E)
28. Quinlan, J. Ross. 1986. Induction of decision trees. Machine Learning, 1(1):81-106. [find](https://scholar.google.com/scholar?q=Quinlan%2C+J%2E+Ross%2E+1986%2E+Induction+of+decision+trees%2E+Machine+Learning%2C+1%281%29%3A81%2D106%2E)
29. Quinlan, J. Ross. 1993. [c4.5: Programs for Machine Learning](https://md.research.elamur.ai/free/en/papers/quinlan-1993-c45-programs-machine-learning/). Morgan Kaufman, San Mateo, CA.
30. Resnik, Philip. 1993. Semantic classes and syntactic ambiguity. In Proceedings of the ARPA Workshop on Human Language Technology (March 1993). [find](https://scholar.google.com/scholar?q=Resnik%2C+Philip%2E+1993%2E+Semantic+classes+and+syntactic+ambiguity%2E+In+Proceedings+of+the+ARPA+Workshop+on+Human+Language+Technology+%28March+1993%29%2E)
31. Riley, Michael D. 1989. Some applications of tree-based modelling to speech and language indexing. In Proceedings of the DARPA Speech and Natural Language Workshop, pages 339-352. Morgan Kaufmann. [find](https://scholar.google.com/scholar?q=Riley%2C+Michael+D%2E+1989%2E+Some+applications+of+tree%2Dbased+modelling+to+speech+and+language+indexing%2E+In+Proceedings+of+the+DARPA+Speech+and+Natural+Language+Workshop%2C+pages+339%2D352%2E+Morgan+Kaufmann%2E)
32. Schmid, Helmut. 1994. Part-of-speech tagging with neural networks. In Proceedings of COLING-94. [find](https://scholar.google.com/scholar?q=Schmid%2C+Helmut%2E+1994%2E+Part%2Dof%2Dspeech+tagging+with+neural+networks%2E+In+Proceedings+of+COLING%2D94%2E)
33. Siegel, Eric V. and Kathleen R. McKeown. 1994. Emergent linguistic rules from inducing decision trees: Disambiguating discourse clue words. In Proceedings of AAA11994, pages 820-826. AAAI Press/MIT Press, July. [find](https://scholar.google.com/scholar?q=Siegel%2C+Eric+V%2E+and+Kathleen+R%2E+McKeown%2E+1994%2E+Emergent+linguistic+rules+from+inducing+decision+trees%3A+Disambiguating+discourse+clue+words%2E+In+Proceedings+of+AAA11994%2C+pages+820%2D826%2E+AAAI+Press%2FMIT+Pr)
34. Soderland, Stephen and Wendy Lehnert. 1994. Corpus-driven knowledge acquisition for discourse analysis. In Proceedings of AAA11994, pages 827-832. [find](https://scholar.google.com/scholar?q=Soderland%2C+Stephen+and+Wendy+Lehnert%2E+1994%2E+Corpus%2Ddriven+knowledge+acquisition+for+discourse+analysis%2E+In+Proceedings+of+AAA11994%2C+pages+827%2D832%2E)

*Updated 30 October 2026*