---
title: "Toward Semantics-Based Answer Pinpointing — Hovy 2001"
description: "The paper tackles a weakness in factoid question answering (QA)."
authors: ["Eduard Hovy", "Laurie Gerber", "Ulf Hermjakob", "Chin-Yew Lin", "Deepak Ravichandran"]
year: 2001
venue: "Proceedings of the First International Conference on Human Language Technology Research"
source: "https://aclanthology.org/H01-1069/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hovy-2001-semantics-based-answer-pinpointing/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hovy-2001-semantics-based-answer-pinpointing/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2001](https://md.research.elamur.ai/free/en/papers/2001/) / Toward Semantics-Based Answer Pinpointing

# Toward Semantics-Based Answer Pinpointing

**Authors:** [Eduard Hovy](https://md.research.elamur.ai/free/en/authors/eduard-hovy/), Laurie Gerber, [Ulf Hermjakob](https://md.research.elamur.ai/free/en/authors/ulf-hermjakob/), [Chin-Yew Lin](https://md.research.elamur.ai/free/en/authors/chin-yew-lin/), [Deepak Ravichandran](https://md.research.elamur.ai/free/en/authors/deepak-ravichandran/) · **Year:** 2001 · **Venue:** Proceedings of the First International Conference on Human Language Technology Research · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/H01-1069/) · [PDF](https://aclanthology.org/H01-1069.pdf) · [OpenAlex](https://openalex.org/W1970409510)

**Citations:** 173 citations (openalex-snapshot · 2026-10-14) · [90](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
The paper tackles a weakness in factoid question answering (QA). Most systems pick the answer by sliding a word-scoring window over retrieved text. That approach cannot mark exact answer boundaries, cannot recognize the semantic type of an answer such as a person or a place, and fails when answer pieces are spread across a sentence. Within ISI's Webclopedia system, the authors keep the window method as a fallback and add two semantics-based layers. The first is a QA Typology, a hierarchy of question intents. The second is a matcher that compares CONTEX parse trees of the question and of candidate sentences, using the expected answer type (Qtarget) and question/answer templates. The hand-built templates were precise but fired too rarely to help much. The Qtarget, however, proved useful. Retraining CONTEX on treebanked questions made question parsing more accurate. The work matters because it moves answer extraction from surface word overlap toward structured semantic matching. It also motivates learning QA patterns automatically with a Noisy Channel Model.

## Key points
- The QA Typology was built from 17,384 question–answer pairs collected from answers.com. It has 72 nodes organized by user intent, such as Why-Famous, Proper-Person and Abbreviation-Expansion, rather than by question word.
- For TREC-9, the team created about 500 answer patterns by crossing roughly 20 question templates with roughly 25 answer templates. These patterns were accurate when they matched but had low coverage.
- CONTEX, trained on 2048 Penn Treebank sentences, reached 87.6% labeled precision and 88.4% labeled recall on English. Question training data grew from 250 to 400 to 975 items, drawn from 1153 questions treebanked at ISI, and was evaluated with five-fold cross-validation on TREC-8/9 questions. The provided excerpt omits the Table 1 figures.
- The pipeline uses BBN IdentiFinder and CONTEX for question analysis and WordNet-based query expansion. MG retrieves the top 1000 documents, which are segmented with TexTiling (C99 was also tried), and CONTEX parses the top 100 segments.
- Window-based pinpointing reaches up to about 30% in independent evaluations. This ceiling motivates the semantic approach.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

90 papers of our library cite it (of the 140884 whose bibliographies we read); 4 of them have a page here, 86 not yet.

- [Learning Question Classifiers](https://md.research.elamur.ai/free/en/papers/li-2002-learning-question-classifiers/)
- [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)
- [Aya Model: An Instruction Finetuned Open-Access Multilingual Language Model](https://md.research.elamur.ai/free/en/papers/ustun-2024-aya-model-instruction-finetuned-open/)
- [Label Words are Anchors: An Information Flow Perspective for Understanding In-Context Learning](https://md.research.elamur.ai/free/en/papers/wang-2023-label-words-anchors-information-flow/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{hovy2001toward,
  title = {{Toward Semantics-Based Answer Pinpointing}},
  author = {Hovy, Eduard and Gerber, Laurie and Hermjakob, Ulf and Lin, Chin-Yew and Ravichandran, Deepak},
  booktitle = {Proceedings of the First International Conference on Human Language Technology Research},
  year = {2001},
  doi = {10.3115/1072133.1072221},
  url = {https://doi.org/10.3115/1072133.1072221}
}
```

- [← previous The Form is the Substance: Classification of Genres in Text](https://md.research.elamur.ai/free/en/papers/dewdney-2001-form-is-substance-classification-genres/)
- [↑ back to papers of 2001](https://md.research.elamur.ai/free/en/papers/2001/)

## References

12 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Bikel, D., R. Schwartz, and R. Weischedel. 1999. An Algorithm that Learns What s in a Name. Machine Learning Special Issue on NL Learning, 34, 1—3. [find](https://scholar.google.com/scholar?q=Bikel%2C+D%2E%2C+R%2E+Schwartz%2C+and+R%2E+Weischedel%2E+1999%2E+An+Algorithm+that+Learns+What+s+in+a+Name%2E+Machine+Learning+Special+Issue+on+NL+Learning%2C+34%2C+1%E2%80%943%2E)
2. Choi, F.Y.Y. 2000. Advances in independent linear text segmentation. Proceedings of the 1 st Conference of the North American Chapter of the Association for Computational Linguistics (NAACL-00), 26—33. [find](https://scholar.google.com/scholar?q=Choi%2C+F%2EY%2EY%2E+2000%2E+Advances+in+independent+linear+text+segmentation%2E+Proceedings+of+the+1+st+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistics+%28NAACL%2D00%29%2C+26%E2%80%9433)
3. Fellbaum, Ch. (ed). 1998. WordNet: An Electronic Lexical Database. Cambridge: MIT Press. [find](https://scholar.google.com/scholar?q=Fellbaum%2C+Ch%2E+%28ed%29%2E+1998%2E+WordNet%3A+An+Electronic+Lexical+Database%2E+Cambridge%3A+MIT+Press%2E)
4. Gerber, L. 2001. A QA Typology for Webclopedia. In prep. [find](https://scholar.google.com/scholar?q=Gerber%2C+L%2E+2001%2E+A+QA+Typology+for+Webclopedia%2E+In+prep%2E)
5. Hearst, M.A. 1994. Multi-Paragraph Segmentation of Expository Text. Proceedings of the Annual Conference of the Association for Computational Linguistics (ACL-94). [find](https://scholar.google.com/scholar?q=Hearst%2C+M%2EA%2E+1994%2E+Multi%2DParagraph+Segmentation+of+Expository+Text%2E+Proceedings+of+the+Annual+Conference+of+the+Association+for+Computational+Linguistics+%28ACL%2D94%29%2E)
6. Hermjakob, U. 1997. Learning Parse and Translation Decisions from Examples with Rich Context. Ph.D. dissertation, University of Texas at Austin. file://ftp.cs.utexas.edu/pub/ mooney/papers/hermjakob-dissertation-97.ps.gz. [find](https://scholar.google.com/scholar?q=Hermjakob%2C+U%2E+1997%2E+Learning+Parse+and+Translation+Decisions+from+Examples+with+Rich+Context%2E+Ph%2ED%2E+dissertation%2C+University+of+Texas+at+Austin%2E+file%3A%2F%2Fftp%2Ecs%2Eutexas%2Eedu%2Fpub%2F+mooney%2Fpapers%2Fhermjakob%2Dd)
7. Hermjakob, U. 2000. Rapid Parser Development: A Machine Learning Approach for Korean. Proceedings of the 1 st Conference of the North American Chapter of the Association for Computational Linguistics (ANLP-NAACL-2000). [http://www.isi.edu/~ulf/papers/kor_naacl00.ps.gz](http://www.isi.edu/~ulf/papers/kor_naacl00.ps.gz). [↗](http://www.isi.edu/~ulf/papers/kor_naacl00.ps.gz)
8. Hermjakob, U. 2001. Parsing and Question Classification for Question Answering. In prep. [find](https://scholar.google.com/scholar?q=Hermjakob%2C+U%2E+2001%2E+Parsing+and+Question+Classification+for+Question+Answering%2E+In+prep%2E)
9. Hovy, E.H., L. Gerber, U. Hermjakob, M. Junk, and C.-Y. Lin. 2000. Question Answering in Webclopedia. Proceedings of the TREC-9 Conference. NIST. Gaithersburg, MD. [find](https://scholar.google.com/scholar?q=Hovy%2C+E%2EH%2E%2C+L%2E+Gerber%2C+U%2E+Hermjakob%2C+M%2E+Junk%2C+and+C%2E%2DY%2E+Lin%2E+2000%2E+Question+Answering+in+Webclopedia%2E+Proceedings+of+the+TREC%2D9+Conference%2E+NIST%2E+Gaithersburg%2C+MD%2E)
10. Moldovan, D., S. Harabagiu, M. Pasca, R. Mihalcea,, R. Girju, R. Goodrum, and V. Rus. 2000. The Structure and Performance of an Open-Domain Question Answering System. Proceedings of the Conference of the Association for Computational Linguistics (ACL-2000), 563—570. [find](https://scholar.google.com/scholar?q=Moldovan%2C+D%2E%2C+S%2E+Harabagiu%2C+M%2E+Pasca%2C+R%2E+Mihalcea%2C%2C+R%2E+Girju%2C+R%2E+Goodrum%2C+and+V%2E+Rus%2E+2000%2E+The+Structure+and+Performance+of+an+Open%2DDomain+Question+Answering+System%2E+Proceedings+of+the+Conference+of+)
11. Srihari, R. and W. Li. 2000. A Question Answering System Supported by Information Extraction. In Proceedings of the 1 st Conference of the North American Chapter of the Association for Computational Linguistics (ANLP-NAACL-00), 166—172. [find](https://scholar.google.com/scholar?q=Srihari%2C+R%2E+and+W%2E+Li%2E+2000%2E+A+Question+Answering+System+Supported+by+Information+Extraction%2E+In+Proceedings+of+the+1+st+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+L)
12. Witten, I.H., A. Moffat, and T.C. Bell. 1994. Managing Gigabytes: Compressing and Indexing Documents and Images. New York: Van Nostrand Reinhold. SURF . CAT D-PERIOD LEX . ROLES (DUMMY) SCORE 0 SURF its Valley of the Kings Pharaonic necropolis and the Karnak temple complex CAT S-NP CLASS I-EN-NECROP… [find](https://scholar.google.com/scholar?q=Witten%2C+I%2EH%2E%2C+A%2E+Moffat%2C+and+T%2EC%2E+Bell%2E+1994%2E+Managing+Gigabytes%3A+Compressing+and+Indexing+Documents+and+Images%2E+New+York%3A+Van+Nostrand+Reinhold%2E+SURF+%2E+CAT+D%2DPERIOD+LEX+%2E+ROLES+%28DUMMY%29+SCORE+0+SURF+i)