---
title: "A Simple Ensemble Method for Hedge Identification (2010)"
description: "The paper addresses CoNLL-2010 Shared Task 1: deciding whether a biomedical sentence states something… 84.8 F-measure. 10 references."
authors: ["Ferenc Szidarovszky", "Illés Solt", "Domonkos Tikk"]
year: 2010
venue: "Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task"
source: "https://aclanthology.org/W10-3021/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/szidarovszky-2010-simple-ensemble-method-hedge-identification/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/szidarovszky-2010-simple-ensemble-method-hedge-identification/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2010](https://md.research.elamur.ai/free/en/papers/2010/page/2/) / A Simple Ensemble Method for Hedge Identification

# A Simple Ensemble Method for Hedge Identification

**Authors:** Ferenc Szidarovszky, Illés Solt and Domonkos Tikk · **Year:** 2010 · **Venue:** [Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task](https://md.research.elamur.ai/free/en/venues/conll/) ([CoNLL 2010](https://md.research.elamur.ai/free/en/venues/conll/2010/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/W10-3021/) · [PDF](https://aclanthology.org/W10-3021.pdf) · [OpenAlex](https://openalex.org/W108081214)

**Citations:** 4 citations (OpenAlex, 2026-10-14)

## Summary
The paper addresses CoNLL-2010 Shared Task 1: deciding whether a biomedical sentence states something as uncertain (hedged) or as fact. The authors restrict themselves to the in-domain BioScope data. Instead of classifying whole sentences directly, they look for hedge cues inside the sentence and label it uncertain whenever at least one cue turns up, which mirrors the annotators' own rule.

Two cue finders are built. The first is a lexicon of annotated cues, filtered by how precise each cue is. The second is a CRF sequence tagger trained in MALLET on a handful of surface features. Their sentence-level outputs are merged with either a permissive OR or a strict AND rule.

On the trial data the OR merge scored best, so it became the official submission. On the evaluation set, however, the CRF alone came out on top with 84.8 F-measure, and the lexicon model's precision collapsed. The authors take this as a warning that ensemble choices tuned on one data distribution may not carry over to another.

## Key points
- Data: the abstract and full-text train/trial pairs were merged, and trial sentences were removed from train. The train set has 14,274 sentences, of which 17.9% are uncertain. The uncertain share is 22.9% in the trial set and 15.8% in the 5,003-sentence evaluation set.
- Statistical model: 186 candidate cues were collected from the annotations. Of these, 83 had precision 1.0 on train and 141 had precision of at least 0.5. The cut-off threshold was tuned on the trial set.
- CRF model: trained with MALLET on tokens split at whitespace and punctuation, with features from a ±2 token window. The best feature subset was the lowercased token plus an "all lowercase" indicator.
- Trial results (F-measure): statistical 86.4, CRF 85.0, AND 83.6, OR 87.5.
- Evaluation results (F-measure): statistical 78.8, CRF 84.8, AND 84.4, OR 79.2 (the official run). The statistical model's precision fell from 84.4 to 70.5, while the CRF stayed stable.

## Interesting ideas
- The sentence-level label is derived from detected sub-sentence cues, so every "uncertain" decision points to a concrete token or phrase as its justification.
- A cue lexicon harvested from annotations is pruned by per-cue precision, with the cut-off picked on held-out data rather than fixed by hand.
- Merging two predictors by OR trades precision for recall and merging by AND does the reverse, yet the better merge flipped between trial and test because the class prior shifted.
- When two feature subsets perform almost equally, the smaller one was preferred, and that lean CRF generalized more steadily than the lexicon model.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (frames): prag could mark frames as hedged or speculative only when a located cue word governs them in the UD tree, storing that cue as the explanation.
- **I2** (learn): self-induced repair rules could be admitted only if their measured precision on a held-out slice clears a tuned threshold, discarding noisy rules such as the weak cues "or" and "not".
- **I3** (learn): when combining witness parsers into silver data, the agreement policy should be validated on several sets with different distributions before it is trusted.
- **I4** (tokens): tagger and morphology features in crate en could be selected with an explicit bias toward fewer features, and the precision drop from dev to test tracked as an overfitting signal.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Humboldt-Universität zu Berlin](https://md.research.elamur.ai/free/en/universities/humboldt-universitat-zu-berlin/) — University, Berlin, Germany · 13 works · 41,845 citations
- [Budapest University of Technology and Economics](https://md.research.elamur.ai/free/en/universities/budapest-university-of-technology-and-economics/) — University, Budapest, Hungary · 2 works · 661 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works · [🧬 Biochemistry, Genetics and Molecular Biology](https://md.research.elamur.ai/free/en/shelves/biochemistry-genetics-and-molecular-biology/) · 116 works · [🧬 Molecular Biology](https://md.research.elamur.ai/free/en/shelves/biochemistry-genetics-and-molecular-biology/molecular-biology/) · 114 works

## Cite

Cite as: Ferenc Szidarovszky, Illés Solt and Domonkos Tikk. 2010. A Simple Ensemble Method for Hedge Identification. Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{szidarovszky2010simple,
  title = {{A Simple Ensemble Method for Hedge Identification}},
  author = {Ferenc Szidarovszky, Illés Solt and Domonkos Tikk},
  booktitle = {Proceedings of the Fourteenth Conference on Computational Natural Language Learning – Shared Task},
  year = {2010},
  url = {https://aclanthology.org/W10-3021/}
}
```

- [← previous Towards multi-lingual summarization: A comparative analysis of sentence extraction methods on English and Hebrew corpora](https://md.research.elamur.ai/free/en/papers/litvak-2010-multi-lingual-summarization-comparative-analysis/)
- [next → How Specialized are Specialized Corpora? Behavioral Evaluation of Corpus Representativeness for Maltese.](https://md.research.elamur.ai/free/en/papers/francom-2010-how-specialized-are-specialized-corpora/)
- [↑ back to papers of 2010](https://md.research.elamur.ai/free/en/papers/2010/page/2/)

## References

10 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Wendy W. Chapman, Will Bridewell, Paul Hanbury, Gregory F. Cooper, and Bruce G. Buchanan. 2001. A simple algorithm for identifying negated findings and diseases in discharge summaries. Journal of Biomedical Informatics, 2001:34–301. [↗](https://doi.org/10.1006/jbin.2001.1029)
2. Richárd Farkas, Veronika Vincze, György Móra, János Csirik, and György Szarvas. 2010. [The CoNLL-2010 Shared Task: Learning to Detect Hedges and their Scope in Natural Language Text](https://md.research.elamur.ai/free/en/papers/farkas-2010-conll-2010-shared-task-learning/). In Proceedings of the Fourteenth Conference on Computational Natural Language Learning (CoNLL-2010): Shared Task, pages 1–12, Uppsala, Sweden. ACL.
3. Jin-Dong Kim, Tomoko Ohta, Sampo Pyysalo, Yoshinobu Kano, and Jun’ichi Tsujii. 2009. Overview Model Combination AND Combination OR 93.9 75.4 83.6 83.6 91.8 87.5 Model Combination AND Combination OR 88.0 81.0 84.4 70.1 91.0 79.2 of BioNLP’09 shared task on event extraction. In BioNLP ’09: Proc. of the Workshop on BioNLP, pages 1–9, Morristown, NJ, USA. ACL. [find](https://scholar.google.com/scholar?q=Jin%2DDong+Kim%2C+Tomoko+Ohta%2C+Sampo+Pyysalo%2C+Yoshinobu+Kano%2C+and+Jun%E2%80%99ichi+Tsujii%2E+2009%2E+Overview+Model+Combination+AND+Combination+OR+93%2E9+75%2E4+83%2E6+83%2E6+91%2E8+87%2E5+Model+Combination+AND+Combination+OR+88)
4. Andrew K. McCallum. 2002. MALLET: A Machine Learning for Language Toolkit. [http://mallet.cs.umass.edu](http://mallet.cs.umass.edu). [↗](http://mallet.cs.umass.edu)
5. Ben Medlock and Ted Briscoe. 2007. [Weakly supervised learning for hedge classification in scientific literature](https://md.research.elamur.ai/free/en/papers/medlock-2007-weakly-supervised-learning-hedge-classification/). In Proc. of the 45th Annual Meeting of the Association of Computational Linguistics, pages 992–999, Prague, Czech Republic, June. ACL.
6. Arzucan Özgür and Dragomir R. Radev. 2009. Detecting speculations and their scopes in scientific text. In EMNLP ’09: Proc. of Conf. on Empirical Methods in Natural Language Processing, pages 1398– 1407, Morristown, NJ, USA. ACL. [find](https://scholar.google.com/scholar?q=Arzucan+O%CC%88zgu%CC%88r+and+Dragomir+R%2E+Radev%2E+2009%2E+Detecting+speculations+and+their+scopes+in+scientific+text%2E+In+EMNLP+%E2%80%9909%3A+Proc%2E+of+Conf%2E+on+Empirical+Methods+in+Natural+Language+Processing%2C+pages+1398%E2%80%93+1)
7. György Szarvas. 2008. [Hedge Classification in Biomedical Texts with a Weakly Supervised Selection of Keywords](https://md.research.elamur.ai/free/en/papers/szarvas-2008-hedge-classification-biomedical-texts-weakly/). In Proceedings of ACL-08: HLT, pages 281–289, Columbus, Ohio, June. ACL.
8. Özlem Uzuner, Xiaoran Zhang, and Tawanda Sibanda. 2009. Machine Learning and Rule-based Approaches to Assertion Classification. Journal of the American Medical Informatics Association, 16(1):109–115. [↗](https://doi.org/10.1197/jamia.m2950)
9. Özlem Uzuner. 2008. Second i2b2 workshop on natural language processing challenges for clinical records. In AMIA Annual Symposium Proceedings, pages 1252–3. [find](https://scholar.google.com/scholar?q=O%CC%88zlem+Uzuner%2E+2008%2E+Second+i2b2+workshop+on+natural+language+processing+challenges+for+clinical+records%2E+In+AMIA+Annual+Symposium+Proceedings%2C+pages+1252%E2%80%933%2E)
10. Veronika Vincze, György Szarvas, Richárd Farkas, György Móra, and János Csirik. 2008. The Bio-Scope corpus: biomedical texts annotated for uncertainty, negation and their scopes. BMC Bioinformatics, 9(Suppl 11):S9. [find](https://scholar.google.com/scholar?q=Veronika+Vincze%2C+Gyo%CC%88rgy+Szarvas%2C+Richa%CC%81rd+Farkas%2C+Gyo%CC%88rgy+Mo%CC%81ra%2C+and+Ja%CC%81nos+Csirik%2E+2008%2E+The+Bio%2DScope+corpus%3A+biomedical+texts+annotated+for+uncertainty%2C+negation+and+their+scopes%2E+BMC+Bioinformati)

*Updated 24 October 2026*