---
title: "*SEM 2012 Shared Task — paper by Morante and Blanco (2012)"
description: "This paper describes the first *SEM shared task (2012), which targeted negation in general-domain… 25 references, cited by 78 papers here."
authors: ["Roser Morante", "Eduardo Blanco"]
year: 2012
venue: "*SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012)"
source: "https://aclanthology.org/S12-1035/"
licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)"
licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/morante-2012-sem-2012-shared-task-resolving/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/morante-2012-sem-2012-shared-task-resolving/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2012](https://md.research.elamur.ai/free/en/papers/2012/) / \*SEM 2012 Shared Task: Resolving the Scope and Focus of Negation

# \*SEM 2012 Shared Task: Resolving the Scope and Focus of Negation

**Authors:** [Roser Morante](https://md.research.elamur.ai/free/en/authors/roser-morante/) and [Eduardo Blanco](https://md.research.elamur.ai/free/en/authors/eduardo-blanco/) · **Year:** 2012 · **Venue:** [\*SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012)](https://md.research.elamur.ai/free/en/venues/starsem/) ([\*SEM 2012](https://md.research.elamur.ai/free/en/venues/starsem/2012/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/)

**Links:** [Publisher's page](https://aclanthology.org/S12-1035/) · [PDF](https://aclanthology.org/S12-1035.pdf) · [OpenAlex](https://openalex.org/W1954622760)

**Citations:** 120 citations (OpenAlex, 2026-10-14) · [78](https://md.research.elamur.ai/free/en/papers/morante-2012-sem-2012-shared-task-resolving/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
This paper describes the first *SEM shared task (2012), which targeted negation in general-domain English rather than the biomedical texts that most earlier work relied on. It defines two tasks. Task 1, scope resolution, has three parts: finding negation cues, marking the tokens they affect, and identifying the negated event. Task 2, focus detection, asks which semantic role is most directly negated. The organizers built two new resources for this. CD-SCO is a set of Conan Doyle stories annotated with cues, scopes and negated events. PB-FOC adds focus annotation on top of PropBank (WSJ). They also introduced new, more lenient evaluation software. Fourteen runs were submitted, using machine learning (SVMs, CRFs), rule-based or hybrid systems.

- Machine-learning systems performed best.
- Cue detection was nearly solved, but scope and negated-event detection remained hard.

The shared datasets, evaluation measures and baseline results give a common benchmark for negation processing beyond the clinical domain.

## Key points
- **Submissions:** 14 runs in total.
  - 12 for scope: 7 closed track, 5 open track.
  - 2 for focus, both open track.
  - A planned joint scope+focus pilot task was cancelled for lack of participants.
- **CD-SCO splits:**
  - Training: *The Hound of the Baskervilles*.
  - Development: *The Adventure of Wisteria Lodge*.
  - Test: two stories, *The Adventure of the Cardboard Box* plus *The Adventure of the Red Circle*.
  - Data use a CoNLL-style token-per-line format.
- **PB-FOC scope:** only verbal negations marked MNEG in PropBank are covered, and the focus is always an entire semantic role.
- **Best Task 1 scores (F1):**
  - Cues: 92.34 (FBK, CRFs).
  - Scope: 72.40 / 72.39 (UWashington and UiO2, both CRF-based).
  - Negated events: 67.02 (UiO1, which first decides factuality, then uses an SVM ranker).
  - Global, closed track: 57.63.
  - Global, open track: 54.82.
  - Best rule-based system: 39.56.
- **Effect of counting partial matches:** the best Scope NCM precision was 90.00%, but only 59.54% under the "B" variant. The system ranking stayed the same under both variants.

## Interesting ideas
- **Don't punish near misses more than total misses.** Partially correct predictions are counted only as false negatives, not also as false positives. Under standard F1, a nearly right span scores worse than no prediction at all; this rule avoids that and could help any span-labeling evaluation (NER, SRL arguments).
- **Report several complementary metrics.** Cue-level, scope-level with strict and relaxed cue matching, token-level, global, and percent of fully correct sentences were all reported. Token-level scores hide weaknesses in handling whole sequences, so publishing multiple views shows where systems really fail.
- **Separate closed and open tracks.** The closed track allows only the provided annotations; the open track permits external tools. This makes it possible to tell modeling gains apart from resource gains, and is useful when designing shared tasks or benchmarks for lower-resourced languages.
- **Annotate focus as a whole semantic role on top of PropBank.** Reusing an existing annotation layer makes the new layer cheaper to create and keeps it consistent. Once the focus is known, the rest of the negated clause can be read as implicitly positive, which is useful for information extraction.
- **Check factuality before choosing a negated event.** UiO1 first filters out non-factual negations and then ranks candidate events. This two-step design could carry over to event extraction in other languages.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Antwerp](https://md.research.elamur.ai/free/en/universities/university-of-antwerp/) — University, Antwerp, Belgium · 9 works · 3,209 citations
- [Lymba](https://md.research.elamur.ai/free/en/companies/lymba/) — Company, Richardson, United States · 2 works · 200 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

## Cite

Cite as: Roser Morante and Eduardo Blanco. 2012. \*SEM 2012 Shared Task: Resolving the Scope and Focus of Negation. \*SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{morante2012sem,
  title = {{*SEM 2012 Shared Task: Resolving the Scope and Focus of Negation}},
  author = {Roser Morante and Eduardo Blanco},
  booktitle = {*SEM 2012: The First Joint Conference on Lexical and Computational Semantics – Volume 1: Proceedings of the main conference and the shared task, and Volume 2: Proceedings of the Sixth International Workshop on Semantic Evaluation (SemEval 2012)},
  year = {2012},
  url = {https://aclanthology.org/S12-1035/}
}
```

- [← previous Did It Happen? The Pragmatic Complexity of Veridicality Assessment](https://md.research.elamur.ai/free/en/papers/marneffe-2012-did-it-happen-pragmatic-complexity/)
- [next → Paraphrasing for Style](https://md.research.elamur.ai/free/en/papers/xu-2012-paraphrasing-style/)
- [↑ back to papers of 2012](https://md.research.elamur.ai/free/en/papers/2012/)

## References

25 references, 12 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Eduardo Blanco and Dan Moldovan. 2011. Semantic Representation of Negation Using Focus Detection. In Proceedings of the 49th Annual Meeting of the Association for C omputational Linguistics: Human Language Technologies, pages 581–589, Portland, Oregon, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Eduardo+Blanco+and+Dan+Moldovan%2E+2011%2E+Semantic+Representation+of+Negation+Using+Focus+Detection%2E+In+Proceedings+of+the+49th+Annual+Meeting+of+the+Association+for+C+omputational+Linguistics%3A+Human+Lan)
2. Eric Brill. 1992. [A simple rule-based part of speech tagger](https://md.research.elamur.ai/free/en/papers/brill-1992-simple-rule-based-part-speech/). In Proceedings of the third conference on Applied natural language processing, ANLC ’92, pages 152– 155, Stroudsburg, PA, USA. Association for Computational Linguistics.
3. Wendy W. Chapman, Will Bridewell, Paul Hanbury, Gregory F. Cooper, and Bruce G. Buchanan. 2001. A simple algorithm for identifying negated findings and diseases in discharge summaries. J Biomed Inform, 34:301–310. [find](https://scholar.google.com/scholar?q=Wendy+W%2E+Chapman%2C+Will+Bridewell%2C+Paul+Hanbury%2C+Gregory+F%2E+Cooper%2C+and+Bruce+G%2E+Buchanan%2E+2001%2E+A+simple+algorithm+for+identifying+negated+findings+and+diseases+in+discharge+summaries%2E+J+Biomed+Inform)
4. Eugene Charniak and Mark Johnson. 2005. [Coarse-to-fine n-best parsing and maxent discriminative reranking](https://md.research.elamur.ai/free/en/papers/charniak-2005-coarse-fine-n-best-parsing/). In Proceedings of the 43rd Annual Meeting of the Association for Computational Linguistics, pages 173–180, Ann Arbor.
5. Eugene Charniak. 2000. [A maximum-entropy-inspired parser](https://md.research.elamur.ai/free/en/papers/charniak-2000-maximum-entropy-inspired-parser/). In Proceedings of the 1st North American chapter of the Association for Computational Linguistics conference, NAACL 2000, pages 132–139, Stroudsburg, PA, USA. Association for Computational Linguistics.
6. Marie-Catherine de Marneffe, Bill MacCartney, and Christopher D. Manning. 2006. [Generating Typed Dependency Parses from Phrase Structure Parses](https://md.research.elamur.ai/free/en/papers/marneffe-2006-generating-typed-dependency-parses-phrase/). In Proceedings of the IEEE / ACL 2006 Workshop on Spoken Language Technology. The Stanford Natural Language Processing Group.
7. Richárd Farkas, Veronika Vincze, György Móra, János Csirik, and György Szarvas. 2010. [The CoNLL-2010 Shared Task: Learning to Detect Hedges and their Scope in Natural Language Text](https://md.research.elamur.ai/free/en/papers/farkas-2010-conll-2010-shared-task-learning/). In Proceedings of the Fourteenth Conference on Computational Natural Language Learning, pages 1–12, Uppsala, Swed…
8. Jenny Rose Finkel, Trond Grenager, and Christopher Manning. 2005. [Incorporating non-local information into information extraction systems by gibbs sampling](https://md.research.elamur.ai/free/en/papers/finkel-2005-incorporating-non-local-information-into/). In Proceedings of the 43rd Annual Meeting on Association for Computational Linguistics, ACL ’05, pages 363–370, Stroudsburg, PA, USA. Associati…
9. Rodney D. Huddleston and Geoffrey K. Pullum. 2002. [The Cambridge Grammar of the English Language](https://md.research.elamur.ai/free/en/papers/huddleston-2002-cambridge-grammar-english-language/). Cambridge University Press.
10. Halil Kilicoglu and Sabine Bergler. 2011. Effective bio-event extraction using trigger words and syntactic dependencies. Computational Intelligence, 27(4):583– 609. [find](https://scholar.google.com/scholar?q=Halil+Kilicoglu+and+Sabine+Bergler%2E+2011%2E+Effective+bio%2Devent+extraction+using+trigger+words+and+syntactic+dependencies%2E+Computational+Intelligence%2C+27%284%29%3A583%E2%80%93+609%2E)
11. Roser Morante and Walter Daelemans. 2009. A metalearning approach to processing the scope of negation. In Proceedings of the 13th Conference on Natural Language Learning, pages 21–29, Boulder, CO. [find](https://scholar.google.com/scholar?q=Roser+Morante+and+Walter+Daelemans%2E+2009%2E+A+metalearning+approach+to+processing+the+scope+of+negation%2E+In+Proceedings+of+the+13th+Conference+on+Natural+Language+Learning%2C+pages+21%E2%80%9329%2C+Boulder%2C+CO%2E)
12. Roser Morante and Walter Daelemans. 2012. ConanDoyle-neg: Annotation of negation cues and their scope in Conan Doyle stories. In Proceedings of LREC 2012, Istambul. [find](https://scholar.google.com/scholar?q=Roser+Morante+and+Walter+Daelemans%2E+2012%2E+ConanDoyle%2Dneg%3A+Annotation+of+negation+cues+and+their+scope+in+Conan+Doyle+stories%2E+In+Proceedings+of+LREC+2012%2C+Istambul%2E)
13. Roser Morante and Caroline Sporleder. 2012. Special issue on modality and negation: An introduction. Computational Linguistics. [find](https://scholar.google.com/scholar?q=Roser+Morante+and+Caroline+Sporleder%2E+2012%2E+Special+issue+on+modality+and+negation%3A+An+introduction%2E+Computational+Linguistics%2E)
14. Roser Morante, Sarah Schrauwen, and Walter Daelemans. 2011. Annotation of negation cues and their scope. guidelines v1.0. Technical Report Series CTR-003, CLiPS, University of Antwerp, Antwerp, April. [find](https://scholar.google.com/scholar?q=Roser+Morante%2C+Sarah+Schrauwen%2C+and+Walter+Daelemans%2E+2011%2E+Annotation+of+negation+cues+and+their+scope%2E+guidelines+v1%2E0%2E+Technical+Report+Series+CTR%2D003%2C+CLiPS%2C+University+of+Antwerp%2C+Antwerp%2C+April%2E)
15. Xuan-Hieu Phan. 2006. Crfchunker: Crf english phrase chunker. [find](https://scholar.google.com/scholar?q=Xuan%2DHieu+Phan%2E+2006%2E+Crfchunker%3A+Crf+english+phrase+chunker%2E)
16. Vasin Punyakanok, Dan Roth, and Wen-tau Yih. 2008. [The importance of syntactic parsing and inference in semantic role labeling](https://md.research.elamur.ai/free/en/papers/punyakanok-2008-importance-syntactic-parsing-inference-semantic/). Computational Linguistics, 34(2):257–287, June.
17. Dan Shen and Mirella Lapata. 2007. [Using Semantic Roles to Improve Question Answering](https://md.research.elamur.ai/free/en/papers/shen-2007-semantic-roles-improve-question-answering/). In Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning (EM NLP-CoNLL), pages 12–21.
18. Ekaterina Shutova. 2010. Models of Metaphor in NLP. In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 688–697, Uppsala, Sweden. ACL. [find](https://scholar.google.com/scholar?q=Ekaterina+Shutova%2E+2010%2E+Models+of+Metaphor+in+NLP%2E+In+Proceedings+of+the+48th+Annual+Meeting+of+the+Association+for+Computational+Linguistics%2C+pages+688%E2%80%93697%2C+Uppsala%2C+Sweden%2E+ACL%2E)
19. Mihai Surdeanu, Sanda Harabagiu, John Williams, and Paul Aarseth. 2003. [Using Predicate-Argument Structures for Information Extraction](https://md.research.elamur.ai/free/en/papers/surdeanu-2003-predicate-argument-structures-information-extraction/). In Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, pages 8–15, Sapporo, Japan. Association for Computational Linguistics.
20. Mihai Surdeanu, Richard Johansson, Adam Meyers, Lluı́s Màrquez, and Joakim Nivre. 2008. [The conll-2008 shared task on joint parsing of syntactic and semantic dependencies](https://md.research.elamur.ai/free/en/papers/surdeanu-2008-conll-2008-shared-task-joint/). In CoNLL 2008: Proceedings of the 12th Conference on Computational Natural Language Learning, page 159177, Manchester.
21. Yoshimasa Tsuruoka and Jun’ichi Tsujii. 2005. [Bidirectional inference with the easiest-first strategy for tagging sequence data](https://md.research.elamur.ai/free/en/papers/tsuruoka-2005-bidirectional-inference-easiest-first-strategy/). In Proceedings of of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing, pages 467–474, Vancouver.
22. Erik Velldal, Lilja Øvrelid, Jonathon Read, and Stephan Oepen. 2012. Speculation and negation: Rules, rankers, and the role of syntax. Computational Linguistics. [find](https://scholar.google.com/scholar?q=Erik+Velldal%2C+Lilja+%C3%98vrelid%2C+Jonathon+Read%2C+and+Stephan+Oepen%2E+2012%2E+Speculation+and+negation%3A+Rules%2C+rankers%2C+and+the+role+of+syntax%2E+Computational+Linguistics%2E)
23. Veronika Vincze, Gyorgy Szarvas, Richard Farkas, Gyorgy Mora, and Janos Csirik. 2008. The Bio-Scope corpus: biomedical texts annotated for uncertainty, negation and their scopes. BMC Bioinformatics, 9(Suppl 11):S9+. [find](https://scholar.google.com/scholar?q=Veronika+Vincze%2C+Gyorgy+Szarvas%2C+Richard+Farkas%2C+Gyorgy+Mora%2C+and+Janos+Csirik%2E+2008%2E+The+Bio%2DScope+corpus%3A+biomedical+texts+annotated+for+uncertainty%2C+negation+and+their+scopes%2E+BMC+Bioinformatics%2C+9)
24. Michael Wiegand, Alexandra Balahur, Benjamin Roth, Dietrich Klakow, and Andrés Montoyo. 2010. A survey on the role of negation in sentiment analysis. In Proceedings of the Workshop on Negation and Speculation in Natural Language Processing, pages 60–68, Uppsala, Sweden. University of Antwerp. [find](https://scholar.google.com/scholar?q=Michael+Wiegand%2C+Alexandra+Balahur%2C+Benjamin+Roth%2C+Dietrich+Klakow%2C+and+Andre%CC%81s+Montoyo%2E+2010%2E+A+survey+on+the+role+of+negation+in+sentiment+analysis%2E+In+Proceedings+of+the+Workshop+on+Negation+and+Sp)
25. Dekai Wu and Pascale Fung. 2009. Semantic Roles for SMT: A Hybrid Two-Pass Model. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Short Papers, pages 13–16, Boulder, Colorado.… [find](https://scholar.google.com/scholar?q=Dekai+Wu+and+Pascale+Fung%2E+2009%2E+Semantic+Roles+for+SMT%3A+A+Hybrid+Two%2DPass+Model%2E+In+Proceedings+of+Human+Language+Technologies%3A+The+2009+Annual+Conference+of+the+North+American+Chapter+of+the+Associa)

*Updated 24 October 2026*