--- title: "Automatic classification of citation function — Teufel 2006" description: "The paper addresses how to determine automatically why an author cites a given work, for example to criticise it, to contrast with it, or to build on its…" authors: ["Simone Teufel", "Advaith Siddharthan", "Dan Tidhar"] year: 2006 venue: "Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing" source: "https://aclanthology.org/W06-1613/" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/teufel-2006-automatic-classification-citation-function/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/teufel-2006-automatic-classification-citation-function/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2006](https://md.research.elamur.ai/free/en/papers/2006/) / Automatic classification of citation function # Automatic classification of citation function **Authors:** [Simone Teufel](https://md.research.elamur.ai/free/en/authors/simone-teufel/), [Advaith Siddharthan](https://md.research.elamur.ai/free/en/authors/advaith-siddharthan/), Dan Tidhar · **Year:** 2006 · **Venue:** Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/W06-1613/) · [PDF](https://aclanthology.org/W06-1613.pdf) · [OpenAlex](https://openalex.org/W2088336913) **Citations:** 423 citations (openalex-snapshot · 2026-10-14) · [93](#cited-by) in our library, of the 140884 papers whose bibliographies we read ## Summary The paper addresses how to determine automatically why an author cites a given work, for example to criticise it, to contrast with it, or to build on its methods. Raw citation counts and the text snippets shown by current indexers miss this information. The authors adapt an older content-analysis scheme by Spiegel-Rüsing into 12 categories tailored to computational linguistics papers. The categories are grouped into weakness, contrast/comparison, positive usage/agreement, and a neutral class. Annotators may only assign a function when explicit textual evidence supports it, and under this rule independent coders reached K=.72. On 116 articles with 2829 citation instances, a memory-based classifier (IBk, k=3, in Weka) used cue phrases, tense, voice, modality, location and self-citation features. It reached K=.57 and 77% accuracy on the full scheme. Coarser groupings scored somewhat higher. This makes richer citation indexes and finer-grained bibliometrics more practical. ## Key points - Each citation receives exactly one of 12 labels: a weakness label (Weak), four contrast/comparison labels (CoCoXY, CoCoGM, CoCoR0, CoCo-), six positive labels (PUse, PBas, PSup, PModi, PSim, PMot), and a neutral label (Neut). - The guidelines run to about 25 pages with roughly 150 rules, a decision tree and aids for ambiguous cases. Human agreement was K=.72 on the full scheme and K=.76 on the four top-level classes. - The evaluation corpus has 116 articles and 2829 citations, evaluated with 10-fold cross-validation. Reported measures are Kappa, accuracy and Macro-F. - The full scheme reached K=.57 (acc .77). The four-way grouping (Weak/Positive/Contrast/Neutral) reached K=.59, acc .79 and Macro-F .68. A three-way sentiment-style grouping (Weak/Positive/Neutral) reached K=.58, acc .83 and Macro-F .71. - In an earlier annotated corpus, 69% of 600 contrast sentences and 21% of 246 continuation sentences did not contain the citation they referred to. Fixed snippets around the citation therefore often miss the relevant statement. ## Interesting ideas - **Evidence-required annotation:** coders must name the textual cue behind each label and may not rely on expert knowledge of the field. This turns an intention-guessing task into a more objective one, which helps any subjective labeling project such as stance or argumentation. - **A neutral catch-all category:** Neut covers truly neutral mentions, cases with too little evidence, and functions the scheme does not list. Every instance gets a label without forcing weak guesses into specific classes, a useful pattern for skewed taxonomies. - **Macro-F and Kappa over accuracy:** with heavily skewed classes, these measures keep frequent categories from dominating the evaluation. Kappa also corrects for chance agreement. This is good practice for any imbalanced classification benchmark. - **Regrouping a fine scheme into coarser views:** collapsing the 12 labels into top-level or sentiment-like groups lets one model serve several applications. It also shows where the classifier's errors concentrate. - **Document-position and self-citation features:** motivation and weakness citations tend to appear early in a paper, while result comparisons appear late. Author overlap between citing and cited papers flags likely reuse. These cheap structural signals could transfer to other discourse tasks on scientific text. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Cited by Cited by 93 papers of our library cite it (of the 140884 whose bibliographies we read); 6 of them have a page here, 87 not yet. - [The ACL Anthology Reference Corpus: A Reference Dataset for Bibliographic Research in Computational Linguistics](https://md.research.elamur.ai/free/en/papers/bird-2008-acl-anthology-reference-corpus-reference/) - [Structural Scaffolds for Citation Intent Classification in Scientific Publications](https://md.research.elamur.ai/free/en/papers/cohan-2019-structural-scaffolds-citation-intent-classification/) - [Measuring the Evolution of a Scientific Field through Citation Frames](https://md.research.elamur.ai/free/en/papers/jurgens-2018-measuring-evolution-scientific-field-through/) - [Argument Mining: A Survey](https://md.research.elamur.ai/free/en/papers/lawrence-2019-argument-mining-survey/) - [S2ORC: The Semantic Scholar Open Research Corpus](https://md.research.elamur.ai/free/en/papers/lo-2019-s2orc-semantic-scholar-open-research/) - [Large Language Models for Citation Function Classification](https://md.research.elamur.ai/free/en/papers/vodicka-2026-large-language-models-citation-function/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{teufel2006automatic, title = {{Automatic classification of citation function}}, author = {Teufel, Simone and Siddharthan, Advaith and Tidhar, Dan}, booktitle = {Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing}, year = {2006}, doi = {10.3115/1610075.1610091}, url = {https://doi.org/10.3115/1610075.1610091} } ``` - [← previous Arabic Preprocessing Schemes for Statistical Machine Translation](https://md.research.elamur.ai/free/en/papers/habash-2006-arabic-preprocessing-schemes-statistical-machine/) - [next → Automatic Identification of Non-Compositional Multi-Word Expressions using Latent Semantic Analysis](https://md.research.elamur.ai/free/en/papers/katz-2006-automatic-identification-non-compositional-multi/) - [↑ back to papers of 2006](https://md.research.elamur.ai/free/en/papers/2006/) ## References 9 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Rashid M. Abdalla and Simone Teufel. 2006. A bootstrapping approach to unsupervised detection of cue phrase variants. In Proc. of ACL/COLING-06. [find](https://scholar.google.com/scholar?q=Rashid+M%2E+Abdalla+and+Simone+Teufel%2E+2006%2E+A+bootstrapping+approach+to+unsupervised+detection+of+cue+phrase+variants%2E+In+Proc%2E+of+ACL%2FCOLING%2D06%2E) 2. Susan Bonzi. 1982. Characteristics of a literature as predictors of relatedness between cited and citing works. JASIS, 33(4):208–216. [find](https://scholar.google.com/scholar?q=Susan+Bonzi%2E+1982%2E+Characteristics+of+a+literature+as+predictors+of+relatedness+between+cited+and+citing+works%2E+JASIS%2C+33%284%29%3A208%E2%80%93216%2E) 3. Christine L. Borgman, editor. 1990. Scholarly Communication and Bibliometrics. Sage Publications, CA. [find](https://scholar.google.com/scholar?q=Christine+L%2E+Borgman%2C+editor%2E+1990%2E+Scholarly+Communication+and+Bibliometrics%2E+Sage+Publications%2C+CA%2E) 4. Jean Carletta. 1996. Assessing agreement on classification tasks: The kappa statistic. Computational Linguistics, 22(2):249–254. [find](https://scholar.google.com/scholar?q=Jean+Carletta%2E+1996%2E+Assessing+agreement+on+classification+tasks%3A+The+kappa+statistic%2E+Computational+Linguistics%2C+22%282%29%3A249%E2%80%93254%2E) [our card](https://md.research.elamur.ai/free/en/papers/carletta-1996-assessing-agreement-classification-tasks-kappa/) 5. Daryl E. Chubin and S. D. Moitra. 1975. Content analysis of references: Adjunct or alternative to citation counting? Social Studies of Science, 5(4):423–441. [find](https://scholar.google.com/scholar?q=Daryl+E%2E+Chubin+and+S%2E+D%2E+Moitra%2E+1975%2E+Content+analysis+of+references%3A+Adjunct+or+alternative+to+citation+counting%3F+Social+Studies+of+Science%2C+5%284%29%3A423%E2%80%93441%2E) 6. Eugene Garfield. 1979. Citation Indexing: Its Theory and Application in Science, Technology and Humanities. J. Wiley, New York, NY. [find](https://scholar.google.com/scholar?q=Eugene+Garfield%2E+1979%2E+Citation+Indexing%3A+Its+Theory+and+Application+in+Science%2C+Technology+and+Humanities%2E+J%2E+Wiley%2C+New+York%2C+NY%2E) 7. C. Lee Giles, Kurt D. Bollacker, and Steve Lawrence. 1998. Citeseer: An automatic citation indexing system. In Proc. of the Third ACM Conference on Digital Libraries, pages 89–98. [find](https://scholar.google.com/scholar?q=C%2E+Lee+Giles%2C+Kurt+D%2E+Bollacker%2C+and+Steve+Lawrence%2E+1998%2E+Citeseer%3A+An+automatic+citation+indexing+system%2E+In+Proc%2E+of+the+Third+ACM+Conference+on+Digital+Libraries%2C+pages+89%E2%80%9398%2E) 8. T.L. Hodges. 1972. Citation Indexing: Its Potential for Bibliographical Control. Ph.D. thesis, University of California at Berkeley. [find](https://scholar.google.com/scholar?q=T%2EL%2E+Hodges%2E+1972%2E+Citation+Indexing%3A+Its+Potential+for+Bibliographical+Control%2E+Ph%2ED%2E+thesis%2C+University+of+California+at+Berkeley%2E) 9. David D. Lewis. 1991. Evaluating text categorisation. In Speech and Natural Language: Proceedings of the ARPA Workshop of Human Language Technology. [find](https://scholar.google.com/scholar?q=David+D%2E+Lewis%2E+1991%2E+Evaluating+text+categorisation%2E+In+Speech+and+Natural+Language%3A+Proceedings+of+the+ARPA+Workshop+of+Human+Language+Technology%2E)