--- title: "Squibs: What Is a Paraphrase? — Bhagat 2013" description: "Paraphrase recognition matters for NLP tasks such as question answering, information extraction, textual entailment and machine translation." authors: ["Rahul Bhagat", "Eduard Hovy"] year: 2013 venue: "Computational Linguistics" source: "https://aclanthology.org/J13-3001/" doi: "10.1162/coli_a_00166" licence: "CC BY-NC-SA 3.0 (ACL Anthology, before 2016)" licence_url: "https://creativecommons.org/licenses/by-nc-sa/3.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/bhagat-2013-squibs-what-is-paraphrase/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bhagat-2013-squibs-what-is-paraphrase/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2013](https://md.research.elamur.ai/free/en/papers/2013/page/2/) / Squibs: What Is a Paraphrase? # Squibs: What Is a Paraphrase? **Authors:** Rahul Bhagat, [Eduard Hovy](https://md.research.elamur.ai/free/en/authors/eduard-hovy/) · **Year:** 2013 · **Venue:** Computational Linguistics · **Licence:** [CC BY-NC-SA 3.0 (ACL Anthology, before 2016)](https://creativecommons.org/licenses/by-nc-sa/3.0/) **Links:** [Publisher's page](https://aclanthology.org/J13-3001/) · [PDF](https://aclanthology.org/J13-3001.pdf) · [DOI 10.1162/coli\_a\_00166](https://doi.org/10.1162/coli_a_00166) · [OpenAlex](https://openalex.org/W2016172157) **Citations:** 175 citations (openalex-snapshot · 2026-10-14) · [41](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary Paraphrase recognition matters for NLP tasks such as question answering, information extraction, textual entailment and machine translation. However, there has been no agreed definition of a paraphrase. Logic demands identical meaning, while linguists accept looser, approximate equivalence. Bhagat and Hovy adopt the broad view and call such pairs "quasi-paraphrases." Drawing on earlier theories (Mel'cuk, Harris, Honeck) and on corpus analysis, they propose an inventory of 25 lexical operations that produce quasi-paraphrases. Examples include synonym, antonym and converse substitution, voice change, nominalization, tense or modality change, semantic implication and use of external knowledge. They test the list in two ways: - **Coverage:** they annotate phrase pairs sampled from the MTC and MSR paraphrase corpora. - **Precision:** two annotators judge sentences rewritten with each operation, built from WordNet, DIRT and Gigaword. The inventory covered every observed phrase pair. A small number of operations account for most real paraphrases, and which ones dominate depends on the corpus. The authors argue that useful paraphrase types are application-dependent. Their taxonomy gives a map for focusing future paraphrase research. ## Key points - **Sampling:** 30 random sentence-level pairs were taken from each corpus, using manual alignments by Cohn, Callison-Burch, and Lapata (2008). This yielded 210 phrase pairs for MTC and 145 for MSR. No pair needed the "unknown" label. - **Most frequent operations:** synonym substitution (37% MTC, 19% MSR) and function word variations (37% MTC, 30% MSR). External knowledge was rare in MTC (6%) but common in MSR (32%). - **Precision test design:** for each operation, two words or phrases were chosen and five Gigaword sentences per item were rewritten, giving 10 test sentences per operation. Inter-annotator agreement was κ = 0.66. - **Precision results:** Repetition/Ellipsis and Verb/Noun conversion reached 100%. Manipulator/Device substitution was lowest at 30%. Synonym substitution and external knowledge both scored 95%. - **Scope:** the paper is a definitional squib and reports no system or application results. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Machine translation](https://md.research.elamur.ai/free/en/shelves/machine-translation/) · [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Cited by Cited by 41 papers of our library cite it (of the 140673 whose bibliographies we read); 6 of them have a page here, 35 not yet. - [Reformulating Unsupervised Style Transfer as Paraphrase Generation](https://md.research.elamur.ai/free/en/papers/krishna-2020-reformulating-unsupervised-style-transfer-as/) - [Paraphrasing evades detectors of AI-generated text, but retrieval is an effective defense](https://md.research.elamur.ai/free/en/papers/krishna-2023-paraphrasing-evades-detectors-ai-generated/) - [Automatic Machine Translation Evaluation in Many Languages via Zero-Shot Paraphrasing](https://md.research.elamur.ai/free/en/papers/thompson-2020-automatic-machine-translation-evaluation-many/) - [Paraphrase Generation as Zero-Shot Multilingual Translation: Disentangling Semantic Similarity from Lexical and Syntactic Diversity](https://md.research.elamur.ai/free/en/papers/thompson-2020-paraphrase-generation-as-zero-shot/) - [Paraphrasing as Zero-shot Translation with Feature-guided Diversity Enhancement](https://md.research.elamur.ai/free/en/papers/yan-2026-paraphrasing-as-zero-shot-translation/) - [CREPE: Open-Domain Question Answering with False Presuppositions](https://md.research.elamur.ai/free/en/papers/yu-2023-crepe-open-domain-question-answering/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @article{bhagat2013squibs, title = {{Squibs: What Is a Paraphrase?}}, author = {Bhagat, Rahul and Hovy, Eduard}, journal = {Computational Linguistics}, year = {2013}, doi = {10.1162/coli\_a\_00166}, url = {https://doi.org/10.1162/coli_a_00166} } ``` - [← previous SemEval-2013 Task 7: The Joint Student Response Analysis and 8th Recognizing Textual Entailment Challenge](https://md.research.elamur.ai/free/en/papers/dzikovska-2013-semeval-2013-task-7-joint/) - [next → Supervised Learning of Complete Morphological Paradigms](https://md.research.elamur.ai/free/en/papers/lex-2013-durrett-morphological-paradigms/) - [↑ back to papers of 2013](https://md.research.elamur.ai/free/en/papers/2013/page/2/) ## References 13 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Clark, E. V. 1992. Conventionality and contrasts: Pragmatic principles with lexical consequences. In Andrienne Lehrer and Eva Feder Kittay, editors, Frame, Fields, and Contrasts: New Essays in Semantic Lexical Organization. Lawrence Erlbaum Associates, Hillsdale, NJ, pages 171–188. [find](https://scholar.google.com/scholar?q=Clark%2C+E%2E+V%2E+1992%2E+Conventionality+and+contrasts%3A+Pragmatic+principles+with+lexical+consequences%2E+In+Andrienne+Lehrer+and+Eva+Feder+Kittay%2C+editors%2C+Frame%2C+Fields%2C+and+Contrasts%3A+New+Essays+in+Semanti) 2. Cohn, T., C. Callison-Burch, and M. Lapata. 2008. Constructing corpora for the development and evaluation of paraphrase systems. Computational Linguistics, 34(4):597–614. [↗](https://doi.org/10.1162/coli.08-003-r1-07-044) [our card](https://md.research.elamur.ai/free/en/papers/cohn-2008-constructing-corpora-development-evaluation-paraphrase/) 3. De Beaugrande, R. and W. V. Dressler. 1981. Introduction to Text Linguistics. Longman, New York, NY. [↗](https://doi.org/10.4324/9781315835839) 4. Dolan, B., C. Quirk, and C. Brockett. 2004. Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources. In Proceedings of the Conference on Computational Linguistics (COLING), pages 350–357, Geneva. [find](https://scholar.google.com/scholar?q=Dolan%2C+B%2E%2C+C%2E+Quirk%2C+and+C%2E+Brockett%2E+2004%2E+Unsupervised+construction+of+large+paraphrase+corpora%3A+Exploiting+massively+parallel+news+sources%2E+In+Proceedings+of+the+Conference+on+Computational+Linguis) [our card](https://md.research.elamur.ai/free/en/papers/dolan-2004-unsupervised-construction-large-paraphrase-corpora/) 5. Fellbaum, C. 1998. An Electronic Lexical Database. MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Fellbaum%2C+C%2E+1998%2E+An+Electronic+Lexical+Database%2E+MIT+Press%2C+Cambridge%2C+MA%2E) 6. Harris, Z. 1981. Co-occurence and transformation in linguistic structure. In Henry Hiz, editor, Papers on Syntax. D. Reidel Publishing Co., Dordrecht, pages 143–210. First published in 1957. [find](https://scholar.google.com/scholar?q=Harris%2C+Z%2E+1981%2E+Co%2Doccurence+and+transformation+in+linguistic+structure%2E+In+Henry+Hiz%2C+editor%2C+Papers+on+Syntax%2E+D%2E+Reidel+Publishing+Co%2E%2C+Dordrecht%2C+pages+143%E2%80%93210%2E+First+published+in+1957%2E) 7. Hirst, G. 2003. Paraphrasing paraphrased. Invited talk at the ACL International Workshop on Paraphrasing, Sapporo. [find](https://scholar.google.com/scholar?q=Hirst%2C+G%2E+2003%2E+Paraphrasing+paraphrased%2E+Invited+talk+at+the+ACL+International+Workshop+on+Paraphrasing%2C+Sapporo%2E) 8. Honeck, Richard P. 1971. A study of paraphrases. Journal of Verbal Learning and Verbal Behavior, 10(4):367–381. [↗](https://doi.org/10.1016/s0022-5371(71)80035-x) 9. Huang, S., D. Graff, and G. Doddington. 2002. Multiple-translation Chinese corpus. Linguistic Data Consortium, Philadelphia, PA. [find](https://scholar.google.com/scholar?q=Huang%2C+S%2E%2C+D%2E+Graff%2C+and+G%2E+Doddington%2E+2002%2E+Multiple%2Dtranslation+Chinese+corpus%2E+Linguistic+Data+Consortium%2C+Philadelphia%2C+PA%2E) 10. Lin, D. and P. Pantel. 2001. Dirt: Discovery of inference rules from text. In ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 323–328, San Francisco, CA. [find](https://scholar.google.com/scholar?q=Lin%2C+D%2E+and+P%2E+Pantel%2E+2001%2E+Dirt%3A+Discovery+of+inference+rules+from+text%2E+In+ACM+SIGKDD+International+Conference+on+Knowledge+Discovery+and+Data+Mining%2C+pages+323%E2%80%93328%2C+San+Francisco%2C+CA%2E) 11. Mel’cuk, I. 1996. Lexical functions: A tool for description of lexical relations in a lexicon. In Leo Wanner, editor, Lexical Functions in Lexicography and Natural Language Processing. John Benjamins Publishing Co., Philadelphia, PA, pages 37–102. [find](https://scholar.google.com/scholar?q=Mel%E2%80%99cuk%2C+I%2E+1996%2E+Lexical+functions%3A+A+tool+for+description+of+lexical+relations+in+a+lexicon%2E+In+Leo+Wanner%2C+editor%2C+Lexical+Functions+in+Lexicography+and+Natural+Language+Processing%2E+John+Benjamins+) 12. Mel’cuk, I., 2012. Semantics: From Meaning to Text. John Benjamins Publishing Co., Philadelphia, PA. [find](https://scholar.google.com/scholar?q=Mel%E2%80%99cuk%2C+I%2E%2C+2012%2E+Semantics%3A+From+Meaning+to+Text%2E+John+Benjamins+Publishing+Co%2E%2C+Philadelphia%2C+PA%2E) 13. Siegal, S. and N. J. Castellan, Jr. 1988. Nonparametric Statistics for the Behavioral Sciences. McGraw-Hill, Columbus, OH. [find](https://scholar.google.com/scholar?q=Siegal%2C+S%2E+and+N%2E+J%2E+Castellan%2C+Jr%2E+1988%2E+Nonparametric+Statistics+for+the+Behavioral+Sciences%2E+McGraw%2DHill%2C+Columbus%2C+OH%2E)