---
title: "Exemplification Modeling — Barba 2021"
description: "The authors introduce a new task called Exemplification Modeling (ExMod)."
authors: ["Edoardo Barba", "Luigi Procopio", "Caterina Lacerra"]
year: 2021
venue: "IJCAI 2021"
source: "https://www.ijcai.org/proceedings/2021/520"
doi: "10.24963/ijcai.2021/520"
licence: "free access (IJCAI proceedings)"
licence_url: "https://www.ijcai.org/proceedings/2021/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/barba-2021-exemplification-modeling/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/barba-2021-exemplification-modeling/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/2/) / Exemplification Modeling: Can You Give Me an Example, Please?

# Exemplification Modeling: Can You Give Me an Example, Please?

**Authors:** [Edoardo Barba](https://md.research.elamur.ai/free/en/authors/edoardo-barba/), Luigi Procopio, Caterina Lacerra · **Year:** 2021 · **Venue:** IJCAI 2021 · **Licence:** [free access (IJCAI proceedings)](https://www.ijcai.org/proceedings/2021/)

**Links:** [Publisher's page](https://doi.org/10.24963/ijcai.2021/520) · [PDF](https://www.ijcai.org/proceedings/2021/0520.pdf) · [DOI 10.24963/ijcai.2021/520](https://doi.org/10.24963/ijcai.2021/520) · [OpenAlex](https://openalex.org/W3191678939)

**Citations:** 8 citations (openalex-snapshot · 2026-10-14) · 15 in our library, of the 140673 papers whose bibliographies we read

## Summary
The authors introduce a new task called Exemplification Modeling (ExMod). A system is given one or more words together with their definitions, and it must compose a sentence in which each word is used precisely in the given sense. In other words, this is the inverse of Definition Modeling. For this task, ExMaker, an encoder-decoder model, is proposed. As input, it receives "lemma — definition" pairs marked with special tokens, and as output it produces a sentence with the target words marked. Lemmatization then recovers the positions at which these words occur. The model is trained on existing annotated corpora. For sense pairs, examples are selected in which both words occur close to each other and form a statistically significant collocation. Quality is assessed "in vivo": the generated sentences are added to the training data for Word Sense Disambiguation. With these data, the WSD model surpasses the previous state of the art as well as the result of training on gold corpora alone. Annotators rated the fluency and semantic adequacy of the sentences almost on par with manually built corpora. The practical value of the approach is that sense-tagged data can be generated on demand for the entire sense inventory.

## Key points
- **ExMaker architecture.** This is a sequence-to-sequence model. Lemmas in the input are wrapped in `<s>`/`</s>` tokens, and target spans in the sentence are wrapped in `<t>`/`</t>` markers. The model is trained with cross-entropy. It can generate examples for three senses at once, although only sets of one or two senses (k ∈ {1, 2}) were used during training.
- **Resources.** The sense inventory is WordNet. Training data: SemCor (33K sentences, 200K instances), Princeton WordNet Gloss Corpus (WNG, 34,275 examples), and Oxford Dictionary Dataset (555,695 sentences with ODE senses). Sense collocations are taken from SyntagNet.
- **Example selection for sense pairs.** Examples in which the two words are more than ξ tokens apart are discarded. The remaining ones are sampled with probability proportional to PPMI; the word frequencies for it are computed on English Wikipedia. Smoothing with α = 0.15 is additionally applied.
- **Evaluation.** The evaluation consists of 5 WSD subtasks. Among them is a zero-shot setting: the model is trained on Oxford senses, and examples are generated for WordNet senses. Examples are generated for all lemma–definition pairs in WordNet. In this way, the authors partially address the problem that existing corpora cover less than 20% of WordNet senses.
- **Human evaluation and open code.** The generated sentences are almost as fluent as those in manually built corpora, and their adequacy to the given sense is only slightly lower. The model, datasets, and code are released at github.com/SapienzaNLP/exmod.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/)

Topics: [Word sense disambiguation](https://md.research.elamur.ai/free/en/topics/word-sense-disambiguation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{barba2021exemplification,
  title = {{Exemplification Modeling: Can You Give Me an Example, Please?}},
  author = {Barba, Edoardo and Procopio, Luigi and Lacerra, Caterina},
  booktitle = {IJCAI 2021},
  year = {2021},
  doi = {10.24963/ijcai.2021/520},
  url = {https://doi.org/10.24963/ijcai.2021/520}
}
```

- [← previous Event Coreference Data (Almost) for Free: Mining Hyperlinks from Online News](https://md.research.elamur.ai/free/en/papers/bugert-2021-event-coreference-data-almost-free/)
- [next → Expanding the JHU Bible Corpus for Machine Translation of the Indigenous Languages of North America](https://md.research.elamur.ai/free/en/papers/nicolai-2021-expanding-jhu-bible-corpus-machine/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/2/)

## References

32 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Edoardo Barba, Luigi Procopio, Niccolò Campolungo, Tommaso Pasini, and Roberto Navigli. MuLaN: Multilingual Label propagatioN for Word Sense Disambiguation. In Proc. of IJCAI, 2020. [find](https://scholar.google.com/scholar?q=Edoardo+Barba%2C+Luigi+Procopio%2C+Niccolo%CC%80+Campolungo%2C+Tommaso+Pasini%2C+and+Roberto+Navigli%2E+MuLaN%3A+Multilingual+Label+propagatioN+for+Word+Sense+Disambiguation%2E+In+Proc%2E+of+IJCAI%2C+2020%2E)
2. Edoardo Barba, Tommaso Pasini, and Roberto Navigli. ESC: Redesigning WSD with extractive sense comprehension. In Proc. of NAACL, pages 4661–4672, 2021. [find](https://scholar.google.com/scholar?q=Edoardo+Barba%2C+Tommaso+Pasini%2C+and+Roberto+Navigli%2E+ESC%3A+Redesigning+WSD+with+extractive+sense+comprehension%2E+In+Proc%2E+of+NAACL%2C+pages+4661%E2%80%934672%2C+2021%2E)
3. Michele Bevilacqua and Roberto Navigli. Breaking through the 80% glass ceiling: Raising the state of the art in word sense disambiguation by incorporating knowledge graph information. In Proc. of ACL, pages 2854– 2864, 2020. [find](https://scholar.google.com/scholar?q=Michele+Bevilacqua+and+Roberto+Navigli%2E+Breaking+through+the+80%25+glass+ceiling%3A+Raising+the+state+of+the+art+in+word+sense+disambiguation+by+incorporating+knowledge+graph+information%2E+In+Proc%2E+of+ACL%2C)
4. Michele Bevilacqua, Marco Maru, and Roberto Navigli. Generationary or: “how we went beyond word sense inventories and learned to gloss”. In Proc. of EMNLP, pages 7207–7221, 2020. [find](https://scholar.google.com/scholar?q=Michele+Bevilacqua%2C+Marco+Maru%2C+and+Roberto+Navigli%2E+Generationary+or%3A+%E2%80%9Chow+we+went+beyond+word+sense+inventories+and+learned+to+gloss%E2%80%9D%2E+In+Proc%2E+of+EMNLP%2C+pages+7207%E2%80%937221%2C+2020%2E) [our card](https://md.research.elamur.ai/free/en/papers/lex-2020-bevilacqua-generationary/)
5. Michele Bevilacqua, Tommaso Pasini, Alessandro Raganato, and Roberto Navigli. Recent trends in Word Sense Disambiguation: A survey. In Proc. of IJCAI, 2021. [find](https://scholar.google.com/scholar?q=Michele+Bevilacqua%2C+Tommaso+Pasini%2C+Alessandro+Raganato%2C+and+Roberto+Navigli%2E+Recent+trends+in+Word+Sense+Disambiguation%3A+A+survey%2E+In+Proc%2E+of+IJCAI%2C+2021%2E)
6. Terra Blevins and Luke Zettlemoyer. Moving down the long tail of word sense disambiguation with gloss informed bi-encoders. In Proc. of ACL, pages 1006– 1017, 2020. [find](https://scholar.google.com/scholar?q=Terra+Blevins+and+Luke+Zettlemoyer%2E+Moving+down+the+long+tail+of+word+sense+disambiguation+with+gloss+informed+bi%2Dencoders%2E+In+Proc%2E+of+ACL%2C+pages+1006%E2%80%93+1017%2C+2020%2E)
7. Ting-Yun Chang, Ta-Chung Chi, Shang-Chi Tsai, and Yun-Nung Chen. xSense: Learning sense-separated sparse representations and textual definitions for explainable word sense networks. arXiv preprint arXiv:1809.03348, 2018. — [arXiv:1809.03348](https://arxiv.org/abs/1809.03348) [find](https://scholar.google.com/scholar?q=Ting%2DYun+Chang%2C+Ta%2DChung+Chi%2C+Shang%2DChi+Tsai%2C+and+Yun%2DNung+Chen%2E+xSense%3A+Learning+sense%2Dseparated+sparse+representations+and+textual+definitions+for+explainable+word+sense+networks%2E+arXiv+preprint+arX)
8. Jacob Cohen. A coefficient of agreement for nominal scales. Educational and psychological measurement, 20(1):37–46, 1960. [find](https://scholar.google.com/scholar?q=Jacob+Cohen%2E+A+coefficient+of+agreement+for+nominal+scales%2E+Educational+and+psychological+measurement%2C+20%281%29%3A37%E2%80%9346%2C+1960%2E)
9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proc. of NAACL, pages 4171–4186, 2019. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proc%2E+of+NAACL%2C+pages+4171%E2%80%934186%2C+2019%2E)
10. Artyom Gadetsky, Ilya Yakubovskiy, and Dmitry Vetrov. Conditional generators of words definitions. In Proc. of ACL, pages 266–271, 2018. [find](https://scholar.google.com/scholar?q=Artyom+Gadetsky%2C+Ilya+Yakubovskiy%2C+and+Dmitry+Vetrov%2E+Conditional+generators+of+words+definitions%2E+In+Proc%2E+of+ACL%2C+pages+266%E2%80%93271%2C+2018%2E)
11. Shonosuke Ishiwatari, Hiroaki Hayashi, Naoki Yoshinaga, Graham Neubig, Shoetsu Sato, Masashi Toyoda, and Masaru Kitsuregawa. Learning to describe unknown phrases with local and global contexts. In Proc. of NAACL, pages 3467–3476, 2019. [find](https://scholar.google.com/scholar?q=Shonosuke+Ishiwatari%2C+Hiroaki+Hayashi%2C+Naoki+Yoshinaga%2C+Graham+Neubig%2C+Shoetsu+Sato%2C+Masashi+Toyoda%2C+and+Masaru+Kitsuregawa%2E+Learning+to+describe+unknown+phrases+with+local+and+global+contexts%2E+In+Pro)
12. J Richard Landis and Gary G Koch. The measurement of observer agreement for categorical data. biometrics, pages 159–174, 1977. [find](https://scholar.google.com/scholar?q=J+Richard+Landis+and+Gary+G+Koch%2E+The+measurement+of+observer+agreement+for+categorical+data%2E+biometrics%2C+pages+159%E2%80%93174%2C+1977%2E)
13. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. BART: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proc. of ACL, pages 7871–7880, 2020. [find](https://scholar.google.com/scholar?q=Mike+Lewis%2C+Yinhan+Liu%2C+Naman+Goyal%2C+Marjan+Ghazvininejad%2C+Abdelrahman+Mohamed%2C+Omer+Levy%2C+Veselin+Stoyanov%2C+and+Luke+Zettlemoyer%2E+BART%3A+Denoising+sequence%2Dto%2Dsequence+pre%2Dtraining+for+natural+languag) [our card](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/)
14. Chia-Wei Liu, Ryan Lowe, Iulian Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Proc. of EMNLP, pages 2122–2132, 2016. [find](https://scholar.google.com/scholar?q=Chia%2DWei+Liu%2C+Ryan+Lowe%2C+Iulian+Serban%2C+Mike+Noseworthy%2C+Laurent+Charlin%2C+and+Joelle+Pineau%2E+How+NOT+to+evaluate+your+dialogue+system%3A+An+empirical+study+of+unsupervised+evaluation+metrics+for+dialogu)
15. Liyuan Liu, Haoming Jiang, Pengcheng He, Weizhu Chen, Xiaodong Liu, Jianfeng Gao, and Jiawei Han. On the variance of the adaptive learning rate and beyond. In ICLR, 2020. [find](https://scholar.google.com/scholar?q=Liyuan+Liu%2C+Haoming+Jiang%2C+Pengcheng+He%2C+Weizhu+Chen%2C+Xiaodong+Liu%2C+Jianfeng+Gao%2C+and+Jiawei+Han%2E+On+the+variance+of+the+adaptive+learning+rate+and+beyond%2E+In+ICLR%2C+2020%2E)
16. Marco Maru, Federico Scozzafava, Federico Martelli, and Roberto Navigli. SyntagNet: Challenging supervised word sense disambiguation with lexical-semantic combinations. In Proc. of EMNLP, pages 3534–3540, 2019. [find](https://scholar.google.com/scholar?q=Marco+Maru%2C+Federico+Scozzafava%2C+Federico+Martelli%2C+and+Roberto+Navigli%2E+SyntagNet%3A+Challenging+supervised+word+sense+disambiguation+with+lexical%2Dsemantic+combinations%2E+In+Proc%2E+of+EMNLP%2C+pages+3534%E2%80%933)
17. Timothee Mickus, Denis Paperno, and Matthieu Constant. Mark my word: A sequence-to-sequence approach to definition modeling. In Proc. of DeepNLP, 2019. [find](https://scholar.google.com/scholar?q=Timothee+Mickus%2C+Denis+Paperno%2C+and+Matthieu+Constant%2E+Mark+my+word%3A+A+sequence%2Dto%2Dsequence+approach+to+definition+modeling%2E+In+Proc%2E+of+DeepNLP%2C+2019%2E)
18. George A. Miller, R.T. Beckwith, Christiane D. Fellbaum, D. Gross, and K. Miller. Introduction to WordNet: an online lexical database. International Journal of Lexicography, 3(4):235–244, 1990. [find](https://scholar.google.com/scholar?q=George+A%2E+Miller%2C+R%2ET%2E+Beckwith%2C+Christiane+D%2E+Fellbaum%2C+D%2E+Gross%2C+and+K%2E+Miller%2E+Introduction+to+WordNet%3A+an+online+lexical+database%2E+International+Journal+of+Lexicography%2C+3%284%29%3A235%E2%80%93244%2C+1990%2E)
19. George A Miller, Claudia Leacock, Randee Tengi, and Ross T Bunker. A semantic concordance. In Proc. of Human Language Technology, pages 303–308, 1993. [find](https://scholar.google.com/scholar?q=George+A+Miller%2C+Claudia+Leacock%2C+Randee+Tengi%2C+and+Ross+T+Bunker%2E+A+semantic+concordance%2E+In+Proc%2E+of+Human+Language+Technology%2C+pages+303%E2%80%93308%2C+1993%2E)
20. Andrea Moro and Roberto Navigli. SemEval-2015 task 13: Multilingual all-words sense disambiguation and entity linking. In Proc. of SemEval, pages 288– 297, 2015. [find](https://scholar.google.com/scholar?q=Andrea+Moro+and+Roberto+Navigli%2E+SemEval%2D2015+task+13%3A+Multilingual+all%2Dwords+sense+disambiguation+and+entity+linking%2E+In+Proc%2E+of+SemEval%2C+pages+288%E2%80%93+297%2C+2015%2E)
21. Ian SP Nation. Learning vocabulary in another language. Cambridge University Press, 2001. [find](https://scholar.google.com/scholar?q=Ian+SP+Nation%2E+Learning+vocabulary+in+another+language%2E+Cambridge+University+Press%2C+2001%2E)
22. Roberto Navigli, David Jurgens, and Daniele Vannella. SemEval-2013 task 12: Multilingual word sense disambiguation. In Proc. of SemEval, pages 222–231, 2013. [find](https://scholar.google.com/scholar?q=Roberto+Navigli%2C+David+Jurgens%2C+and+Daniele+Vannella%2E+SemEval%2D2013+task+12%3A+Multilingual+word+sense+disambiguation%2E+In+Proc%2E+of+SemEval%2C+pages+222%E2%80%93231%2C+2013%2E)
23. Roberto Navigli. Word Sense Disambiguation: A survey. ACM computing surveys (CSUR), 41(2):1–69, 2009. [find](https://scholar.google.com/scholar?q=Roberto+Navigli%2E+Word+Sense+Disambiguation%3A+A+survey%2E+ACM+computing+surveys+%28CSUR%29%2C+41%282%29%3A1%E2%80%9369%2C+2009%2E)
24. Yoshiki Niwa and Yoshihiko Nitta. Co-occurrence vectors from corpora vs. distance vectors from dictionaries. In Proc. of COLING, 1994. [find](https://scholar.google.com/scholar?q=Yoshiki+Niwa+and+Yoshihiko+Nitta%2E+Co%2Doccurrence+vectors+from+corpora+vs%2E+distance+vectors+from+dictionaries%2E+In+Proc%2E+of+COLING%2C+1994%2E)
25. Thanapon Noraset, Chen Liang, Larry Birnbaum, and Doug Downey. Definition modeling: Learning to define word embeddings in natural language. In Proc. of AAAI, 2017. [find](https://scholar.google.com/scholar?q=Thanapon+Noraset%2C+Chen+Liang%2C+Larry+Birnbaum%2C+and+Doug+Downey%2E+Definition+modeling%3A+Learning+to+define+word+embeddings+in+natural+language%2E+In+Proc%2E+of+AAAI%2C+2017%2E)
26. Martha Palmer, Christiane Fellbaum, Scott Cotton, Lauren Delfs, and Hoa Trang Dang. English tasks: All-words and verb lexical sample. In Proc. of Senseval-2, pages 21–24, 2001. [find](https://scholar.google.com/scholar?q=Martha+Palmer%2C+Christiane+Fellbaum%2C+Scott+Cotton%2C+Lauren+Delfs%2C+and+Hoa+Trang+Dang%2E+English+tasks%3A+All%2Dwords+and+verb+lexical+sample%2E+In+Proc%2E+of+Senseval%2D2%2C+pages+21%E2%80%9324%2C+2001%2E)
27. Sameer Pradhan, Edward Loper, Dmitriy Dligach, and Martha Palmer. SemEval-2007 task-17: English lexical sample, SRL and all words. In Proc. of SemEval, pages 87–92, 2007. [find](https://scholar.google.com/scholar?q=Sameer+Pradhan%2C+Edward+Loper%2C+Dmitriy+Dligach%2C+and+Martha+Palmer%2E+SemEval%2D2007+task%2D17%3A+English+lexical+sample%2C+SRL+and+all+words%2E+In+Proc%2E+of+SemEval%2C+pages+87%E2%80%9392%2C+2007%2E)
28. Alessandro Raganato, Jose Camacho-Collados, and Roberto Navigli. Word Sense Disambiguation: A unified evaluation framework and empirical comparison. In Proc. of EACL, pages 99–110, 2017. [find](https://scholar.google.com/scholar?q=Alessandro+Raganato%2C+Jose+Camacho%2DCollados%2C+and+Roberto+Navigli%2E+Word+Sense+Disambiguation%3A+A+unified+evaluation+framework+and+empirical+comparison%2E+In+Proc%2E+of+EACL%2C+pages+99%E2%80%93110%2C+2017%2E) [our card](https://md.research.elamur.ai/free/en/papers/onto-2017-raganato-word-sense-disambiguation-unified-evaluation/)
29. Bianca Scarlini, Tommaso Pasini, and Roberto Navigli. Just “OneSeC” for producing multilingual sense-annotated data. In Proc. of ACL, pages 699–709, 2019. [find](https://scholar.google.com/scholar?q=Bianca+Scarlini%2C+Tommaso+Pasini%2C+and+Roberto+Navigli%2E+Just+%E2%80%9COneSeC%E2%80%9D+for+producing+multilingual+sense%2Dannotated+data%2E+In+Proc%2E+of+ACL%2C+pages+699%E2%80%93709%2C+2019%2E)
30. Bianca Scarlini, Tommaso Pasini, and Roberto Navigli. With more contexts comes better performance: Contextualized sense embeddings for all-round word sense disambiguation. In Proc. of EMNLP, pages 3528–3539, 2020. [find](https://scholar.google.com/scholar?q=Bianca+Scarlini%2C+Tommaso+Pasini%2C+and+Roberto+Navigli%2E+With+more+contexts+comes+better+performance%3A+Contextualized+sense+embeddings+for+all%2Dround+word+sense+disambiguation%2E+In+Proc%2E+of+EMNLP%2C+pages+352)
31. Benjamin Snyder and Martha Palmer. The English all-words task. In Proc. of Senseval-3, pages 41– 43, 2004. [find](https://scholar.google.com/scholar?q=Benjamin+Snyder+and+Martha+Palmer%2E+The+English+all%2Dwords+task%2E+In+Proc%2E+of+Senseval%2D3%2C+pages+41%E2%80%93+43%2C+2004%2E)
32. Kaveh Taghipour and Hwee Tou Ng. One million sense-tagged instances for word sense disambiguation and induction. In Proc. of CoNLL, pages 338–344, 2015. [find](https://scholar.google.com/scholar?q=Kaveh+Taghipour+and+Hwee+Tou+Ng%2E+One+million+sense%2Dtagged+instances+for+word+sense+disambiguation+and+induction%2E+In+Proc%2E+of+CoNLL%2C+pages+338%E2%80%93344%2C+2015%2E)