---
title: "English WordNet Random Walk Pseudo-Corpora — Klubička 2020"
description: "This resource paper builds on the random-walk idea of Goikoetxea et al. A walk moves only along… 30 references, cited by 1 paper here."
authors: ["Filip Klubička", "Alfredo Maldonado", "Abhijit Mahalunkar", "John Kelleher"]
year: 2020
venue: "Proceedings of the Twelfth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2020.lrec-1.602/"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/klubicka-2020-english-wordnet-random-walk-pseudo/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/klubicka-2020-english-wordnet-random-walk-pseudo/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/10/) / English WordNet Random Walk Pseudo-Corpora

# English WordNet Random Walk Pseudo-Corpora

**Authors:** Filip Klubička, Alfredo Maldonado, Abhijit Mahalunkar and [John Kelleher](https://md.research.elamur.ai/free/en/authors/john-kelleher/) · **Year:** 2020 · **Venue:** [Proceedings of the Twelfth Language Resources and Evaluation Conference](https://md.research.elamur.ai/free/en/venues/lrec/) ([LREC 2020](https://md.research.elamur.ai/free/en/venues/lrec/2020/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.lrec-1.602/) · [PDF](https://aclanthology.org/2020.lrec-1.602.pdf) · [OpenAlex](https://openalex.org/W2987754621)

**Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Most word embeddings and language models learn from natural text. Co-occurrence in text mainly captures thematic relatedness, such as words that play complementary roles in one event. Taxonomic relatedness, meaning shared category membership, is rarely stated outright in text and lives mostly in curated resources like WordNet. This resource paper builds on the random-walk idea of Goikoetxea et al. A walk moves only along WordNet's hypernym/hyponym links and emits one lemma from each synset it visits. Each finished walk becomes a pseudo-sentence, and a set of walks forms a pseudo-corpus that ordinary embedding tools such as Skip-Gram can train on without modification. The authors keep the dampening factor α at 0.85 and add two controls of their own: a direction constraint (up, down or both) and a minimum sentence length. They show that changing these settings alters the statistical character of the resulting corpora. They release 81 pseudo-corpora and the generator code. This gives a practical way to inject taxonomic knowledge into standard distributional pipelines.

## Key points
- Walks traverse only the WordNet taxonomy (hypernym/hyponym edges), so the edges can be treated as directed. Goikoetxea et al. instead treated a mix of relation types as undirected.
- Three hyperparameters control generation:
  - The dampening factor α is fixed at 0.85, because varying it barely affected sentence length.
  - Directionality can be up, down or both.
  - Minimum sentence length n rejects walks shorter than n.
- The length filter is needed because many synsets sit outside the taxonomy. Walks starting there yield one-word "sentences" that carry no relational signal.
- 81 pseudo-corpora are published, plus a Python codebase built on NLTK (GitHub: GreenParachute/wordnet-randomwalk-python), so further combinations can be generated.
- Related work cited: Goikoetxea et al. (2016) found that plain concatenation of text-trained and random-walk embeddings worked best. Random-walk embeddings outperformed corpus embeddings on SimLex-999 at the time.

## Interesting ideas
- Turning a graph into ordinary token sequences lets unchanged sequence learners absorb graph structure, with no new objective function needed.
- Restricting walk direction (only toward hypernyms, only toward hyponyms, or both) controls what kind of relation the output encodes.
- A simple length threshold removes degenerate samples that start and end on isolated nodes, which would otherwise flood the data with uninformative items.
- The authors release the generator as well as fixed outputs, because the hyperparameter space was not fully explored. This keeps the resource extensible and the experiments reproducible.
- The authors profile generated corpora statistically, for example by the share of rare words, before using them. This surfaces how generation settings distort the data.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (world): The WordNet/FrameNet ontology could be serialized as walk-sequences. These would be cheap auxiliary training input for any learned component that must respect class hierarchies, while the explicit graph stays the explainable source of truth.
- **I2** (answer): Up-only walks give generalization chains (dog → canine → animal), which could serve as human-readable justifications when a reader answers category questions.
- **I3** (learn): Silver-data generation could use a similar minimum-content filter, dropping trivially short or unconnected examples before training.
- **I4** (learn): Synthetic training sets should be stored as versioned generator code plus settings rather than as frozen files, so variants can be regenerated and compared.
- **I5** (back): Text generated from the meaning graph could be profiled the same way and compared against natural text, to detect unnatural lexical distributions in generation or compression output.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Trinity College Dublin](https://md.research.elamur.ai/free/en/universities/trinity-college-dublin/) — University, Dublin, Ireland · 24 works · 1,231 citations
- [Technological University Dublin](https://md.research.elamur.ai/free/en/universities/technological-university-dublin/) — University, Dublin, Ireland · 5 works · 188 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 2,263 works

## Cite

Cite as: Filip Klubička, Alfredo Maldonado, Abhijit Mahalunkar and John Kelleher. 2020. English WordNet Random Walk Pseudo-Corpora. Proceedings of the Twelfth Language Resources and Evaluation Conference.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{klubicka2020english,
  title = {{English WordNet Random Walk Pseudo-Corpora}},
  author = {Filip Klubička, Alfredo Maldonado, Abhijit Mahalunkar and John Kelleher},
  booktitle = {Proceedings of the Twelfth Language Resources and Evaluation Conference},
  year = {2020},
  doi = {10.21427/he55-6481},
  url = {https://doi.org/10.21427/he55-6481}
}
```

- [← previous Embed More Ignore Less (EMIL): Exploiting Enriched Representations for Arabic NLP](https://md.research.elamur.ai/free/en/papers/younes-2020-embed-more-ignore-less-emil/)
- [next → Extending ImageNet to Arabic using Arabic WordNet](https://md.research.elamur.ai/free/en/papers/alsudais-2020-extending-imagenet-arabic-arabic-wordnet/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/10/)

## References

30 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Agirre, E., Cuadros, M., Rigau, G., and Soroa, A. (2010). Exploring knowledge bases for similarity. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’10). [find](https://scholar.google.com/scholar?q=Agirre%2C+E%2E%2C+Cuadros%2C+M%2E%2C+Rigau%2C+G%2E%2C+and+Soroa%2C+A%2E+%282010%29%2E+Exploring+knowledge+bases+for+similarity%2E+In+Proceedings+of+the+Eight+International+Conference+on+Language+Resources+and+Evaluation+%28LREC%E2%80%9910%29%2E)
2. Altmann, E. G. and Gerlach, M., (2016). Statistical Laws in Linguistics, pages 7–26. Springer International Publishing, Cham. [find](https://scholar.google.com/scholar?q=Altmann%2C+E%2E+G%2E+and+Gerlach%2C+M%2E%2C+%282016%29%2E+Statistical+Laws+in+Linguistics%2C+pages+7%E2%80%9326%2E+Springer+International+Publishing%2C+Cham%2E)
3. Bird, S. and Loper, E. (2004). [Nltk: the natural language toolkit](https://md.research.elamur.ai/free/en/papers/bird-2006-nltk-natural-language-toolkit/). In Proceedings of the ACL 2004 on Interactive poster and demonstration sessions, page 31. Association for Computational Linguistics.
4. Devlin, J., Chang, M., Lee, K., and Toutanova, K. (2018). [BERT: pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). CoRR, abs/1810.04805. [↗](https://arxiv.org/abs/1810.04805)
5. Ebeling, W. and Pöschel, T. (1994). Entropy and long-range correlations in literary english. EPL (Europhysics Letters), 26(4):241. [↗](https://doi.org/10.1209/0295-5075/26/4/001)
6. Faruqui, M. and Dyer, C. (2015). Non-distributional Word Vector Representations. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Short Papers), pages 464–469, Beijing. [↗](https://doi.org/10.48550/arxiv.1506.05230)
7. Fellbaum, C. (1998). WordNet: An Electronic Lexical Database. MIT Press, Cambridge, MA. [find](https://scholar.google.com/scholar?q=Fellbaum%2C+C%2E+%281998%29%2E+WordNet%3A+An+Electronic+Lexical+Database%2E+MIT+Press%2C+Cambridge%2C+MA%2E)
8. Francis, W. N. (1964). A standard sample of present-day english for use with digital computers. [find](https://scholar.google.com/scholar?q=Francis%2C+W%2E+N%2E+%281964%29%2E+A+standard+sample+of+present%2Dday+english+for+use+with+digital+computers%2E)
9. Gerlach, M. and Altmann, E. (2014). Scaling laws and fluctuations in the statistics of word frequencies. New Journal of Physics, 16:113010, 11. [↗](https://doi.org/10.1088/1367-2630/16/11/113010)
10. Goikoetxea, J., Soroa, A., and Agirre, E. (2015). Random Walks and Neural Network Language Models on Knowledge Bases. In Human Language Technologies: The 2015 Conference of the North American Chapter of the Association for Computational Linguistics, pages 1434–1439, Denver, CO. [find](https://scholar.google.com/scholar?q=Goikoetxea%2C+J%2E%2C+Soroa%2C+A%2E%2C+and+Agirre%2C+E%2E+%282015%29%2E+Random+Walks+and+Neural+Network+Language+Models+on+Knowledge+Bases%2E+In+Human+Language+Technologies%3A+The+2015+Conference+of+the+North+American+Chapter+)
11. Goikoetxea, J., Agirre, E., and Soroa, A. (2016). Single or multiple? combining word representations independently learned from text and wordnet. In AAAI. [find](https://scholar.google.com/scholar?q=Goikoetxea%2C+J%2E%2C+Agirre%2C+E%2E%2C+and+Soroa%2C+A%2E+%282016%29%2E+Single+or+multiple%3F+combining+word+representations+independently+learned+from+text+and+wordnet%2E+In+AAAI%2E)
12. Hearst, M. A. (1992). [Automatic acquisition of hyponyms from large text corpora](https://md.research.elamur.ai/free/en/papers/hearst-1992-automatic-acquisition-hyponyms-large-text/). In Proceedings of the 14th conference on Computational linguistics-Volume 2, pages 539–545. Association for Computational Linguistics.
13. Hill, F., Reichart, R., and Korhonen, A. (2015). [SimLex-999: Evaluating Semantic Models With (Genuine) Similarity Estimation](https://md.research.elamur.ai/free/en/papers/hill-2015-simlex-999-evaluating-semantic-models/). Computational Linguistics, 41(4):665–695. [↗](https://doi.org/10.1162/coli_a_00237)
14. Kacmajor, M. and Kelleher, J. D. (2019). Capturing and measuring thematic relatedness. Language Resources and Evaluation. [↗](https://doi.org/10.1007/s10579-019-09452-w)
15. Klubička, F., Maldonado, A., and Kelleher, J. (2019). Synthetic, yet natural: Properties of wordnet random walk corpora and the impact of rare words on embedding performance. In Proceedings of GWC2019: 10th Global WordNet Conference. [find](https://scholar.google.com/scholar?q=Klubic%CC%8Cka%2C+F%2E%2C+Maldonado%2C+A%2E%2C+and+Kelleher%2C+J%2E+%282019%29%2E+Synthetic%2C+yet+natural%3A+Properties+of+wordnet+random+walk+corpora+and+the+impact+of+rare+words+on+embedding+performance%2E+In+Proceedings+of+GWC201)
16. Lazaridou, A., Marelli, M., and Baroni, M. (2017). Multimodal word meaning induction from minimal exposure to natural text. Cognitive science, 41:677–705. [↗](https://doi.org/10.1111/cogs.12481)
17. Maldonado, A., Klubička, F., and Kelleher, J. D. (2019). Size matters: The impact of training size in taxonomically-enriched word embeddings. Open Computer Science. [↗](https://doi.org/10.1515/comp-2019-0009)
18. Merity, S., Xiong, C., Bradbury, J., and Socher, R. (2016). [Pointer sentinel mixture models](https://md.research.elamur.ai/free/en/papers/merity-2016-pointer-sentinel-mixture-models/). arXiv preprint arXiv:1609.07843. [↗](https://arxiv.org/abs/1609.07843)
19. Mikolov, T., Stutskever, I., Chen, K., Corrado, G., and Dean, J. (2013). [Distributed Representations of Words and Phrases and their Compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). In Proceedings of the Twenty-Seventh Annual Conference on Neural Information Processing Systems (NIPS) In Advances in Neural Information Processing Systems 26, pages 3111– 3119, Lake Tahoe, NV. [↗](https://doi.org/10.48550/arxiv.1310.4546)
20. Nickel, M. and Kiela, D. (2017). [Poincaré Embeddings for Learning Hierarchical Representations](https://md.research.elamur.ai/free/en/papers/nickel-2017-poincare-embeddings-learning-hierarchical-representations/). In I. Guyon, et al., editors, Advances in Neural Information Processing Systems 30, pages 6338–6347. Curran Associates, Inc., Long Beach, CA. [↗](https://doi.org/10.48550/arxiv.1705.08039)
21. Pagliardini, M., Gupta, P., and Jaggi, M. (2018). [Unsupervised learning of sentence embeddings using compositional n-gram features](https://md.research.elamur.ai/free/en/papers/pagliardini-2018-unsupervised-learning-sentence-embeddings-compositional/). In Proceedings of NAACL-HLT, pages 528–540.
22. Peters, M. E., Neumann, M., Iyyer, M., Gardner, M., Clark, C., Lee, K., and Zettlemoyer, L. (2018). [Deep contextualized word representations](https://md.research.elamur.ai/free/en/papers/peters-2018-deep-contextualized-word-representations/). CoRR, abs/1802.05365. [↗](https://arxiv.org/abs/1802.05365)
23. Salton, G., Ross, R., and Kelleher, J. (2017). Attentive language models. In Proceedings of the Eighth International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 441–450. [find](https://scholar.google.com/scholar?q=Salton%2C+G%2E%2C+Ross%2C+R%2E%2C+and+Kelleher%2C+J%2E+%282017%29%2E+Attentive+language+models%2E+In+Proceedings+of+the+Eighth+International+Joint+Conference+on+Natural+Language+Processing+%28Volume+1%3A+Long+Papers%29%2C+pages+441%E2%80%93)
24. Simov, K., Popov, A., and Osenova, P. (2015). Improving word sense disambiguation with linguistic knowledge from a sense annotated treebank. In Proceedings of the International Conference Recent Advances in Natural Language Processing, pages 596–603. [find](https://scholar.google.com/scholar?q=Simov%2C+K%2E%2C+Popov%2C+A%2E%2C+and+Osenova%2C+P%2E+%282015%29%2E+Improving+word+sense+disambiguation+with+linguistic+knowledge+from+a+sense+annotated+treebank%2E+In+Proceedings+of+the+International+Conference+Recent+Advan)
25. Simov, K., Popov, A., and Osenova, P. (2016a). The role of the wordnet relations in the knowledge-based word sense disambiguation task. In Proceedings of Eighth Global WordNet Conference, pages 391–398. [find](https://scholar.google.com/scholar?q=Simov%2C+K%2E%2C+Popov%2C+A%2E%2C+and+Osenova%2C+P%2E+%282016a%29%2E+The+role+of+the+wordnet+relations+in+the+knowledge%2Dbased+word+sense+disambiguation+task%2E+In+Proceedings+of+Eighth+Global+WordNet+Conference%2C+pages+391%E2%80%9339)
26. Simov, K. I., Osenova, P., and Popov, A. (2016b). Using context information for knowledge-based word sense disambiguation. In AIMSA. [find](https://scholar.google.com/scholar?q=Simov%2C+K%2E+I%2E%2C+Osenova%2C+P%2E%2C+and+Popov%2C+A%2E+%282016b%29%2E+Using+context+information+for+knowledge%2Dbased+word+sense+disambiguation%2E+In+AIMSA%2E)
27. Simov, K., Osenova, P., and Popov, A. (2017a). Comparison of word embeddings from different knowledge graphs. In International Conference on Language, Data and Knowledge, pages 213–221. Springer. [find](https://scholar.google.com/scholar?q=Simov%2C+K%2E%2C+Osenova%2C+P%2E%2C+and+Popov%2C+A%2E+%282017a%29%2E+Comparison+of+word+embeddings+from+different+knowledge+graphs%2E+In+International+Conference+on+Language%2C+Data+and+Knowledge%2C+pages+213%E2%80%93221%2E+Springer%2E)
28. Simov, K. I., Boytcheva, S., and Osenova, P. (2017b). Towards lexical chains for knowledge-graph-based word embeddings. In RANLP, pages 679–685. [find](https://scholar.google.com/scholar?q=Simov%2C+K%2E+I%2E%2C+Boytcheva%2C+S%2E%2C+and+Osenova%2C+P%2E+%282017b%29%2E+Towards+lexical+chains+for+knowledge%2Dgraph%2Dbased+word+embeddings%2E+In+RANLP%2C+pages+679%E2%80%93685%2E)
29. Takahashi, S. and Tanaka-Ishii, K. (2019). Evaluating computational language models with scaling properties of natural language. Computational Linguistics, 45(3):481–513. [↗](https://doi.org/10.1162/coli_a_00355)
30. Zipf, G. K. (1949). Human behavior and the principle of least effort: An introduction to human ecology, volume 47. 01. [find](https://scholar.google.com/scholar?q=Zipf%2C+G%2E+K%2E+%281949%29%2E+Human+behavior+and+the+principle+of+least+effort%3A+An+introduction+to+human+ecology%2C+volume+47%2E+01%2E)

*Updated 1 November 2026*