---
title: "Towards Ontology Construction with Language Models — Funk 2023"
description: "The authors test whether a large language model can act as the expert and also propose the vocabulary itself. 26 references, cited by 6 papers here."
authors: ["Maurice Funk", "Simon Hosemann", "Jean Christoph Jung", "Carsten Lutz"]
year: 2023
venue: "arXiv:2309.09898"
source: "https://arxiv.org/abs/2309.09898"
doi: "10.48550/arxiv.2309.09898"
arxiv: "2309.09898"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/funk-2023-towards-ontology-construction-language-models/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/funk-2023-towards-ontology-construction-language-models/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/) / Towards Ontology Construction with Language Models

# Towards Ontology Construction with Language Models

**Authors:** Maurice Funk, Simon Hosemann, Jean Christoph Jung and Carsten Lutz · **Year:** 2023 · **Venue:** arXiv:2309.09898 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2309.09898) · [PDF](https://arxiv.org/pdf/2309.09898) · [DOI 10.48550/arxiv.2309.09898](https://doi.org/10.48550/arxiv.2309.09898) · [arXiv 2309.09898](https://arxiv.org/abs/2309.09898) · [OpenAlex](https://openalex.org/W4386876236)

**Citations:** 6 citations (OpenAlex, 2026-10-14) · 6 in our library, of the 140,884 papers whose bibliographies we read

## Summary

Building ontologies by hand is slow and expensive, and it usually needs both an ontology engineer and a domain expert. The authors test whether a large language model can act as the expert and also propose the vocabulary itself. They limit the task to the simplest backbone, a concept hierarchy that uses only the is-a relation. Their algorithm starts from one seed concept and queries GPT 3.5 about each concept already in the hierarchy: whether it has subcategories, which ones, and how each is described. Every candidate is checked with an extra verification query before it is inserted. The result can be a DAG rather than a tree. To improve coverage, they sample the first token of the listing answer many times and expand the frequent beginnings. Hierarchies were generated for several areas, among them Plants, Music, Drinks and Animals. There is no gold standard, so the evaluation was a manual, subjective inspection. The authors find the results reasonable, and verification and prompt design reduce hallucinations. The work suggests that LLMs can substantially assist human ontology engineers but not replace them.

## Key points

- **Pipeline:** the algorithm has five steps: existence, listing, description, verification and insertion. It explores only downward from the seed concept (e.g., Animals) so that it stays inside the domain.
- **Prompt wording:** the authors compared terms like subclass, subconcept and type. "Subcategory" gave the most convincing answers.
- **Coverage trick:**
  - The listing prompt is sent 100 times with `max_tokens` set to 1.
  - First tokens that appear between 5 and 20 times (the threshold is tuned) are each used to restart the listing ("Start your answer with t").
  - The union of the resulting lists is kept.
- **Uses of the descriptions:** the generated descriptions feed into later prompts and help humans tell near-duplicates apart, such as chocolate porters versus chocolaty porters.
- **Observed limitations:**
  - The hierarchies follow everyday rather than scientific taxonomies.
  - They show a Western/US cultural bias (e.g., Chai Tea treated as Spiced Tea).
  - The ontologies are released without manual post-processing.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [AI & machine learning](https://md.research.elamur.ai/free/en/shelves/ai/)

Topics: [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · [🕸️ Knowledge graphs & ontologies](https://md.research.elamur.ai/free/en/topics/knowledge-graphs/)

## Institutions

- [TU Dortmund University](https://md.research.elamur.ai/free/en/universities/tu-dortmund-university/) — University, Dortmund, Germany · 5 works · 9,745 citations
- [Leipzig University](https://md.research.elamur.ai/free/en/universities/leipzig-university/) — University, Leipzig, Germany · 12 works · 4,703 citations
- [Center for Scalable Data Analytics and Artificial Intelligence](https://md.research.elamur.ai/free/en/universities/center-for-scalable-data-analytics-and-artificial-intelligence/) — University, Dresden, Germany · 3 works · 13 citations

## Cite

Cite as: Maurice Funk, Simon Hosemann, Jean Christoph Jung and Carsten Lutz. 2023. Towards Ontology Construction with Language Models. arXiv:2309.09898.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{funk2023ontology,
  title = {{Towards Ontology Construction with Language Models}},
  author = {Maurice Funk, Simon Hosemann, Jean Christoph Jung and Carsten Lutz},
  year = {2023},
  doi = {10.48550/arxiv.2309.09898},
  eprint = {2309.09898},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2309.09898}
}
```

- [← previous MWE as WSD: Solving Multiword Expression Identification with Word Sense Disambiguation](https://md.research.elamur.ai/free/en/papers/tanner-2023-mwe-as-wsd-solving-multiword/)
- [next → Adversarial Network with External Knowledge for Zero-Shot Stance Detection](https://md.research.elamur.ai/free/en/papers/wang-2023-adversarial-network-external-knowledge-zero/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/)

## References

26 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. A. Ozaki, Learning description logic ontologies: Five approaches. where do they stand?, Künstliche Intell. 34 (2020) 317–327. URL: https://doi.org/10.1007/s13218-020-00656-9. doi: 10.1007/s13218-020-00656-9 . — [doi:10.1007/s13218-020-00656-9](https://doi.org/10.1007/s13218-020-00656-9) [↗](https://doi.org/10.1007/s13218-020-00656-9)
2. J. Vrolijk, I. Reklos, M. Vafaie, A. Massari, M. Mohammadi, S. Rudolph, Toward a comparison framework for interactive ontology enrichment methodologies, in: Proc. of VOILA@ISWC, volume 3253 of CEUR Workshop Proceedings, CEUR-WS.org, 2022, pp. 41–50. [find](https://scholar.google.com/scholar?q=J%2E+Vrolijk%2C+I%2E+Reklos%2C+M%2E+Vafaie%2C+A%2E+Massari%2C+M%2E+Mohammadi%2C+S%2E+Rudolph%2C+Toward+a+comparison+framework+for+interactive+ontology+enrichment+methodologies%2C+in%3A+Proc%2E+of+VOILA%40ISWC%2C+volume+3253+of+CEUR+Wo)
3. B. Konev, C. Lutz, A. Ozaki, F. Wolter, Exact learning of lightweight description logic ontologies, J. Mach. Learn. Res. 18 (2017) 201:1–201:63. [find](https://scholar.google.com/scholar?q=B%2E+Konev%2C+C%2E+Lutz%2C+A%2E+Ozaki%2C+F%2E+Wolter%2C+Exact+learning+of+lightweight+description+logic+ontologies%2C+J%2E+Mach%2E+Learn%2E+Res%2E+18+%282017%29+201%3A1%E2%80%93201%3A63%2E)
4. F. Baader, B. Ganter, B. Sertkaya, U. Sattler, Completing description logic knowledge bases using formal concept analysis, in: Proc. of IJCAI, 2007, pp. 230–235. [find](https://scholar.google.com/scholar?q=F%2E+Baader%2C+B%2E+Ganter%2C+B%2E+Sertkaya%2C+U%2E+Sattler%2C+Completing+description+logic+knowledge+bases+using+formal+concept+analysis%2C+in%3A+Proc%2E+of+IJCAI%2C+2007%2C+pp%2E+230%E2%80%93235%2E)
5. S. Rudolph, J. Völker, P. Hitzler, Supporting lexical ontology learning by relational exploration, in: Proc. of ICCS, volume 4604 of LNCS, Springer, 2007, pp. 488–491. [find](https://scholar.google.com/scholar?q=S%2E+Rudolph%2C+J%2E+V%C3%B6lker%2C+P%2E+Hitzler%2C+Supporting+lexical+ontology+learning+by+relational+exploration%2C+in%3A+Proc%2E+of+ICCS%2C+volume+4604+of+LNCS%2C+Springer%2C+2007%2C+pp%2E+488%E2%80%93491%2E)
6. T. Brown, B. Mann, N. Ryder, M. Subbiah, J. D. Kaplan, P. Dhariwal, A. Neelakantan, P. Shyam, G. Sastry, A. Askell, S. Agarwal, A. Herbert-Voss, G. Krueger, T. Henighan, R. Child, A. Ramesh, D. Ziegler, J. Wu, C. Winter, C. Hesse, M. Chen, E. Sigler, M. Litwin, S. Gray, B. Chess, J. Clark, C. Berner… [find](https://scholar.google.com/scholar?q=T%2E+Brown%2C+B%2E+Mann%2C+N%2E+Ryder%2C+M%2E+Subbiah%2C+J%2E+D%2E+Kaplan%2C+P%2E+Dhariwal%2C+A%2E+Neelakantan%2C+P%2E+Shyam%2C+G%2E+Sastry%2C+A%2E+Askell%2C+S%2E+Agarwal%2C+A%2E+Herbert%2DVoss%2C+G%2E+Krueger%2C+T%2E+Henighan%2C+R%2E+Child%2C+A%2E+Ramesh%2C+D%2E+Ziegle)
7. J. W. Rae, S. Borgeaud, T. Cai, K. Millican, J. Hoffmann, H. F. Song, J. Aslanides, S. Henderson, R. Ring, S. Young, E. Rutherford, T. Hennigan, J. Menick, A. Cassirer, R. Powell, G. van den Driessche, L. A. Hendricks, M. Rauh, P. Huang, A. Glaese, J. Welbl, S. Dathathri, S. Huang, J. Uesato, J. Mel… [find](https://scholar.google.com/scholar?q=J%2E+W%2E+Rae%2C+S%2E+Borgeaud%2C+T%2E+Cai%2C+K%2E+Millican%2C+J%2E+Hoffmann%2C+H%2E+F%2E+Song%2C+J%2E+Aslanides%2C+S%2E+Henderson%2C+R%2E+Ring%2C+S%2E+Young%2C+E%2E+Rutherford%2C+T%2E+Hennigan%2C+J%2E+Menick%2C+A%2E+Cassirer%2C+R%2E+Powell%2C+G%2E+van+den+Driessche)
8. A. Chowdhery, S. Narang, J. Devlin, M. Bosma, G. Mishra, A. Roberts, P. Barham, H. W. Chung, C. Sutton, S. Gehrmann, P. Schuh, K. Shi, S. Tsvyashchenko, J. Maynez, A. Rao, P. Barnes, Y. Tay, N. Shazeer, V. Prabhakaran, E. Reif, N. Du, B. Hutchinson, R. Pope, J. Bradbury, J. Austin, M. Isard, G. Gur-… [our card](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
9. S. Smith, M. Patwary, B. Norick, P. LeGresley, S. Rajbhandari, J. Casper, Z. Liu, S. Prabhumoye, G. Zerveas, V. Korthikanti, E. Zheng, R. Child, R. Y. Aminabadi, J. Bernauer, X. Song, M. Shoeybi, Y. He, M. Houston, S. Tiwary, B. Catanzaro, Using deepspeed and megatron to train megatron-turing NLG 53… [our card](https://md.research.elamur.ai/free/en/papers/smith-2022-using-deepspeed-megatron-train-megatron/)
10. F. Petroni, T. Rocktäschel, S. Riedel, P. S. H. Lewis, A. Bakhtin, Y. Wu, A. H. Miller, Language models as knowledge bases?, in: Proc. of EMNLP-IJCNLP, Association for Computational Linguistics, 2019, pp. 2463–2473. [our card](https://md.research.elamur.ai/free/en/papers/petroni-2019-language-models-as-knowledge-bases/)
11. T. Shin, Y. Razeghi, R. L. L. IV, E. Wallace, S. Singh, Autoprompt: Eliciting knowledge from language models with automatically generated prompts, in: Proc. of EMNLP, Association for Computational Linguistics, 2020, pp. 4222–4235. URL: https://doi.org/10.18653/v1/2020. emnlp-main.346. doi: 10.18653/… [↗](https://doi.org/10.18653/v1/2020) [our card](https://md.research.elamur.ai/free/en/papers/shin-2020-autoprompt-eliciting-knowledge-language-models/)
12. A. Roberts, C. Raffel, N. Shazeer, How much knowledge can you pack into the parameters of a language model?, in: Proc. of EMNLP, Association for Computational Linguistics, 2020, pp. 5418–5426. [find](https://scholar.google.com/scholar?q=A%2E+Roberts%2C+C%2E+Raffel%2C+N%2E+Shazeer%2C+How+much+knowledge+can+you+pack+into+the+parameters+of+a+language+model%3F%2C+in%3A+Proc%2E+of+EMNLP%2C+Association+for+Computational+Linguistics%2C+2020%2C+pp%2E+5418%E2%80%935426%2E)
13. G. Qin, J. Eisner, Learning how to ask: Querying lms with mixtures of soft prompts, in: Proc. of NAACL-HLT, Association for Computational Linguistics, 2021, pp. 5203–5212. URL: https://doi.org/10.18653/v1/2021.naacl-main.410. doi: 10.18653/v1/2021.naacl-main. 410 . [↗](https://doi.org/10.18653/v1/2021.naacl-main.410) [our card](https://md.research.elamur.ai/free/en/papers/qin-2021-learning-ask-querying-lms-mixtures/)
14. D. Alivanistos, S. B. Santamaría, M. Cochez, J. Kalo, E. van Krieken, T. Thanapalasingam, Prompting as probing: Using language models for knowledge base construction, CoRR abs/2208.11057 (2022). URL: https://doi.org/10.48550/arXiv.2208.11057. doi: 10.48550/ arXiv.2208.11057 . arXiv:2208.11057 . — [doi:10.48550/arxiv.2208.11057](https://doi.org/10.48550/arxiv.2208.11057) [↗](https://doi.org/10.48550/arXiv.2208.11057)
15. A. Haviv, J. Berant, A. Globerson, Bertese: Learning to speak to BERT, in: Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, EACL 2021, Online, April 19 - 23, 2021, Association for Computational Linguistics, 2021, pp. 3618–3623.… [find](https://scholar.google.com/scholar?q=A%2E+Haviv%2C+J%2E+Berant%2C+A%2E+Globerson%2C+Bertese%3A+Learning+to+speak+to+BERT%2C+in%3A+Proceedings+of+the+16th+Conference+of+the+European+Chapter+of+the+Association+for+Computational+Linguistics%3A+Main+Volume%2C+EAC)
16. R. Cohen, M. Geva, J. Berant, A. Globerson, Crawling the internal knowledge-base of language models, in: Findings of EACL, Association for Computational Linguistics, 2023, pp. 1811–1824. URL: https://aclanthology.org/2023.findings-eacl.139. — [ACL 2023.findings-eacl.139](https://aclanthology.org/2023.findings-eacl.139/) [↗](https://aclanthology.org/2023.findings-eacl.139)
17. B. Veseli, S. Singhania, S. Razniewski, G. Weikum, Evaluating language models for knowledge base completion, in: Proc. of ESWC, volume 13870 of LNCS, Springer, 2023, pp. 227–243. URL: https://doi.org/10.1007/978-3-031-33455-9_14. doi: 10.1007/ 978-3-031-33455-9\\_14 . — [doi:10.1007/978-3-031-33455-9_14](https://doi.org/10.1007/978-3-031-33455-9_14) [↗](https://doi.org/10.1007/978-3-031-33455-9_14)
18. H. Liu, Y. Perl, J. Geller, Concept placement using bert trained by transforming and summarizing biomedical ontology structure, Journal of Biomedical Informatics 112 (2020) 103607. doi: https://doi.org/10.1016/j.jbi.2020.103607 . — [doi:10.1016/j.jbi.2020.103607](https://doi.org/10.1016/j.jbi.2020.103607) [↗](https://doi.org/10.1016/j.jbi.2020.103607)
19. J. Chen, Y. He, Y. Geng, E. Jimenez-Ruiz, H. Dong, I. Horrocks, Contextual semantic embeddings for ontology subsumption prediction, 2023. arXiv:2202.09791 . — [arXiv:2202.09791](https://arxiv.org/abs/2202.09791) [find](https://scholar.google.com/scholar?q=J%2E+Chen%2C+Y%2E+He%2C+Y%2E+Geng%2C+E%2E+Jimenez%2DRuiz%2C+H%2E+Dong%2C+I%2E+Horrocks%2C+Contextual+semantic+embeddings+for+ontology+subsumption+prediction%2C+2023%2E+arXiv%3A2202%2E09791+%2E+%E2%80%94+%5BarXiv%3A2202%2E09791%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F)
20. Z. Ji, N. Lee, R. Frieske, T. Yu, D. Su, Y. Xu, E. Ishii, Y. J. Bang, A. Madotto, P. Fung, Survey of hallucination in natural language generation, ACM Comput. Surv. 55 (2023). URL: https://doi.org/10.1145/3571730. doi: 10.1145/3571730 . [↗](https://doi.org/10.1145/3571730) [our card](https://md.research.elamur.ai/free/en/papers/ji-2022-survey-hallucination-natural-language-generation/)
21. F. Baader, B. Hollunder, B. Nebel, H. Profitlich, E. Franconi, An empirical analysis of optimization techniques for terminological representation systems, or: Making KRIS get a move on, in: Proc. of KR, 1992, pp. 270–281. [find](https://scholar.google.com/scholar?q=F%2E+Baader%2C+B%2E+Hollunder%2C+B%2E+Nebel%2C+H%2E+Profitlich%2C+E%2E+Franconi%2C+An+empirical+analysis+of+optimization+techniques+for+terminological+representation+systems%2C+or%3A+Making+KRIS+get+a+move+on%2C+in%3A+Proc%2E+of+K)
22. B. Glimm, I. Horrocks, B. Motik, R. D. C. Shearer, G. Stoilos, A novel approach to ontology classification, J. Web Semant. 14 (2012) 84–101. URL: https://doi.org/10.1016/j.websem. 2011.12.007. doi: 10.1016/j.websem.2011.12.007 . — [doi:10.1016/j.websem](https://doi.org/10.1016/j.websem) [↗](https://doi.org/10.1016/j.websem)
23. A. Holtzman, J. Buys, M. Forbes, Y. Choi, The curious case of neural text degeneration, CoRR abs/1904.09751 (2019). URL: http://arxiv.org/abs/1904.09751. [↗](http://arxiv.org/abs/1904.09751)
24. D. Guilbeault, A. Baronchelli, D. Centola, Experimental evidence for scale-induced category convergence across populations, Nature Communications 12 (2021) 327. URL: https://doi.org/10.1038/s41467-020-20037-y. doi: 10.1038/s41467-020-20037-y . — [doi:10.1038/s41467-020-20037-y](https://doi.org/10.1038/s41467-020-20037-y) [↗](https://doi.org/10.1038/s41467-020-20037-y)
25. Z. Zhao, E. Wallace, S. Feng, D. Klein, S. Singh, Calibrate before use: Improving few-shot performance of language models, in: M. Meila, T. Zhang (Eds.), Proc. of ICML, volume 139 of Proceedings of Machine Learning Research, PMLR, 2021, pp. 12697–12706. URL: [http://proceedings.mlr.press/v139/zhao21c…](http://proceedings.mlr.press/v139/zhao21c…) [↗](http://proceedings.mlr.press/v139/zhao21c…) [our card](https://md.research.elamur.ai/free/en/papers/zhao-2021-calibrate-before-use-improving-few/)
26. A. Holtzman, P. West, V. Shwartz, Y. Choi, L. Zettlemoyer, Surface form competition: Why the highest probability answer isn’t always right, in: M. Moens, X. Huang, L. Specia, S. W. Yih (Eds.), Proc. of EMNLP, Association for Computational Linguistics, 2021, pp. 7038–7051. URL: https://doi.org/10.186… [our card](https://md.research.elamur.ai/free/en/papers/holtzman-2021-surface-form-competition-why-highest/)