---
title: "Extract, Define, Canonicalize — paper by Zhang and Soh (2024)"
description: "Zhang and Soh propose EDC, a framework that splits KGC into three steps: An LLM extracts triplets… 57 references, cited by 31 papers here."
authors: ["Bowen Zhang", "Harold Soh"]
year: 2024
venue: "Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2024.emnlp-main.548/"
doi: "10.18653/v1/2024.emnlp-main.548"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhang-2024-extract-define-canonicalize-llm-based/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhang-2024-extract-define-canonicalize-llm-based/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/2/) / Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction

# Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction

**Authors:** [Bowen Zhang](https://md.research.elamur.ai/free/en/authors/bowen-zhang/) and Harold Soh · **Year:** 2024 · **Venue:** [Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2024](https://md.research.elamur.ai/free/en/venues/emnlp/2024/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2024.emnlp-main.548/) · [PDF](https://aclanthology.org/2024.emnlp-main.548.pdf) · [DOI 10.18653/v1/2024.emnlp-main.548](https://doi.org/10.18653/v1/2024.emnlp-main.548) · [OpenAlex](https://openalex.org/W4404783226)

**Citations:** 124 citations (OpenAlex, 2026-10-14) · [31](https://md.research.elamur.ai/free/en/papers/zhang-2024-extract-define-canonicalize-llm-based/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary

Prompt-based LLM approaches to knowledge graph construction (KGC) work on small, domain-specific datasets but do not scale well. They must place the whole schema in the prompt, and large schemas overflow the context window or get ignored. They also cannot handle cases where no schema exists in advance.

Zhang and Soh propose EDC, a framework that splits KGC into three steps:
1. An LLM extracts triplets freely, as open information extraction.
2. The LLM writes a natural-language definition for each extracted schema element.
3. Those definitions are embedded and used to map each element to its closest schema counterpart. The LLM confirms whether the substitution is valid.

EDC supports both alignment to a large existing schema (Target Alignment) and building a compact schema of its own (Self Canonicalization). An optional refinement pass, EDC+R, adds a trained Schema Retriever that supplies relevant schema items to the prompt, much like retrieval-augmented generation.

On three KGC benchmarks, EDC outperforms prior methods without fine-tuning the LLM and handles much larger schemas. This makes automatic extraction against schemas such as Wikidata practical.

## Key points

- In the ablation, removing the Schema Retriever lowered performance. It helps the LLM choose finer-grained relations, for example `campus` instead of a generic `location`.
- In Self Canonicalization with GPT-3.5-turbo, human-judged precision was 0.956 / 0.867 / 0.898 for EDC across the three datasets, compared with 0.724 / 0.504 / 0.753 for the clustering baseline CESI. The two annotators agreed at 0.94.
- EDC produced smaller schemas: 200 / 225 / 106 relations, versus 280 / 307 / 114 for CESI and 529 / 667 / 204 for the raw open KG.
- CESI tends to over-generalize. For instance, it merged birth/death place, date, and cause into a single relation, a failure that EDC's LLM verification step avoids.
- Running every component on GPT-3.5-turbo cost about 0.009 USD per example. Code is available at github.com/clear-nus/edc, and WebNLG is among the datasets used (cc-by-nc-sa-4.0).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [National University of Singapore](https://md.research.elamur.ai/free/en/universities/national-university-of-singapore/) — University, Singapore, Singapore · 180 works · 34,304 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🕸️ Knowledge graphs & ontologies](https://md.research.elamur.ai/free/en/topics/knowledge-graphs/) · 192 works

## Cite

Cite as: Bowen Zhang and Harold Soh. 2024. Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction. Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{zhang2024extract,
  title = {{Extract, Define, Canonicalize: An LLM-based Framework for Knowledge Graph Construction}},
  author = {Bowen Zhang and Harold Soh},
  booktitle = {Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing},
  year = {2024},
  doi = {10.18653/v1/2024.emnlp-main.548},
  url = {https://doi.org/10.18653/v1/2024.emnlp-main.548}
}
```

- [← previous ReConcile: Round-Table Conference Improves Reasoning via Consensus among Diverse LLMs](https://md.research.elamur.ai/free/en/papers/chen-2024-reconcile-round-table-conference-improves/)
- [next → PyramidKV: Dynamic KV Cache Compression based on Pyramidal Information Funneling](https://md.research.elamur.ai/free/en/papers/cai-2024-pyramidkv-dynamic-kv-cache-compression/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/2/)

## References

57 references, 20 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
2. Oshin Agarwal, Heming Ge, Siamak Shakeri, and Rami Al-Rfou. 2020. [Knowledge graph based synthetic corpus generation for knowledge-enhanced language model pre-training](https://md.research.elamur.ai/free/en/papers/agarwal-2021-knowledge-graph-based-synthetic-corpus/). arXiv preprint arXiv:2010.12688.
3. Zhen Bi, Jing Chen, Yinuo Jiang, Feiyu Xiong, Wei Guo, Huajun Chen, and Ningyu Zhang. 2024. Codekgc: Code language model for generative knowledge graph construction. ACM Transactions on Asian and Low-Resource Language Information Processing, 23(3):1–16. [find](https://scholar.google.com/scholar?q=Zhen+Bi%2C+Jing+Chen%2C+Yinuo+Jiang%2C+Feiyu+Xiong%2C+Wei+Guo%2C+Huajun+Chen%2C+and+Ningyu+Zhang%2E+2024%2E+Codekgc%3A+Code+language+model+for+generative+knowledge+graph+construction%2E+ACM+Transactions+on+Asian+and+Low%2D)
4. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901.
5. Pere-Lluís Huguet Cabot and Roberto Navigli. 2021. Rebel: Relation extraction by end-to-end language generation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2370– 2381. [find](https://scholar.google.com/scholar?q=Pere%2DLlu%C3%ADs+Huguet+Cabot+and+Roberto+Navigli%2E+2021%2E+Rebel%3A+Relation+extraction+by+end%2Dto%2Dend+language+generation%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2021%2C+pages+2370%E2%80%93+2)
6. Eunsol Choi, Omer Levy, Yejin Choi, and Luke Zettlemoyer. 2018. Ultra-fine entity typing. arXiv preprint arXiv:1807.04905. [find](https://scholar.google.com/scholar?q=Eunsol+Choi%2C+Omer+Levy%2C+Yejin+Choi%2C+and+Luke+Zettlemoyer%2E+2018%2E+Ultra%2Dfine+entity+typing%2E+arXiv+preprint+arXiv%3A1807%2E04905%2E)
7. Sarthak Dash, Gaetano Rossiello, Nandana Mihindukulasooriya, Sugato Bagchi, and Alfio Gliozzo. 2020. Open knowledge graphs canonicalization using variational autoencoders. arXiv preprint arXiv:2012.04780. [find](https://scholar.google.com/scholar?q=Sarthak+Dash%2C+Gaetano+Rossiello%2C+Nandana+Mihindukulasooriya%2C+Sugato+Bagchi%2C+and+Alfio+Gliozzo%2E+2020%2E+Open+knowledge+graphs+canonicalization+using+variational+autoencoders%2E+arXiv+preprint+arXiv%3A2012%2E04)
8. Bayu Distiawan, Gerhard Weikum, Jianzhong Qi, and Rui Zhang. 2019. Neural relation extraction for knowledge base enrichment. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 229–240. [find](https://scholar.google.com/scholar?q=Bayu+Distiawan%2C+Gerhard+Weikum%2C+Jianzhong+Qi%2C+and+Rui+Zhang%2E+2019%2E+Neural+relation+extraction+for+knowledge+base+enrichment%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computatio)
9. Pierre L Dognin, Inkit Padhi, Igor Melnyk, and Payel Das. 2021. Regen: Reinforcement learning for text and knowledge base generation using pretrained language models. arXiv preprint arXiv:2108.12472. [find](https://scholar.google.com/scholar?q=Pierre+L+Dognin%2C+Inkit+Padhi%2C+Igor+Melnyk%2C+and+Payel+Das%2E+2021%2E+Regen%3A+Reinforcement+learning+for+text+and+knowledge+base+generation+using+pretrained+language+models%2E+arXiv+preprint+arXiv%3A2108%2E12472%2E)
10. Thiago Castro Ferreira, Claire Gardent, Nikolai Ilinykh, Chris Van Der Lee, Simon Mille, Diego Moussallem, and Anastasia Shimorina. 2020. [find](https://scholar.google.com/scholar?q=Thiago+Castro+Ferreira%2C+Claire+Gardent%2C+Nikolai+Ilinykh%2C+Chris+Van+Der+Lee%2C+Simon+Mille%2C+Diego+Moussallem%2C+and+Anastasia+Shimorina%2E+2020%2E)
11. The 2020 [bilingual, bi-directional webnlg+ shared task overview and evaluation results (webnlg+ 2020)](https://md.research.elamur.ai/free/en/papers/ferreira-2020-2020-bilingual-bi-directional-webnlg/). In Proceedings of the 3rd International Workshop on Natural Language Generation from the Semantic Web (WebNLG+).
12. Debasis Ganguly, Dwaipayan Roy, Mandar Mitra, and Gareth JF Jones. 2015. Word embedding based generalized language model for information retrieval. In Proceedings of the 38th international ACM SIGIR conference on research and development in information retrieval, pages 795–798. [find](https://scholar.google.com/scholar?q=Debasis+Ganguly%2C+Dwaipayan+Roy%2C+Mandar+Mitra%2C+and+Gareth+JF+Jones%2E+2015%2E+Word+embedding+based+generalized+language+model+for+information+retrieval%2E+In+Proceedings+of+the+38th+international+ACM+SIGIR+c)
13. Juri Ganitkevitch, Benjamin Van Durme, and Chris Callison-Burch. 2013. [Ppdb: The paraphrase database](https://md.research.elamur.ai/free/en/papers/ganitkevitch-2013-ppdb-paraphrase-database/). In Proceedings of the 2013 conference of the north american chapter of the association for computational linguistics: Human language technologies, pages 758–764.
14. Liang Guo, Fu Yan, Yuqian Lu, Ming Zhou, and Tao Yang. 2021. An automatic machining process decision-making system based on knowledge graph. International journal of computer integrated manufacturing, 34(12):1348–1369. [find](https://scholar.google.com/scholar?q=Liang+Guo%2C+Fu+Yan%2C+Yuqian+Lu%2C+Ming+Zhou%2C+and+Tao+Yang%2E+2021%2E+An+automatic+machining+process+decision%2Dmaking+system+based+on+knowledge+graph%2E+International+journal+of+computer+integrated+manufacturing%2C)
15. Qingyu Guo, Fuzhen Zhuang, Chuan Qin, Hengshu Zhu, Xing Xie, Hui Xiong, and Qing He. 2020. A survey on knowledge graph-based recommender systems. IEEE Transactions on Knowledge and Data Engineering, 34(8):3549–3568. [find](https://scholar.google.com/scholar?q=Qingyu+Guo%2C+Fuzhen+Zhuang%2C+Chuan+Qin%2C+Hengshu+Zhu%2C+Xing+Xie%2C+Hui+Xiong%2C+and+Qing+He%2E+2020%2E+A+survey+on+knowledge+graph%2Dbased+recommender+systems%2E+IEEE+Transactions+on+Knowledge+and+Data+Engineering%2C+3)
16. Harsha Gurulingappa, Abdul Mateen Rajput, Angus Roberts, Juliane Fluck, Martin Hofmann-Apitius, and Luca Toldo. 2012. Development of a benchmark corpus to support the automatic extraction of drug-related adverse effects from medical case reports. Journal of biomedical informatics, 45(5):885–892. [find](https://scholar.google.com/scholar?q=Harsha+Gurulingappa%2C+Abdul+Mateen+Rajput%2C+Angus+Roberts%2C+Juliane+Fluck%2C+Martin+Hofmann%2DApitius%2C+and+Luca+Toldo%2E+2012%2E+Development+of+a+benchmark+corpus+to+support+the+automatic+extraction+of+drug%2Drela)
17. Ridong Han, Tao Peng, Chaohao Yang, Benyou Wang, Lu Liu, and Xiang Wan. 2023. Is information extraction solved by chatgpt? an analysis of performance, evaluation criteria, robustness and errors. arXiv preprint arXiv:2305.14450. — [arXiv:2305.14450](https://arxiv.org/abs/2305.14450) [find](https://scholar.google.com/scholar?q=Ridong+Han%2C+Tao+Peng%2C+Chaohao+Yang%2C+Benyou+Wang%2C+Lu+Liu%2C+and+Xiang+Wan%2E+2023%2E+Is+information+extraction+solved+by+chatgpt%3F+an+analysis+of+performance%2C+evaluation+criteria%2C+robustness+and+errors%2E+arXiv)
18. Xiao Huang, Jingyuan Zhang, Dingcheng Li, and Ping Li. 2019. Knowledge graph embedding based question answering. In Proceedings of the twelfth ACM international conference on web search and data mining, pages 105–113. [find](https://scholar.google.com/scholar?q=Xiao+Huang%2C+Jingyuan+Zhang%2C+Dingcheng+Li%2C+and+Ping+Li%2E+2019%2E+Knowledge+graph+embedding+based+question+answering%2E+In+Proceedings+of+the+twelfth+ACM+international+conference+on+web+search+and+data+minin)
19. Shaoxiong Ji, Shirui Pan, Erik Cambria, Pekka Marttinen, and S Yu Philip. 2021. A survey on knowledge graphs: Representation, acquisition, and applications. IEEE transactions on neural networks and learning systems, 33(2):494–514. [find](https://scholar.google.com/scholar?q=Shaoxiong+Ji%2C+Shirui+Pan%2C+Erik+Cambria%2C+Pekka+Marttinen%2C+and+S+Yu+Philip%2E+2021%2E+A+survey+on+knowledge+graphs%3A+Representation%2C+acquisition%2C+and+applications%2E+IEEE+transactions+on+neural+networks+and+le)
20. Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al. 2023. Mistral 7b. arXiv preprint arXiv:2310.06825. [find](https://scholar.google.com/scholar?q=Albert+Q+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+et+al%2E+2023%2E+Mist)
21. Martin Josifoski, Nicola De Cao, Maxime Peyrard, Fabio Petroni, and Robert West. 2022. GenIE: Generative information extraction. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 4626–4643, Seattle… [find](https://scholar.google.com/scholar?q=Martin+Josifoski%2C+Nicola+De+Cao%2C+Maxime+Peyrard%2C+Fabio+Petroni%2C+and+Robert+West%2E+2022%2E+GenIE%3A+Generative+information+extraction%2E+In+Proceedings+of+the+2022+Conference+of+the+North+American+Chapter+of+)
22. Serafina Kamp, Morteza Fayazi, Zineb Benameur-El, Shuyan Yu, and Ronald Dreslinski. 2023. Open information extraction: A review of baseline techniques, approaches, and applications. arXiv preprint arXiv:2310.11644. — [arXiv:2310.11644](https://arxiv.org/abs/2310.11644) [find](https://scholar.google.com/scholar?q=Serafina+Kamp%2C+Morteza+Fayazi%2C+Zineb+Benameur%2DEl%2C+Shuyan+Yu%2C+and+Ronald+Dreslinski%2E+2023%2E+Open+information+extraction%3A+A+review+of+baseline+techniques%2C+approaches%2C+and+applications%2E+arXiv+preprint+arX)
23. Keshav Kolluru, Vaibhav Adlakha, Samarth Aggarwal, Soumen Chakrabarti, et al. 2020. Openie6: Iterative grid labeling and coordination analysis for open information extraction. arXiv preprint arXiv:2010.03147. [find](https://scholar.google.com/scholar?q=Keshav+Kolluru%2C+Vaibhav+Adlakha%2C+Samarth+Aggarwal%2C+Soumen+Chakrabarti%2C+et+al%2E+2020%2E+Openie6%3A+Iterative+grid+labeling+and+coordination+analysis+for+open+information+extraction%2E+arXiv+preprint+arXiv%3A201)
24. Luong Thi Hong Lan, Tran Manh Tuan, Tran Thi Ngan, Nguyen Long Giang, Vo Truong Nhu Ngoc, Pham Van Hai, et al. 2020. A new complex fuzzy inference system with fuzzy knowledge graph and extensions in decision making. Ieee Access, 8:164899–164921. [find](https://scholar.google.com/scholar?q=Luong+Thi+Hong+Lan%2C+Tran+Manh+Tuan%2C+Tran+Thi+Ngan%2C+Nguyen+Long+Giang%2C+Vo+Truong+Nhu+Ngoc%2C+Pham+Van+Hai%2C+et+al%2E+2020%2E+A+new+complex+fuzzy+inference+system+with+fuzzy+knowledge+graph+and+extensions+in+d)
25. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Ves Stoyanov, and Luke Zettlemoyer. 2019. [Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/). arXiv preprint arXiv:1910.13461.
26. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, et al. 2020. [Retrieval-augmented generation for knowledge-intensive nlp tasks](https://md.research.elamur.ai/free/en/papers/lewis-2020-retrieval-augmented-generation-knowledge-intensive/). Advances in Neural Information Processing Systems, 33:9459–9474.
27. Bo Li, Gexiang Fang, Yang Yang, Quansen Wang, Wei Ye, Wen Zhao, and Shikun Zhang. 2023. Evaluating chatgpt’s information extraction capabilities: An assessment of performance, explainability, calibration, and faithfulness. arXiv preprint arXiv:2304.11633. on computational natural language learning (… [find](https://scholar.google.com/scholar?q=Bo+Li%2C+Gexiang+Fang%2C+Yang+Yang%2C+Quansen+Wang%2C+Wei+Ye%2C+Wen+Zhao%2C+and+Shikun+Zhang%2E+2023%2E+Evaluating+chatgpt%E2%80%99s+information+extraction+capabilities%3A+An+assessment+of+performance%2C+explainability%2C+calibrat)
28. Alisa Smirnova and Philippe Cudré-Mauroux. 2018. Relation extraction using distant supervision: A survey. ACM Computing Surveys (CSUR), 51(5):1–35. [find](https://scholar.google.com/scholar?q=Alisa+Smirnova+and+Philippe+Cudr%C3%A9%2DMauroux%2E+2018%2E+Relation+extraction+using+distant+supervision%3A+A+survey%2E+ACM+Computing+Surveys+%28CSUR%29%2C+51%285%29%3A1%E2%80%9335%2E)
29. Nelson F Liu, Kevin Lin, John Hewitt, Ashwin Paranjape, Michele Bevilacqua, Fabio Petroni, and Percy Liang. 2024. [Lost in the middle: How language models use long contexts](https://md.research.elamur.ai/free/en/papers/liu-2023-lost-middle-language-models-use/). Transactions of the Association for Computational Linguistics, 12:157–173.
30. Rhea Sukthanker, Soujanya Poria, Erik Cambria, and Ramkumar Thirunavukarasu. 2020. Anaphora and coreference resolution: A review. Information Fusion, 59:139–162. [find](https://scholar.google.com/scholar?q=Rhea+Sukthanker%2C+Soujanya+Poria%2C+Erik+Cambria%2C+and+Ramkumar+Thirunavukarasu%2E+2020%2E+Anaphora+and+coreference+resolution%3A+A+review%2E+Information+Fusion%2C+59%3A139%E2%80%93162%2E)
31. Pai Liu, Wenyang Gao, Wenjie Dong, Songfang Huang, and Yue Zhang. 2022. Open information extraction from 2007 to 2022–a survey. arXiv preprint arXiv:2208.08690. — [arXiv:2208.08690](https://arxiv.org/abs/2208.08690) [find](https://scholar.google.com/scholar?q=Pai+Liu%2C+Wenyang+Gao%2C+Wenjie+Dong%2C+Songfang+Huang%2C+and+Yue+Zhang%2E+2022%2E+Open+information+extraction+from+2007+to+2022%E2%80%93a+survey%2E+arXiv+preprint+arXiv%3A2208%2E08690%2E+%E2%80%94+%5BarXiv%3A2208%2E08690%5D%28https%3A%2F%2Farxiv%2Eorg%2F)
32. Qingyu Tan, Lu Xu, Lidong Bing, Hwee Tou Ng, and Sharifah Mahani Aljunied. 2022. [Revisiting docred – addressing the false negative problem in relation extraction](https://md.research.elamur.ai/free/en/papers/tan-2022-revisiting-docred-addressing-false-negative/). In Proceedings of EMNLP.
33. Yi Luan, Luheng He, Mari Ostendorf, and Hannaneh Hajishirzi. 2018. [Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction](https://md.research.elamur.ai/free/en/papers/luan-2018-multi-task-identification-entities-relations/). arXiv preprint arXiv:1808.09602.
34. Shikhar Vashishth, Prince Jain, and Partha Talukdar. 2018. Cesi: Canonicalizing open knowledge bases using embeddings and side information. In Proceedings of the 2018 World Wide Web Conference, pages 1317–1327. [find](https://scholar.google.com/scholar?q=Shikhar+Vashishth%2C+Prince+Jain%2C+and+Partha+Talukdar%2E+2018%2E+Cesi%3A+Canonicalizing+open+knowledge+bases+using+embeddings+and+side+information%2E+In+Proceedings+of+the+2018+World+Wide+Web+Conference%2C+pages+)
35. Pedro Henrique Martins, Zita Marinho, and André FT Martins. 2019. Joint learning of named entity recognition and entity linking. arXiv preprint arXiv:1907.08243. [find](https://scholar.google.com/scholar?q=Pedro+Henrique+Martins%2C+Zita+Marinho%2C+and+Andr%C3%A9+FT+Martins%2E+2019%2E+Joint+learning+of+named+entity+recognition+and+entity+linking%2E+arXiv+preprint+arXiv%3A1907%2E08243%2E)
36. Denny Vrandečić and Markus Krötzsch. 2014. [Wikidata: a free collaborative knowledgebase](https://md.research.elamur.ai/free/en/papers/vrandecic-2014-wikidata-free-collaborative-knowledgebase/). Communications of the ACM, 57(10):78–85.
37. Igor Melnyk, Pierre Dognin, and Payel Das. 2022. Knowledge graph generation from text. arXiv preprint arXiv:2211.10511. [find](https://scholar.google.com/scholar?q=Igor+Melnyk%2C+Pierre+Dognin%2C+and+Payel+Das%2E+2022%2E+Knowledge+graph+generation+from+text%2E+arXiv+preprint+arXiv%3A2211%2E10511%2E)
38. George A Miller. 1995. [Wordnet: a lexical database for english](https://md.research.elamur.ai/free/en/papers/miller-1994-wordnet/). Communications of the ACM, 38(11):39–41.
39. Yasumasa Onoe and Greg Durrett. 2020. Fine-grained entity typing for domain independent entity linking. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8576–8583. [find](https://scholar.google.com/scholar?q=Yasumasa+Onoe+and+Greg+Durrett%2E+2020%2E+Fine%2Dgrained+entity+typing+for+domain+independent+entity+linking%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+34%2C+pages+8576%E2%80%938583%2E)
40. Shon Otmazgin, Arie Cattan, and Yoav Goldberg. 2023. [LingMess: Linguistically informed multi expert scorers for coreference resolution](https://md.research.elamur.ai/free/en/papers/otmazgin-2023-lingmess-linguistically-informed-multi-expert/). In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2752– 2760, Dubrovnik, Croatia. Association fo…
41. Rifki Afina Putri, Giwon Hong, and Sung-Hyon Myaeng. 2019. Aligning open ie relations and kb relations using a siamese network based on word embedding. In Proceedings of the 13th International Conference on Computational Semantics-Long Papers, pages 142–153. [find](https://scholar.google.com/scholar?q=Rifki+Afina+Putri%2C+Giwon+Hong%2C+and+Sung%2DHyon+Myaeng%2E+2019%2E+Aligning+open+ie+relations+and+kb+relations+using+a+siamese+network+based+on+word+embedding%2E+In+Proceedings+of+the+13th+International+Confere)
42. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of machine learning research, 21(140):1–67.
43. Dan Roth and Wen-tau Yih. 2004. [A linear programming formulation for global inference in natural language tasks](https://md.research.elamur.ai/free/en/papers/roth-2004-linear-programming-formulation-global-inference/). In Proceedings of the eighth conference Somin Wadhwa, Silvio Amir, and Byron C Wallace. 2023. Revisiting relation extraction in the era of large language models. In Proceedings of the con…
44. Hongwei Wang, Miao Zhao, Xing Xie, Wenjie Li, and Minyi Guo. 2019. Knowledge graph convolutional networks for recommender systems. In The world wide web conference, pages 3307–3313. [find](https://scholar.google.com/scholar?q=Hongwei+Wang%2C+Miao+Zhao%2C+Xing+Xie%2C+Wenjie+Li%2C+and+Minyi+Guo%2E+2019%2E+Knowledge+graph+convolutional+networks+for+recommender+systems%2E+In+The+world+wide+web+conference%2C+pages+3307%E2%80%933313%2E)
45. Liang Wang, Nan Yang, Xiaolong Huang, Linjun Yang, Rangan Majumder, and Furu Wei. 2023. [Improving text embeddings with large language models](https://md.research.elamur.ai/free/en/papers/wang-2024-improving-text-embeddings-large-language/). arXiv preprint arXiv:2401.00368.
46. Xiang Wei, Xingyu Cui, Ning Cheng, Xiaobin Wang, Xin Zhang, Shen Huang, Pengjun Xie, Jinan Xu, Yufeng Chen, Meishan Zhang, et al. 2023. [Zero-shot information extraction via chatting with chatgpt](https://md.research.elamur.ai/free/en/papers/wei-2023-chatie-zero-shot-information-extraction/). arXiv preprint arXiv:2302.10205.
47. Yaqi Xie, Chen Yu, Tongyao Zhu, Jinbin Bai, Ze Gong, and Harold Soh. 2023. Translating natural language to planning goals with large-language models. arXiv preprint arXiv:2302.05128. — [arXiv:2302.05128](https://arxiv.org/abs/2302.05128) [find](https://scholar.google.com/scholar?q=Yaqi+Xie%2C+Chen+Yu%2C+Tongyao+Zhu%2C+Jinbin+Bai%2C+Ze+Gong%2C+and+Harold+Soh%2E+2023%2E+Translating+natural+language+to+planning+goals+with+large%2Dlanguage+models%2E+arXiv+preprint+arXiv%3A2302%2E05128%2E+%E2%80%94+%5BarXiv%3A2302%2E051)
48. Ziwei Xu, Sanjay Jain, and Mohan Kankanhalli. 2024. [Hallucination is inevitable: An innate limitation of large language models](https://md.research.elamur.ai/free/en/papers/xu-2024-hallucination-inevitable-innate-limitation-large/). arXiv preprint arXiv:2401.11817.
49. Yifan Yao, Jinhao Duan, Kaidi Xu, Yuanfang Cai, Zhibo Sun, and Yue Zhang. 2024. [A survey on large language model (llm) security and privacy: The good, the bad, and the ugly](https://md.research.elamur.ai/free/en/papers/yao-2023-survey-large-language-model-llm/). High-Confidence Computing, page 100211.
50. Michihiro Yasunaga, Hongyu Ren, Antoine Bosselut, Percy Liang, and Jure Leskovec. 2021. Qa-gnn: Reasoning with language models and knowledge graphs for question answering. arXiv preprint arXiv:2104.06378. [find](https://scholar.google.com/scholar?q=Michihiro+Yasunaga%2C+Hongyu+Ren%2C+Antoine+Bosselut%2C+Percy+Liang%2C+and+Jure+Leskovec%2E+2021%2E+Qa%2Dgnn%3A+Reasoning+with+language+models+and+knowledge+graphs+for+question+answering%2E+arXiv+preprint+arXiv%3A2104%2E06)
51. Hongbin Ye, Ningyu Zhang, Hui Chen, and Huajun Chen. 2022. Generative knowledge graph construction: A review. arXiv preprint arXiv:2210.12714. [find](https://scholar.google.com/scholar?q=Hongbin+Ye%2C+Ningyu+Zhang%2C+Hui+Chen%2C+and+Huajun+Chen%2E+2022%2E+Generative+knowledge+graph+construction%3A+A+review%2E+arXiv+preprint+arXiv%3A2210%2E12714%2E)
52. Daojian Zeng, Kang Liu, Yubo Chen, and Jun Zhao. 2015. Distant supervision for relation extraction via piecewise convolutional neural networks. In Proceedings of the 2015 conference on empirical methods in natural language processing, pages 1753–1762. [find](https://scholar.google.com/scholar?q=Daojian+Zeng%2C+Kang+Liu%2C+Yubo+Chen%2C+and+Jun+Zhao%2E+2015%2E+Distant+supervision+for+relation+extraction+via+piecewise+convolutional+neural+networks%2E+In+Proceedings+of+the+2015+conference+on+empirical+metho)
53. Daojian Zeng, Kang Liu, Siwei Lai, Guangyou Zhou, and Jun Zhao. 2014. [Relation classification via convolutional deep neural network](https://md.research.elamur.ai/free/en/papers/zeng-2014-relation-classification-convolutional-deep-neural/). In Proceedings of COLING 2014, the 25th international conference on computational linguistics: technical papers, pages 2335–2344.
54. Bowen Zhang and Harold Soh. 2023. Large language models as zero-shot human models for human-robot interaction. In 2023 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 7961–7968. IEEE. [find](https://scholar.google.com/scholar?q=Bowen+Zhang+and+Harold+Soh%2E+2023%2E+Large+language+models+as+zero%2Dshot+human+models+for+human%2Drobot+interaction%2E+In+2023+IEEE%2FRSJ+International+Conference+on+Intelligent+Robots+and+Systems+%28IROS%29%2C+pages)
55. Lingfeng Zhong, Jia Wu, Qian Li, Hao Peng, and Xindong Wu. 2023. A comprehensive survey on automatic knowledge graph construction. ACM Computing Surveys, 56(4):1–62. [find](https://scholar.google.com/scholar?q=Lingfeng+Zhong%2C+Jia+Wu%2C+Qian+Li%2C+Hao+Peng%2C+and+Xindong+Wu%2E+2023%2E+A+comprehensive+survey+on+automatic+knowledge+graph+construction%2E+ACM+Computing+Surveys%2C+56%284%29%3A1%E2%80%9362%2E)
56. Shaowen Zhou, Bowen Yu, Aixin Sun, Cheng Long, Jingyang Li, Haiyang Yu, Jian Sun, and Yongbin Li. 2022. A survey on neural open information extraction: Current status and future directions. arXiv preprint arXiv:2205.11725. — [arXiv:2205.11725](https://arxiv.org/abs/2205.11725) [find](https://scholar.google.com/scholar?q=Shaowen+Zhou%2C+Bowen+Yu%2C+Aixin+Sun%2C+Cheng+Long%2C+Jingyang+Li%2C+Haiyang+Yu%2C+Jian+Sun%2C+and+Yongbin+Li%2E+2022%2E+A+survey+on+neural+open+information+extraction%3A+Current+status+and+future+directions%2E+arXiv+prep)
57. Andrej Žukov-Gregorič, Yoram Bachrach, and Sam Coope. 2018. Named entity recognition with parallel recurrent neural networks. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 69–74. [find](https://scholar.google.com/scholar?q=Andrej+%C5%BDukov%2DGregoric%CC%8C%2C+Yoram+Bachrach%2C+and+Sam+Coope%2E+2018%2E+Named+entity+recognition+with+parallel+recurrent+neural+networks%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computat)

*Updated 29 October 2026*