---
title: "Dipping PLMs Sauce — Chen 2023"
description: "Knowledge Graph Completion (KGC) predicts missing entities in incomplete facts."
authors: ["Chen Chen", "Yufei Wang", "Aixin Sun", "Bing Li", "Kwok-Yan Lam"]
year: 2023
venue: "Findings of the Association for Computational Linguistics: ACL 2023"
source: "https://aclanthology.org/2023.findings-acl.729/"
doi: "10.18653/v1/2023.findings-acl.729"
arxiv: "2307.01709"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/chen-2023-dipping-plms-sauce-bridging-structure/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/chen-2023-dipping-plms-sauce-bridging-structure/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2023](https://md.research.elamur.ai/free/papers/2023/page/2/) / Dipping PLMs Sauce: Bridging Structure and Text for Effective Knowledge Graph Completion via Conditional Soft Prompting

# Dipping PLMs Sauce: Bridging Structure and Text for Effective Knowledge Graph Completion via Conditional Soft Prompting

**Authors:** [Chen Chen](https://md.research.elamur.ai/free/authors/chen-chen/), [Yufei Wang](https://md.research.elamur.ai/free/authors/yufei-wang/), [Aixin Sun](https://md.research.elamur.ai/free/authors/aixin-sun/), Bing Li, Kwok-Yan Lam · **Year:** 2023 · **Venue:** Findings of the Association for Computational Linguistics: ACL 2023 · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.findings-acl.729/) · [PDF](https://aclanthology.org/2023.findings-acl.729.pdf) · [DOI 10.18653/v1/2023.findings-acl.729](https://doi.org/10.18653/v1/2023.findings-acl.729) · [arXiv 2307.01709](https://arxiv.org/abs/2307.01709) · [OpenAlex](https://openalex.org/W4385572435)

**Citations:** 56 citations (openalex-snapshot · 2026-10-14) · [19](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
Knowledge Graph Completion (KGC) predicts missing entities in incomplete facts. Purely graph-based embedding models capture structure but ignore text. PLM-based models read entity text, but full fine-tuning makes them over-rely on surface wording. For example, they may mix up Leonardo da Vinci and Leonardo DiCaprio. The authors propose CSProm-KG, which keeps the PLM frozen and trains only a small set of soft prompt vectors. These prompts are generated from the entity and relation embeddings of a graph-based KGC model. Inside the frozen PLM, the prompts interact with the query text (names, descriptions and, for temporal KGs, timestamps). The PLM outputs at the prompt positions are passed back to the graph model, which ranks candidate entities. A Local Adversarial Regularization step adds textually similar distractor entities during training. Across static and temporal benchmarks, CSProm-KG beats both graph-based and PLM-based baselines and sets new state-of-the-art results. It is also much faster at inference than generative PLM approaches. The work shows that conditional prompting can balance structural and textual signals in KG reasoning.

## Key points
- Evaluated on static KGC (WN18RR, FB15K-237, Wikidata5M) and temporal KGC (ICEWS14, ICEWS05-15); code is released at github.com/chenchens190009/CSProm-KG.
- The framework works with different graph backbones (ConvE, TransE, DistMult). With TransE on WN18RR, MRR roughly doubles (.243 to .499) and H@1 rises about tenfold (.043 to .462).
- It uses BERT only to encode queries and scores entities directly against the embedding matrix. Inference takes about 0.2 min, compared with 104 and 115 min for the seq2seq models GenKGC and KG-S2S.
- Ablations indicate that prompt tuning outperforms full fine-tuning. Case studies show the fine-tuned variant drifting toward keyword-similar but wrong entities.
- Limitations: it needs more compute than pure graph models, and training occasionally collapses on small datasets such as WN18RR with certain random seeds; this was not observed on Wikidata5M.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cited by

Cited by

19 papers of our library cite it (of the 140673 whose bibliographies we read); 1 of them have a page here, 18 not yet.

- [Unifying Large Language Models and Knowledge Graphs: A Roadmap](https://md.research.elamur.ai/free/papers/pan-2023-unifying-large-language-models-knowledge/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{chen2023dipping,
  title = {{Dipping PLMs Sauce: Bridging Structure and Text for Effective Knowledge Graph Completion via Conditional Soft Prompting}},
  author = {Chen, Chen and Wang, Yufei and Sun, Aixin and Li, Bing and Lam, Kwok-Yan},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2023},
  year = {2023},
  doi = {10.18653/v1/2023.findings-acl.729},
  eprint = {2307.01709},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2023.findings-acl.729}
}
```

- [← previous DINOv2: Learning Robust Visual Features without Supervision](https://md.research.elamur.ai/free/papers/oquab-2023-dinov2-learning-robust-visual-features/)
- [next → DiscoFlan: Instruction Fine-tuning and Refined Text Generation for Discourse Relation Label Classification](https://md.research.elamur.ai/free/papers/anuranjana-2023-discoflan-instruction-fine-tuning-refined/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/papers/2023/page/2/)

## References

45 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Farahnaz Akrami, Mohammed Samiul Saeef, Qingheng Zhang, Wei Hu, and Chengkai Li. 2020. Realistic re-evaluation of knowledge graph completion methods: An experimental study. In Proceedings of the 2020 International Conference on Management of Data, SIGMOD Conference 2020, online conference [Portland,… [find](https://scholar.google.com/scholar?q=Farahnaz+Akrami%2C+Mohammed+Samiul+Saeef%2C+Qingheng+Zhang%2C+Wei+Hu%2C+and+Chengkai+Li%2E+2020%2E+Realistic+re%2Devaluation+of+knowledge+graph+completion+methods%3A+An+experimental+study%2E+In+Proceedings+of+the+2020+)
2. Antoine Bordes, Nicolas Usunier, Alberto García-Durán, Jason Weston, and Oksana Yakhnenko. 2013. Translating embeddings for modeling multi-relational data. In Advances in Neural Information Processing Systems 26: 27th Annual Conference on Neural Information Processing Systems 2013. Proceedings of a… [find](https://scholar.google.com/scholar?q=Antoine+Bordes%2C+Nicolas+Usunier%2C+Alberto+Garc%C3%ADa%2DDur%C3%A1n%2C+Jason+Weston%2C+and+Oksana+Yakhnenko%2E+2013%2E+Translating+embeddings+for+modeling+multi%2Drelational+data%2E+In+Advances+in+Neural+Information+Processing)
3. Elizabeth Boschee, Jennifer Lautenschlager, Sean O’Brien, Steve Shellman, James Starz, and Michael Ward. 2015. ICEWS Coded Event Data. Harvard Dataverse. [find](https://scholar.google.com/scholar?q=Elizabeth+Boschee%2C+Jennifer+Lautenschlager%2C+Sean+O%E2%80%99Brien%2C+Steve+Shellman%2C+James+Starz%2C+and+Michael+Ward%2E+2015%2E+ICEWS+Coded+Event+Data%2E+Harvard+Dataverse%2E)
4. Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter,… [find](https://scholar.google.com/scholar?q=Tom+B%2E+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+Sandhini+Agarwal%2C+Ariel+Herbert%2DVoss%2C+Gretche)
5. Chen Chen, Yufei Wang, Bing Li, and Kwok-Yan Lam. 2022. Knowledge is flat: A seq2seq generative framework for various knowledge graph completion. In Proceedings of the 29th International Conference on Computational Linguistics, COLING 2022, Gyeongju, Republic of Korea, October 12-17, 2022, pages 400… [find](https://scholar.google.com/scholar?q=Chen+Chen%2C+Yufei+Wang%2C+Bing+Li%2C+and+Kwok%2DYan+Lam%2E+2022%2E+Knowledge+is+flat%3A+A+seq2seq+generative+framework+for+various+knowledge+graph+completion%2E+In+Proceedings+of+the+29th+International+Conference+on)
6. Louis Clouâtre, Philippe Trempe, Amal Zouaq, and Sarath Chandar. 2021. MLMLM: link prediction with mean likelihood masked language model. In Findings of the Association for Computational Linguistics: ACL/IJCNLP 2021, Online Event, August 1-6, 2021, volume ACL/IJCNLP 2021 of Findings of ACL, pages 43… [find](https://scholar.google.com/scholar?q=Louis+Clou%C3%A2tre%2C+Philippe+Trempe%2C+Amal+Zouaq%2C+and+Sarath+Chandar%2E+2021%2E+MLMLM%3A+link+prediction+with+mean+likelihood+masked+language+model%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+)
7. Shib Sankar Dasgupta, Swayambhu Nath Ray, and Partha P. Talukdar. 2018. Hyte: Hyperplane-based temporally aware knowledge graph embedding. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 2001–2011. Ass… [find](https://scholar.google.com/scholar?q=Shib+Sankar+Dasgupta%2C+Swayambhu+Nath+Ray%2C+and+Partha+P%2E+Talukdar%2E+2018%2E+Hyte%3A+Hyperplane%2Dbased+temporally+aware+knowledge+graph+embedding%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in)
8. Tim Dettmers, Pasquale Minervini, Pontus Stenetorp, and Sebastian Riedel. 2018. Convolutional 2d knowledge graph embeddings. In Proceedings of the Thirty-Second AAAI Conference on Artificial Intelligence, (AAAI-18), the 30th innovative Applications of Artificial Intelligence (IAAI-18), and the 8th A… [find](https://scholar.google.com/scholar?q=Tim+Dettmers%2C+Pasquale+Minervini%2C+Pontus+Stenetorp%2C+and+Sebastian+Riedel%2E+2018%2E+Convolutional+2d+knowledge+graph+embeddings%2E+In+Proceedings+of+the+Thirty%2DSecond+AAAI+Conference+on+Artificial+Intellige)
9. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2019%2E+BERT%3A+pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In+Proceedings+of+the+2019+Conference+of+the+N)
10. Alberto García-Durán, Sebastijan Dumancic, and Mathias Niepert. 2018. Learning sequence encoders for temporal knowledge graph completion. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Belgium, October 31 - November 4, 2018, pages 4816–4821. Asso… [find](https://scholar.google.com/scholar?q=Alberto+Garc%C3%ADa%2DDur%C3%A1n%2C+Sebastijan+Dumancic%2C+and+Mathias+Niepert%2E+2018%2E+Learning+sequence+encoders+for+temporal+knowledge+graph+completion%2E+In+Proceedings+of+the+2018+Conference+on+Empirical+Methods+in+)
11. Rishab Goel, Seyed Mehran Kazemi, Marcus Brubaker, and Pascal Poupart. 2020. Diachronic embedding for temporal knowledge graph completion. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI… [find](https://scholar.google.com/scholar?q=Rishab+Goel%2C+Seyed+Mehran+Kazemi%2C+Marcus+Brubaker%2C+and+Pascal+Poupart%2E+2020%2E+Diachronic+embedding+for+temporal+knowledge+graph+completion%2E+In+The+Thirty%2DFourth+AAAI+Conference+on+Artificial+Intelligen)
12. Ian J. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and harnessing adversarial examples. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. [find](https://scholar.google.com/scholar?q=Ian+J%2E+Goodfellow%2C+Jonathon+Shlens%2C+and+Christian+Szegedy%2E+2015%2E+Explaining+and+harnessing+adversarial+examples%2E+In+3rd+International+Conference+on+Learning+Representations%2C+ICLR+2015%2C+San+Diego%2C+CA%2C+) [our card](https://md.research.elamur.ai/free/papers/goodfellow-2014-explaining-harnessing-adversarial-examples/)
13. Zhen Han, Gengyuan Zhang, Yunpu Ma, and Volker Tresp. 2021. Time-dependent entity embedding is not all you need: A re-evaluation of temporal knowledge graph completion models under a unified framework. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2… [find](https://scholar.google.com/scholar?q=Zhen+Han%2C+Gengyuan+Zhang%2C+Yunpu+Ma%2C+and+Volker+Tresp%2E+2021%2E+Time%2Ddependent+entity+embedding+is+not+all+you+need%3A+A+re%2Devaluation+of+temporal+knowledge+graph+completion+models+under+a+unified+framework)
14. Bosung Kim, Taesuk Hong, Youngjoong Ko, and Jungyun Seo. 2020. Multi-task learning for knowledge graph completion with pre-trained language models. In Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020, pages 173… [find](https://scholar.google.com/scholar?q=Bosung+Kim%2C+Taesuk+Hong%2C+Youngjoong+Ko%2C+and+Jungyun+Seo%2E+2020%2E+Multi%2Dtask+learning+for+knowledge+graph+completion+with+pre%2Dtrained+language+models%2E+In+Proceedings+of+the+28th+International+Conference+)
15. Findings of the Association for Computational Linguistics: ACL 2022, pages 3570–3581, Dublin, Ireland. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2022%2C+pages+3570%E2%80%933581%2C+Dublin%2C+Ireland%2E+Association+for+Computational+Linguistics%2E)
16. Timothée Lacroix, Guillaume Obozinski, and Nicolas Usunier. 2020. Tensor decompositions for temporal knowledge base completion. In 8th International Conference on Learning Representations, ICLR 2020, Addis Ababa, Ethiopia, April 26-30, 2020. OpenRe-view.net. [find](https://scholar.google.com/scholar?q=Timoth%C3%A9e+Lacroix%2C+Guillaume+Obozinski%2C+and+Nicolas+Usunier%2E+2020%2E+Tensor+decompositions+for+temporal+knowledge+base+completion%2E+In+8th+International+Conference+on+Learning+Representations%2C+ICLR+2020%2C+)
17. Aleksander Madry, Aleksandar Makelov, Ludwig Schmidt, Dimitris Tsipras, and Adrian Vladu. 2018. Towards deep learning models resistant to adversarial attacks. In 6th International Conference on Learning Representations, ICLR 2018, Vancouver, BC, Canada, April 30 - May 3, 2018, Conference Track Proce… [find](https://scholar.google.com/scholar?q=Aleksander+Madry%2C+Aleksandar+Makelov%2C+Ludwig+Schmidt%2C+Dimitris+Tsipras%2C+and+Adrian+Vladu%2E+2018%2E+Towards+deep+learning+models+resistant+to+adversarial+attacks%2E+In+6th+International+Conference+on+Learni) [our card](https://md.research.elamur.ai/free/papers/madry-2017-towards-deep-learning-models-resistant/)
18. Julien Leblay and Melisachew Wudage Chekol. 2018. Deriving validity time in knowledge graph. In Companion of the The Web Conference 2018 on The Web Conference 2018, WWW 2018, Lyon , France, April 23-27, 2018, pages 1771–1776. ACM. [find](https://scholar.google.com/scholar?q=Julien+Leblay+and+Melisachew+Wudage+Chekol%2E+2018%2E+Deriving+validity+time+in+knowledge+graph%2E+In+Companion+of+the+The+Web+Conference+2018+on+The+Web+Conference+2018%2C+WWW+2018%2C+Lyon+%2C+France%2C+April+23%2D2)
19. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 7-11 November, 2021, pages 3045– 3059.… [find](https://scholar.google.com/scholar?q=Brian+Lester%2C+Rami+Al%2DRfou%2C+and+Noah+Constant%2E+2021%2E+The+power+of+scale+for+parameter%2Defficient+prompt+tuning%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Processing)
20. Jia Li, Yuyuan Zhao, Zhi Jin, Ge Li, Tao Shen, Zheng-wei Tao, and Chongyang Tao. 2022. Sk2: Integrating implicit sentiment knowledge and explicit syntax knowledge for aspect-based sentiment analysis. In Proceedings of the 31st ACM International Conference on Information & Knowledge Management, CIKM… [find](https://scholar.google.com/scholar?q=Jia+Li%2C+Yuyuan+Zhao%2C+Zhi+Jin%2C+Ge+Li%2C+Tao+Shen%2C+Zheng%2Dwei+Tao%2C+and+Chongyang+Tao%2E+2022%2E+Sk2%3A+Integrating+implicit+sentiment+knowledge+and+explicit+syntax+knowledge+for+aspect%2Dbased+sentiment+analysis%2E+)
21. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, ACL/IJCNLP 2021, (Volume 1: Long Pa… [find](https://scholar.google.com/scholar?q=Xiang+Lisa+Li+and+Percy+Liang%2E+2021%2E+Prefix%2Dtuning%3A+Optimizing+continuous+prompts+for+generation%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+11t)
22. Yankai Lin, Zhiyuan Liu, Maosong Sun, Yang Liu, and Xuan Zhu. 2015. Learning entity and relation embeddings for knowledge graph completion. In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence, January 25-30, 2015, Austin, Texas, USA, pages 2181–2187. AAAI Press. [find](https://scholar.google.com/scholar?q=Yankai+Lin%2C+Zhiyuan+Liu%2C+Maosong+Sun%2C+Yang+Liu%2C+and+Xuan+Zhu%2E+2015%2E+Learning+entity+and+relation+embeddings+for+knowledge+graph+completion%2E+In+Proceedings+of+the+Twenty%2DNinth+AAAI+Conference+on+Artifi)
23. Xiao Liu, Kaixuan Ji, Yicheng Fu, Zhengxiao Du, Zhilin Yang, and Jie Tang. 2021. P-tuning v2: Prompt tuning can be comparable to fine-tuning universally across scales and tasks. CoRR, abs/2110.07602. [find](https://scholar.google.com/scholar?q=Xiao+Liu%2C+Kaixuan+Ji%2C+Yicheng+Fu%2C+Zhengxiao+Du%2C+Zhilin+Yang%2C+and+Jie+Tang%2E+2021%2E+P%2Dtuning+v2%3A+Prompt+tuning+can+be+comparable+to+fine%2Dtuning+universally+across+scales+and+tasks%2E+CoRR%2C+abs%2F2110%2E07602%2E) [our card](https://md.research.elamur.ai/free/papers/liu-2021-p-tuning-v2-prompt-tuning/)
24. Takeru Miyato, Andrew M. Dai, and Ian J. Goodfellow. 2017. Adversarial training methods for semi-supervised text classification. In 5th International Conference on Learning Representations, ICLR 2017, Toulon, France, April 24-26, 2017, Conference Track Proceedings. OpenReview.net. [find](https://scholar.google.com/scholar?q=Takeru+Miyato%2C+Andrew+M%2E+Dai%2C+and+Ian+J%2E+Goodfellow%2E+2017%2E+Adversarial+training+methods+for+semi%2Dsupervised+text+classification%2E+In+5th+International+Conference+on+Learning+Representations%2C+ICLR+2017%2C) [our card](https://md.research.elamur.ai/free/papers/miyato-2016-adversarial-training-methods-semi-supervised/)
25. Maximilian Nickel, Volker Tresp, and Hans-Peter Kriegel. 2011. A three-way model for collective learning on multi-relational data. In Proceedings of the 28th International Conference on Machine Learning, ICML 2011, Bellevue, Washington, USA, June 28 - July 2, 2011, pages 809–816. Omnipress. [find](https://scholar.google.com/scholar?q=Maximilian+Nickel%2C+Volker+Tresp%2C+and+Hans%2DPeter+Kriegel%2E+2011%2E+A+three%2Dway+model+for+collective+learning+on+multi%2Drelational+data%2E+In+Proceedings+of+the+28th+International+Conference+on+Machine+Learni)
26. Apoorv Saxena, Adrian Kochsiek, and Rainer Gemulla. 2022. Sequence-to-sequence knowledge graph completion and question answering. CoRR, abs/2203.10321. [find](https://scholar.google.com/scholar?q=Apoorv+Saxena%2C+Adrian+Kochsiek%2C+and+Rainer+Gemulla%2E+2022%2E+Sequence%2Dto%2Dsequence+knowledge+graph+completion+and+question+answering%2E+CoRR%2C+abs%2F2203%2E10321%2E)
27. Taylor Shin, Yasaman Razeghi, Robert L. Logan IV, Eric Wallace, and Sameer Singh. 2020. Autoprompt: Eliciting knowledge from language models with automatically generated prompts. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, EMNLP 2020, Online, November 1… [find](https://scholar.google.com/scholar?q=Taylor+Shin%2C+Yasaman+Razeghi%2C+Robert+L%2E+Logan+IV%2C+Eric+Wallace%2C+and+Sameer+Singh%2E+2020%2E+Autoprompt%3A+Eliciting+knowledge+from+language+models+with+automatically+generated+prompts%2E+In+Proceedings+of+the) [our card](https://md.research.elamur.ai/free/papers/shin-2020-autoprompt-eliciting-knowledge-language-models/)
28. Zhiqing Sun, Zhi-Hong Deng, Jian-Yun Nie, and Jian Tang. 2019. Rotate: Knowledge graph embedding by relational rotation in complex space. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. [find](https://scholar.google.com/scholar?q=Zhiqing+Sun%2C+Zhi%2DHong+Deng%2C+Jian%2DYun+Nie%2C+and+Jian+Tang%2E+2019%2E+Rotate%3A+Knowledge+graph+embedding+by+relational+rotation+in+complex+space%2E+In+7th+International+Conference+on+Learning+Representations%2C+I)
29. Kristina Toutanova and Danqi Chen. 2015. Observed versus latent features for knowledge base and text inference. In Proceedings of the 3rd workshop on continuous vector space models and their compositionality, pages 57–66. [find](https://scholar.google.com/scholar?q=Kristina+Toutanova+and+Danqi+Chen%2E+2015%2E+Observed+versus+latent+features+for+knowledge+base+and+text+inference%2E+In+Proceedings+of+the+3rd+workshop+on+continuous+vector+space+models+and+their+compositi)
30. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenRe-view.net. [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+2019%2E+Decoupled+weight+decay+regularization%2E+In+7th+International+Conference+on+Learning+Representations%2C+ICLR+2019%2C+New+Orleans%2C+LA%2C+USA%2C+May+6%2D9%2C+2019%2E+OpenRe%2Dview%2E)
31. Théo Trouillon, Johannes Welbl, Sebastian Riedel, Éric Gaussier, and Guillaume Bouchard. 2016. Complex embeddings for simple link prediction. In Proceedings of the 33nd International Conference on Machine Learning, ICML 2016, New York City, NY, USA, June 19-24, 2016, volume 48 of JMLR Workshop and C… [find](https://scholar.google.com/scholar?q=Th%C3%A9o+Trouillon%2C+Johannes+Welbl%2C+Sebastian+Riedel%2C+%C3%89ric+Gaussier%2C+and+Guillaume+Bouchard%2E+2016%2E+Complex+embeddings+for+simple+link+prediction%2E+In+Proceedings+of+the+33nd+International+Conference+on+Mac)
32. Xin Lv, Yankai Lin, Yixin Cao, Lei Hou, Juanzi Li, Zhiyuan Liu, Peng Li, and Jie Zhou. 2022. Do pre-trained models benefit knowledge graph completion? a reliable evaluation and a reasonable approach. In Shikhar Vashishth, Soumya Sanyal, Vikram Nitin, and Partha P. Talukdar. 2020. Composition-based m… [find](https://scholar.google.com/scholar?q=Xin+Lv%2C+Yankai+Lin%2C+Yixin+Cao%2C+Lei+Hou%2C+Juanzi+Li%2C+Zhiyuan+Liu%2C+Peng+Li%2C+and+Jie+Zhou%2E+2022%2E+Do+pre%2Dtrained+models+benefit+knowledge+graph+completion%3F+a+reliable+evaluation+and+a+reasonable+approach%2E+)
33. Eric Wallace, Shi Feng, Nikhil Kandpal, Matt Gardner, and Sameer Singh. 2019. Universal adversarial triggers for attacking and analyzing NLP. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Proce… [find](https://scholar.google.com/scholar?q=Eric+Wallace%2C+Shi+Feng%2C+Nikhil+Kandpal%2C+Matt+Gardner%2C+and+Sameer+Singh%2E+2019%2E+Universal+adversarial+triggers+for+attacking+and+analyzing+NLP%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods)
34. Bo Wang, Tao Shen, Guodong Long, Tianyi Zhou, Ying Wang, and Yi Chang. 2021a. Structure-augmented text representation learning for efficient knowledge graph completion. In WWW ’21: The Web Conference 2021, Virtual Event / Ljubljana, Slovenia, April 19-23, 2021, pages 1737–1748. ACM / IW3C2. [find](https://scholar.google.com/scholar?q=Bo+Wang%2C+Tao+Shen%2C+Guodong+Long%2C+Tianyi+Zhou%2C+Ying+Wang%2C+and+Yi+Chang%2E+2021a%2E+Structure%2Daugmented+text+representation+learning+for+efficient+knowledge+graph+completion%2E+In+WWW+%E2%80%9921%3A+The+Web+Conference+)
35. Xiaozhi Wang, Tianyu Gao, Zhaocheng Zhu, Zhengyan Zhang, Zhiyuan Liu, Juanzi Li, and Jian Tang. 2021b. KEPLER: A unified model for knowledge embedding and pre-trained language representation. Trans. Assoc. Comput. Linguistics, 9:176–194. [find](https://scholar.google.com/scholar?q=Xiaozhi+Wang%2C+Tianyu+Gao%2C+Zhaocheng+Zhu%2C+Zhengyan+Zhang%2C+Zhiyuan+Liu%2C+Juanzi+Li%2C+and+Jian+Tang%2E+2021b%2E+KEPLER%3A+A+unified+model+for+knowledge+embedding+and+pre%2Dtrained+language+representation%2E+Trans%2E+A)
36. A time-aware knowledge graph embedding via temporal rotation. In Proceedings of the 28th International Conference on Computational Linguistics, COLING 2020, Barcelona, Spain (Online), December 8-13, 2020, pages 1583–1593. International Committee on Computational Linguistics. [find](https://scholar.google.com/scholar?q=A+time%2Daware+knowledge+graph+embedding+via+temporal+rotation%2E+In+Proceedings+of+the+28th+International+Conference+on+Computational+Linguistics%2C+COLING+2020%2C+Barcelona%2C+Spain+%28Online%29%2C+December+8%2D13%2C+2)
37. Bishan Yang, Wen-tau Yih, Xiaodong He, Jianfeng Gao, and Li Deng. 2015. Embedding entities and relations for learning and inference in knowledge bases. In 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, May 7-9, 2015, Conference Track Proceedings. [find](https://scholar.google.com/scholar?q=Bishan+Yang%2C+Wen%2Dtau+Yih%2C+Xiaodong+He%2C+Jianfeng+Gao%2C+and+Li+Deng%2E+2015%2E+Embedding+entities+and+relations+for+learning+and+inference+in+knowledge+bases%2E+In+3rd+International+Conference+on+Learning+Repr)
38. Liang Yao, Chengsheng Mao, and Yuan Luo. 2019. KG-BERT: BERT for knowledge graph completion. CoRR, abs/1909.03193. [find](https://scholar.google.com/scholar?q=Liang+Yao%2C+Chengsheng+Mao%2C+and+Yuan+Luo%2E+2019%2E+KG%2DBERT%3A+BERT+for+knowledge+graph+completion%2E+CoRR%2C+abs%2F1909%2E03193%2E)
39. Manzil Zaheer, Guru Guruganesh, Kumar Avinava Dubey, Joshua Ainslie, Chris Alberti, Santiago On-tanon, Philip Pham, Anirudh Ravula, Qifan Wang, Li Yang, and Amr Ahmed. 2020. Big bird: Transformers for longer sequences. In Advances in Neural Information Processing Systems, volume 33, pages 17283–1729… [find](https://scholar.google.com/scholar?q=Manzil+Zaheer%2C+Guru+Guruganesh%2C+Kumar+Avinava+Dubey%2C+Joshua+Ainslie%2C+Chris+Alberti%2C+Santiago+On%2Dtanon%2C+Philip+Pham%2C+Anirudh+Ravula%2C+Qifan+Wang%2C+Li+Yang%2C+and+Amr+Ahmed%2E+2020%2E+Big+bird%3A+Transformers+for) [our card](https://md.research.elamur.ai/free/papers/zaheer-2020-big-bird-transformers-longer-sequences/)
40. Yufei Wang, Can Xu, Qingfeng Sun, Huang Hu, Chongyang Tao, Xiubo Geng, and Daxin Jiang. 2022. PromDA: Prompt-based data augmentation for low-resource NLU tasks. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4242– 4255, Dubli… [find](https://scholar.google.com/scholar?q=Yufei+Wang%2C+Can+Xu%2C+Qingfeng+Sun%2C+Huang+Hu%2C+Chongyang+Tao%2C+Xiubo+Geng%2C+and+Daxin+Jiang%2E+2022%2E+PromDA%3A+Prompt%2Dbased+data+augmentation+for+low%2Dresource+NLU+tasks%2E+In+Proceedings+of+the+60th+Annual+Meeti)
41. Zhen Wang, Jianwen Zhang, Jianlin Feng, and Zheng Chen. 2014. Knowledge graph embedding by translating on hyperplanes. In Proceedings of the Twenty-Eighth AAAI Conference on Artificial Intelligence, July 27 -31, 2014, Québec City, Québec, Canada, pages 1112–1119. AAAI Press. [find](https://scholar.google.com/scholar?q=Zhen+Wang%2C+Jianwen+Zhang%2C+Jianlin+Feng%2C+and+Zheng+Chen%2E+2014%2E+Knowledge+graph+embedding+by+translating+on+hyperplanes%2E+In+Proceedings+of+the+Twenty%2DEighth+AAAI+Conference+on+Artificial+Intelligence%2C+J)
42. Zeguan Xiao, Jiarun Wu, Qingliang Chen, and Congjian Deng. 2021. BERT4GCN: Using BERT intermediate layers to augment GCN for aspect-based sentiment classification. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 9193–9200, Online and Punta Cana, Domin… [find](https://scholar.google.com/scholar?q=Zeguan+Xiao%2C+Jiarun+Wu%2C+Qingliang+Chen%2C+and+Congjian+Deng%2E+2021%2E+BERT4GCN%3A+Using+BERT+intermediate+layers+to+augment+GCN+for+aspect%2Dbased+sentiment+classification%2E+In+Proceedings+of+the+2021+Conferenc)
43. Xin Xie, Ningyu Zhang, Zhoubo Li, Shumin Deng, Hui Chen, Feiyu Xiong, Mosha Chen, and Huajun Chen. 2022. From discrimination to generation: Knowledge graph completion with generative transformer. CoRR, abs/2202.02113. — [arXiv:2202.02113](https://arxiv.org/abs/2202.02113) [find](https://scholar.google.com/scholar?q=Xin+Xie%2C+Ningyu+Zhang%2C+Zhoubo+Li%2C+Shumin+Deng%2C+Hui+Chen%2C+Feiyu+Xiong%2C+Mosha+Chen%2C+and+Huajun+Chen%2E+2022%2E+From+discrimination+to+generation%3A+Knowledge+graph+completion+with+generative+transformer%2E+CoRR)
44. Chengjin Xu, Mojtaba Nayyeri, Fouad Alkhoury, Jens Lehmann, and Hamed Shariat Yazdi. 2019. Temporal knowledge graph embedding model based on additive time series decomposition. CoRR, abs/1911.07893. — [arXiv:1911.07893](https://arxiv.org/abs/1911.07893) [find](https://scholar.google.com/scholar?q=Chengjin+Xu%2C+Mojtaba+Nayyeri%2C+Fouad+Alkhoury%2C+Jens+Lehmann%2C+and+Hamed+Shariat+Yazdi%2E+2019%2E+Temporal+knowledge+graph+embedding+model+based+on+additive+time+series+decomposition%2E+CoRR%2C+abs%2F1911%2E07893%2E+%E2%80%94)
45. Chengjin Xu, Mojtaba Nayyeri, Fouad Alkhoury, Hamed Shariat Yazdi, and Jens Lehmann. 2020. Tero: A Dataset [find](https://scholar.google.com/scholar?q=Chengjin+Xu%2C+Mojtaba+Nayyeri%2C+Fouad+Alkhoury%2C+Hamed+Shariat+Yazdi%2C+and+Jens+Lehmann%2E+2020%2E+Tero%3A+A+Dataset)