---
title: "QA-KGNet: 一种语言模型驱动的知识图谱问答模型 — 乔少杰 2023"
description: "The paper addresses question answering that draws on both a pre-trained language model and a knowledge graph."
authors: ["乔少杰", "杨国平", "于泳", "韩楠", "覃晓", "屈露露", "冉黎琼", "李贺"]
year: 2023
venue: "软件学报 (Journal of Software)"
source: "https://jos.org.cn/jos/article/abstract/6882"
doi: "10.13328/j.cnki.jos.006882"
licence: "Free access (jos.org.cn), no open licence stated — readonly; © 中国科学院软件研究所 / 软件学报"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/qiao-2023-qa-kgnet-language/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/qiao-2023-qa-kgnet-language/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2023](https://md.research.elamur.ai/free/papers/2023/page/5/) / QA-KGNet: 一种语言模型驱动的知识图谱问答模型

# QA-KGNet: 一种语言模型驱动的知识图谱问答模型

**Authors:** 乔少杰, 杨国平, 于泳, 韩楠, 覃晓, 屈露露, 冉黎琼, 李贺 · **Year:** 2023 · **Venue:** 软件学报 (Journal of Software) · **Licence:** Free access (jos.org.cn), no open licence stated — readonly; © 中国科学院软件研究所 / 软件学报

**Links:** [Publisher's page](https://doi.org/10.13328/j.cnki.jos.006882) · [PDF](https://www.jos.org.cn//jos/article/pdf/6882) · [DOI 10.13328/j.cnki.jos.006882](https://doi.org/10.13328/j.cnki.jos.006882)

**Citations:** [1](#cited-by) in our library, of the 140673 papers whose bibliographies we read

## Summary
The paper addresses question answering that draws on both a pre-trained language model and a knowledge graph. Two issues motivate it. First, retrieving the few-hop neighbourhood of topic entities pulls in many nodes that have nothing to do with the question. Second, existing LM+KG systems process the text and the graph separately, which weakens structured reasoning. QA-KGNet extends the earlier QA-GNN design in three steps:

- **Joint graph:** the question and a candidate answer are concatenated into a QA context. This context becomes an extra node linked to the question entities and answer-choice entities of a retrieved KG subgraph.
- **Relevance scoring:** RoBERTa scores how relevant each KG node is to that context, and the score is attached to the node as an additional feature.
- **Reasoning and output:** a multi-head graph attention network passes messages over several rounds. Graph attention pooling then downsamples the graph. An MLP combines the LM representation, the context node and the pooled graph to score each answer.

On CommonsenseQA, OpenBookQA and MedQA-USMLE, the authors report that QA-KGNet beats the baseline models. They also claim it is more interpretable. The work matters because it fuses text and graph knowledge in one representation rather than keeping them apart.

## Key points
- Published in 软件学报 (Journal of Software), 2023, 34(10): 4584–4600, by 乔少杰 (Qiao Shao-Jie) et al. (Chengdu University of Information Technology and partner institutions).
- Subgraphs contain the nodes on paths of up to g = 3 hops between topic entities. On ConceptNet this yields roughly 400 nodes per question–answer pair, many of them noise.
- Relevance scoring feeds the concatenated context and entity text into RoBERTa as the encoder, with RobertaLMHead as the scoring head.
- QA-GNN uses a single set of attention weights. The multi-head attention here is meant to capture graph information from several perspectives, which the authors argue improves generalization.
- The provided text omits the accuracy figures, so specific scores cannot be cited. The references suggest the medical setup uses UMLS/DrugBank-based knowledge, but this is not confirmed in the excerpt.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) · [Natural language processing](https://md.research.elamur.ai/free/shelves/nlp/)

## Cited by

Cited by

1 paper of our library cites it (of the 140673 whose bibliographies we read); each has a page here.

- [华谱通: 基于知识推理的家谱问答大语言模型](https://md.research.elamur.ai/free/papers/wu-2025-huaputong-large-language/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@article{2023qa,
  title = {{QA-KGNet: 一种语言模型驱动的知识图谱问答模型}},
  author = {乔少杰 and 杨国平 and 于泳 and 韩楠 and 覃晓 and 屈露露 and 冉黎琼 and 李贺},
  journal = {软件学报 (Journal of Software)},
  year = {2023},
  doi = {10.13328/j.cnki.jos.006882},
  url = {https://doi.org/10.13328/j.cnki.jos.006882}
}
```

- [← previous ProofNet: Autoformalizing and Formally Proving Undergraduate-Level Mathematics](https://md.research.elamur.ai/free/papers/azerbayev-2023-proofnet-autoformalizing-formally-proving-undergraduate/)
- [next → QUDeval: The Evaluation of Questions Under Discussion Discourse Parsing](https://md.research.elamur.ai/free/papers/wu-2023-qudeval-evaluation-questions-under-discussion/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/papers/2023/page/5/)

## References

51 references, 6 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Xiong F, Wang HZ. Mining simple path traversal patterns in knowledge graph. Journal of Web Engineering, 2022, 21(2): 258  271. [doi: 10.13052/jwe1540-9589.2128] — [doi:10.13052/jwe1540-9589.2128](https://doi.org/10.13052/jwe1540-9589.2128) [find](https://scholar.google.com/scholar?q=Xiong+F%2C+Wang+HZ%2E+Mining+simple+path+traversal+patterns+in+knowledge+graph%2E+Journal+of+Web+Engineering%2C+2022%2C+21%282%29%3A+258+%EF%80%AD+271%2E+%5Bdoi%3A+10%2E13052%2Fjwe1540%2D9589%2E2128%5D+%E2%80%94+%5Bdoi%3A10%2E13052%2Fjwe1540%2D9589%2E2128%5D%28htt)
2. Jamehshourani M, Fatemi A, Nematbakhsh MA. Implicit relation-based question answering to answer simple questions over DBpedia. Turkish Journal of Electrical Engineering and Computer Sciences, 2020, 28(3):1474  1490. [doi: 10.3906/elk-1908-102] — [doi:10.3906/elk-1908-102](https://doi.org/10.3906/elk-1908-102) [find](https://scholar.google.com/scholar?q=Jamehshourani+M%2C+Fatemi+A%2C+Nematbakhsh+MA%2E+Implicit+relation%2Dbased+question+answering+to+answer+simple+questions+over+DBpedia%2E+Turkish+Journal+of+Electrical+Engineering+and+Computer+Sciences%2C+2020%2C+28)
3. Jiang K, Wu DK, Jiang H. FreebaseQA: A new factoid QA data set matching trivia-style question-answer pairs with Freebase. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. New York: ACM, 2019. 318  323. [doi: 10.… [find](https://scholar.google.com/scholar?q=Jiang+K%2C+Wu+DK%2C+Jiang+H%2E+FreebaseQA%3A+A+new+factoid+QA+data+set+matching+trivia%2Dstyle+question%2Danswer+pairs+with+Freebase%2E+In%3A+Proc%2E+of+the+2019+Conf%2E+of+the+North+American+Chapter+of+the+Association+f)
4. Lu XY, Wang LF, Jiang ZJ, He SC, Liu SZ. MMKRL: A robust embedding approach for multi-modal knowledge graph representation learning. Applied Intelligence, 2022, 52(7): 7480  7497. [doi: 10.1007/s10489-021-02693-9] — [doi:10.1007/s10489-021-02693-9](https://doi.org/10.1007/s10489-021-02693-9) [find](https://scholar.google.com/scholar?q=Lu+XY%2C+Wang+LF%2C+Jiang+ZJ%2C+He+SC%2C+Liu+SZ%2E+MMKRL%3A+A+robust+embedding+approach+for+multi%2Dmodal+knowledge+graph+representation+learning%2E+Applied+Intelligence%2C+2022%2C+52%287%29%3A+7480+%EF%80%AD+7497%2E+%5Bdoi%3A+10%2E1007%2Fs1048)
5. Menin A, Torchelsen RP, Nedel LP. The effects of VR in training simulators: Exploring perception and knowledge gain. Computers & Graphics, 2022, 102(5): 402  412. [doi: 10.1016/j.cag.2021.09.015] — [doi:10.1016/j.cag.2021.09.015](https://doi.org/10.1016/j.cag.2021.09.015) [find](https://scholar.google.com/scholar?q=Menin+A%2C+Torchelsen+RP%2C+Nedel+LP%2E+The+effects+of+VR+in+training+simulators%3A+Exploring+perception+and+knowledge+gain%2E+Computers+%26+Graphics%2C+2022%2C+102%285%29%3A+402+%EF%80%AD+412%2E+%5Bdoi%3A+10%2E1016%2Fj%2Ecag%2E2021%2E09%2E015%5D+%E2%80%94+%5B)
6. Li JZ, Hou L. Reviews on knowledge graph research. Journal of Shanxi University (Nat. Sci. Ed.), 2017, 40(3): 454  459 (in Chinese with English abstract). [doi: 10.13451/j.cnki.shanxi.univ(nat.sci.).2017.03.008] — [doi:10.13451/j.cnki.shanxi.univ(nat.sci.).2017.03.008](https://doi.org/10.13451/j.cnki.shanxi.univ%28nat.sci.%29.2017.03.008) [find](https://scholar.google.com/scholar?q=Li+JZ%2C+Hou+L%2E+Reviews+on+knowledge+graph+research%2E+Journal+of+Shanxi+University+%28Nat%2E+Sci%2E+Ed%2E%29%2C+2017%2C+40%283%29%3A+454+%EF%80%AD+459+%28in+Chinese+with+English+abstract%29%2E+%5Bdoi%3A+10%2E13451%2Fj%2Ecnki%2Eshanxi%2Euniv%28nat%2Esci%2E%29%2E)
7. Nayyeri M, Cil GM, Vahdati S, Osborne F, Rahman M, Angioni S, Salatino AA, Recupero DR, Vassilyeva R, Motta E, Lehmann J. Trans4E: Link prediction on scholarly knowledge graphs. Neurocomputing, 2021, 461(3): 530  542. [doi: 10.1016/j.neucom.2021.02.100] — [doi:10.1016/j.neucom.2021.02.100](https://doi.org/10.1016/j.neucom.2021.02.100) [find](https://scholar.google.com/scholar?q=Nayyeri+M%2C+Cil+GM%2C+Vahdati+S%2C+Osborne+F%2C+Rahman+M%2C+Angioni+S%2C+Salatino+AA%2C+Recupero+DR%2C+Vassilyeva+R%2C+Motta+E%2C+Lehmann+J%2E+Trans4E%3A+Link+prediction+on+scholarly+knowledge+graphs%2E+Neurocomputing%2C+2021%2C+)
8. Dessì D, Osborne F, Recupero DR, Buscaldi D, Motta E. Generating knowledge graphs by employing natural language processing and machine learning techniques within the scholarly domain. Future Generation Computer Systems, 2021, 116(7): 253  264. [doi: 10.1016/j.future.2020.10.026] — [doi:10.1016/j.future.2020.10.026](https://doi.org/10.1016/j.future.2020.10.026) [find](https://scholar.google.com/scholar?q=Dess%C3%AC+D%2C+Osborne+F%2C+Recupero+DR%2C+Buscaldi+D%2C+Motta+E%2E+Generating+knowledge+graphs+by+employing+natural+language+processing+and+machine+learning+techniques+within+the+scholarly+domain%2E+Future+Generatio)
9. Lin QK, Zhu YF, Lu H, Shi K, Niu ZD. Improving university faculty evaluations via multi-view knowledge graph. Future Generation Computer Systems, 2021, 117(5): 181  192. [doi: 10.1016/j.future.2020.11.021] — [doi:10.1016/j.future.2020.11.021](https://doi.org/10.1016/j.future.2020.11.021) [find](https://scholar.google.com/scholar?q=Lin+QK%2C+Zhu+YF%2C+Lu+H%2C+Shi+K%2C+Niu+ZD%2E+Improving+university+faculty+evaluations+via+multi%2Dview+knowledge+graph%2E+Future+Generation+Computer+Systems%2C+2021%2C+117%285%29%3A+181+%EF%80%AD+192%2E+%5Bdoi%3A+10%2E1016%2Fj%2Efuture%2E2020%2E1)
10. Yue Q, Li X, Li D. Chinese relation extraction on forestry knowledge graph construction. Computer Systems: Science & Engineering, 2021, 37(3): 423  442. [doi: 10.32604/csse.2021.014448] — [doi:10.32604/csse.2021.014448](https://doi.org/10.32604/csse.2021.014448) [find](https://scholar.google.com/scholar?q=Yue+Q%2C+Li+X%2C+Li+D%2E+Chinese+relation+extraction+on+forestry+knowledge+graph+construction%2E+Computer+Systems%3A+Science+%26+Engineering%2C+2021%2C+37%283%29%3A+423+%EF%80%AD+442%2E+%5Bdoi%3A+10%2E32604%2Fcsse%2E2021%2E014448%5D+%E2%80%94+%5Bdoi%3A10%2E326)
11. Yu CM, Wang F, Liu YH, An L. Research on knowledge graph alignment model based on deep learning. Expert Systems with Applications, 2021, 186(4): 115  129. [doi: 10.1016/j.eswa.2021.115768] — [doi:10.1016/j.eswa.2021.115768](https://doi.org/10.1016/j.eswa.2021.115768) [find](https://scholar.google.com/scholar?q=Yu+CM%2C+Wang+F%2C+Liu+YH%2C+An+L%2E+Research+on+knowledge+graph+alignment+model+based+on+deep+learning%2E+Expert+Systems+with+Applications%2C+2021%2C+186%284%29%3A+115+%EF%80%AD+129%2E+%5Bdoi%3A+10%2E1016%2Fj%2Eeswa%2E2021%2E115768%5D+%E2%80%94+%5Bdoi%3A10%2E)
12. Petroni F, Rocktaschel T, Riedel S, Lewis PSH, Bakhtin A, Wu YX, Miller AH. Language models as knowledge bases? In: Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing. Washington: ACL, 2019. 2463  2473. [doi: 10.18653/v1/D19-1250] [find](https://scholar.google.com/scholar?q=Petroni+F%2C+Rocktaschel+T%2C+Riedel+S%2C+Lewis+PSH%2C+Bakhtin+A%2C+Wu+YX%2C+Miller+AH%2E+Language+models+as+knowledge+bases%3F+In%3A+Proc%2E+of+the+2019+Conf%2E+on+Empirical+Methods+in+Natural+Language+Processing%2E+Washing) [our card](https://md.research.elamur.ai/free/papers/petroni-2019-language-models-as-knowledge-bases/)
13. Bosselut A, Rashkin H, Sap M, Malaviya C, Celikyilmaz A, Choi Y. COMET: Commonsense transformers for automatic knowledge graph construction. In: Proc. of the 57th Conf. of the Association for Computational Linguistics. Washington: ACL, 2019. 4762  4779. [doi: 10.18653/v1/p19-1470] [find](https://scholar.google.com/scholar?q=Bosselut+A%2C+Rashkin+H%2C+Sap+M%2C+Malaviya+C%2C+Celikyilmaz+A%2C+Choi+Y%2E+COMET%3A+Commonsense+transformers+for+automatic+knowledge+graph+construction%2E+In%3A+Proc%2E+of+the+57th+Conf%2E+of+the+Association+for+Computat) [our card](https://md.research.elamur.ai/free/papers/bosselut-2019-comet-commonsense-transformers-automatic-knowledge/)
14. Speer R, Chin J, Havasi C. ConceptNet 5.5: An open multilingual graph of general knowledge. In: Proc. of the 31st AAAI Conf. on Artificial Intelligence. San Francisco: AAAI, 2017. 4444  4451. [find](https://scholar.google.com/scholar?q=Speer+R%2C+Chin+J%2C+Havasi+C%2E+ConceptNet+5%2E5%3A+An+open+multilingual+graph+of+general+knowledge%2E+In%3A+Proc%2E+of+the+31st+AAAI+Conf%2E+on+Artificial+Intelligence%2E+San+Francisco%3A+AAAI%2C+2017%2E+4444+%EF%80%AD+4451%2E)
15. Huang SS, Zhu KQ, Liao QZ, Shen LB, Zhao YG. Enhanced story representation by ConceptNet for predicting story endings. In: Proc. of the 29th ACM Int’l Conf. on Information and Knowledge Management. New York: ACM, 2020. 3277  3280. [doi: 10.1145/3340531.3417466] — [doi:10.1145/3340531.3417466](https://doi.org/10.1145/3340531.3417466) [find](https://scholar.google.com/scholar?q=Huang+SS%2C+Zhu+KQ%2C+Liao+QZ%2C+Shen+LB%2C+Zhao+YG%2E+Enhanced+story+representation+by+ConceptNet+for+predicting+story+endings%2E+In%3A+Proc%2E+of+the+29th+ACM+Int%E2%80%99l+Conf%2E+on+Information+and+Knowledge+Management%2E+Ne)
16. Raffel C, Shazeer N, Roberts A, Lee K, Narang S, Matena M, Zhou YQ, Li W, Liu PJ. Exploring the limits of transfer learning with a unified text-to-text transformer. Journal of Machine Learning Research, 2020, 21(2): 1  14. [find](https://scholar.google.com/scholar?q=Raffel+C%2C+Shazeer+N%2C+Roberts+A%2C+Lee+K%2C+Narang+S%2C+Matena+M%2C+Zhou+YQ%2C+Li+W%2C+Liu+PJ%2E+Exploring+the+limits+of+transfer+learning+with+a+unified+text%2Dto%2Dtext+transformer%2E+Journal+of+Machine+Learning+Researc)
17. Liao WX, Zeng B, Yin XW, Wei PF. An improved aspect-category sentiment analysis model for text sentiment analysis based on RoBERTa. Applied Intelligence, 2021, 51(6): 3522  3533. [find](https://scholar.google.com/scholar?q=Liao+WX%2C+Zeng+B%2C+Yin+XW%2C+Wei+PF%2E+An+improved+aspect%2Dcategory+sentiment+analysis+model+for+text+sentiment+analysis+based+on+RoBERTa%2E+Applied+Intelligence%2C+2021%2C+51%286%29%3A+3522+%EF%80%AD+3533%2E)
18. Lin YB, Chen XY, Chen JM, Ren X. KagNet: Knowledge-aware graph networks for commonsense reasoning. In: Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing. New York: ACM, 2019. 2829  2839. [doi: 10.18653/v1/D19-1282] [find](https://scholar.google.com/scholar?q=Lin+YB%2C+Chen+XY%2C+Chen+JM%2C+Ren+X%2E+KagNet%3A+Knowledge%2Daware+graph+networks+for+commonsense+reasoning%2E+In%3A+Proc%2E+of+the+2019+Conf%2E+on+Empirical+Methods+in+Natural+Language+Processing%2E+New+York%3A+ACM%2C+2019%2E)
19. Bao JW, Duan N, Yan Z, Zhou M, Zhao TJ. Constraint-based question answering with knowledge graph. In: Proc. of the 26th Int’l Conf. on Computational Linguistics. Washington: ACL, 2016. 2503  2514. [find](https://scholar.google.com/scholar?q=Bao+JW%2C+Duan+N%2C+Yan+Z%2C+Zhou+M%2C+Zhao+TJ%2E+Constraint%2Dbased+question+answering+with+knowledge+graph%2E+In%3A+Proc%2E+of+the+26th+Int%E2%80%99l+Conf%2E+on+Computational+Linguistics%2E+Washington%3A+ACL%2C+2016%2E+2503+%EF%80%AD+2514%2E)
20. Sun HT, Dhingra B, Zaheer M, Mazaitis K, Salakhutdinov R, Cohen WW. Open domain question answering using early fusion of knowledge bases and text. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing. New York: ACM, 2018. 4231  4242. [doi: 10.18653/v1/d18-1455] [find](https://scholar.google.com/scholar?q=Sun+HT%2C+Dhingra+B%2C+Zaheer+M%2C+Mazaitis+K%2C+Salakhutdinov+R%2C+Cohen+WW%2E+Open+domain+question+answering+using+early+fusion+of+knowledge+bases+and+text%2E+In%3A+Proc%2E+of+the+2018+Conf%2E+on+Empirical+Methods+in+N)
21. Wang XY, Kapanipathi P, Musa R, Yu M, Talamadupula K, Abdelaziz I, Chang M, Fokoue A, Makni B, Mattei N, Witbrock M. Improving natural language inference using external knowledge in the science questions domain. In: Proc. of the 33rd AAAI Conf. on Artificial Intelligence. San Francisco: AAAI, 2019.… — [doi:10.1609/aaai.v33i01.33017208](https://doi.org/10.1609/aaai.v33i01.33017208) [find](https://scholar.google.com/scholar?q=Wang+XY%2C+Kapanipathi+P%2C+Musa+R%2C+Yu+M%2C+Talamadupula+K%2C+Abdelaziz+I%2C+Chang+M%2C+Fokoue+A%2C+Makni+B%2C+Mattei+N%2C+Witbrock+M%2E+Improving+natural+language+inference+using+external+knowledge+in+the+science+questi)
22. Feng YL, Chen XY, Lin YB, Wang PF, Yan J, Ren X. Scalable multi-hop relational reasoning for knowledge-aware question answering. In: Proc. of the 2020 Conf. on Empirical Methods in Natural Language Processing. New York: ACM, 2020. 1295  1306. [doi: 10.18653/v1/2020.emnlp-main.99] [find](https://scholar.google.com/scholar?q=Feng+YL%2C+Chen+XY%2C+Lin+YB%2C+Wang+PF%2C+Yan+J%2C+Ren+X%2E+Scalable+multi%2Dhop+relational+reasoning+for+knowledge%2Daware+question+answering%2E+In%3A+Proc%2E+of+the+2020+Conf%2E+on+Empirical+Methods+in+Natural+Language+Pr)
23. Yasunaga M, Ren HY, Bosselut A, Liang P, Leskovec J. QA-GNN: Reasoning with language models and knowledge graphs for question answering. In: Proc. of the 2021 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Washington: ACL, 2021. 535… [find](https://scholar.google.com/scholar?q=Yasunaga+M%2C+Ren+HY%2C+Bosselut+A%2C+Liang+P%2C+Leskovec+J%2E+QA%2DGNN%3A+Reasoning+with+language+models+and+knowledge+graphs+for+question+answering%2E+In%3A+Proc%2E+of+the+2021+Conf%2E+of+the+North+American+Chapter+of+th)
24. Pan XM, Sun K, Yu D, Chen JS, Ji H, Cardie C, Yu D. Improving question answering with external knowledge. In: Proc. of the 2nd Workshop on Machine Reading for Question Answering. Washington: ACL, 2019. 27  37. [doi: 10.18653/v1/D19-5804] [find](https://scholar.google.com/scholar?q=Pan+XM%2C+Sun+K%2C+Yu+D%2C+Chen+JS%2C+Ji+H%2C+Cardie+C%2C+Yu+D%2E+Improving+question+answering+with+external+knowledge%2E+In%3A+Proc%2E+of+the+2nd+Workshop+on+Machine+Reading+for+Question+Answering%2E+Washington%3A+ACL%2C+2019)
25. Mihaylov T, Frank A. Knowledgeable reader: Enhancing cloze-style reading comprehension with external commonsense knowledge. In: Proc. of the 56th Annual Meeting of the Association for Computational Linguistics. Washington: ACL, 2018. 821  832. [doi: 10.18653/v1/P18-1076] [find](https://scholar.google.com/scholar?q=Mihaylov+T%2C+Frank+A%2E+Knowledgeable+reader%3A+Enhancing+cloze%2Dstyle+reading+comprehension+with+external+commonsense+knowledge%2E+In%3A+Proc%2E+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Li)
26. Bosselut A, Bras RL, Choi YJ. Dynamic neuro-symbolic knowledge graph construction for zero-shot commonsense question answering. In: Proc. of 35th AAAI Conf. on Artificial Intelligence. San Francisco: AAAI, 2021. 4923  4931. [find](https://scholar.google.com/scholar?q=Bosselut+A%2C+Bras+RL%2C+Choi+YJ%2E+Dynamic+neuro%2Dsymbolic+knowledge+graph+construction+for+zero%2Dshot+commonsense+question+answering%2E+In%3A+Proc%2E+of+35th+AAAI+Conf%2E+on+Artificial+Intelligence%2E+San+Francisco%3A+)
27. Paul D, Frank A. Ranking and selecting multi-hop knowledge paths to better predict human needs. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Washington: ACL, 2019. 3671  3681. [doi: 10.18653/v1/n19-1368] [find](https://scholar.google.com/scholar?q=Paul+D%2C+Frank+A%2E+Ranking+and+selecting+multi%2Dhop+knowledge+paths+to+better+predict+human+needs%2E+In%3A+Proc%2E+of+the+2019+Conf%2E+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguisti)
28. Yang A, Wang Q, Liu J, Liu K, Lyu YJ, Wu H, She QQ, Li SJ. Enhancing pre-trained language representations with rich knowledge for machine reading comprehension. In: Proc. of the 57th Conf. of the Association for Computational Linguistics. Washington: ACL, 2019. 2346  2357. [doi: 10.18653/v1/p19-122… [find](https://scholar.google.com/scholar?q=Yang+A%2C+Wang+Q%2C+Liu+J%2C+Liu+K%2C+Lyu+YJ%2C+Wu+H%2C+She+QQ%2C+Li+SJ%2E+Enhancing+pre%2Dtrained+language+representations+with+rich+knowledge+for+machine+reading+comprehension%2E+In%3A+Proc%2E+of+the+57th+Conf%2E+of+the+Asso)
29. Yih WT, Richardson M, Meek C, Chang MW, Suh J. The value of semantic parse labeling for knowledge base question answering. In: Proc. of the 54th Annual Meeting of the Association for Computational Linguistics. Washington: ACL, 2019. 546  555. [doi: 10.18653/v1/p16-2033] [find](https://scholar.google.com/scholar?q=Yih+WT%2C+Richardson+M%2C+Meek+C%2C+Chang+MW%2C+Suh+J%2E+The+value+of+semantic+parse+labeling+for+knowledge+base+question+answering%2E+In%3A+Proc%2E+of+the+54th+Annual+Meeting+of+the+Association+for+Computational+Lin)
30. Bevilacqua M, Navigli R. Quasi bidirectional encoder representations from transformers for word sense disambiguation. In: Proc. of the Int’l Conf. on Recent Advances in Natural Language Processing. Varna: INCOMA Ltd, 2019. 122  131. [doi: 10.26615/978-954-452-056-4\\_015] [find](https://scholar.google.com/scholar?q=Bevilacqua+M%2C+Navigli+R%2E+Quasi+bidirectional+encoder+representations+from+transformers+for+word+sense+disambiguation%2E+In%3A+Proc%2E+of+the+Int%E2%80%99l+Conf%2E+on+Recent+Advances+in+Natural+Language+Processing%2E+Va)
31. Sun HT, Weiss TB, Cohen WW. PullNet: Open domain question answering with iterative retrieval on knowledge bases and text. In: Proc. of the 2019 Conf. on Empirical Methods in Natural Language Processing. Washington: ACL, 2019. 2380  2390. [doi: 10.18653/v1/D19-1242] [find](https://scholar.google.com/scholar?q=Sun+HT%2C+Weiss+TB%2C+Cohen+WW%2E+PullNet%3A+Open+domain+question+answering+with+iterative+retrieval+on+knowledge+bases+and+text%2E+In%3A+Proc%2E+of+the+2019+Conf%2E+on+Empirical+Methods+in+Natural+Language+Processin)
32. Xiong WH, Yu M, Chang SY, Guo XX, Wang YW. Improving question answering over incomplete kbs with knowledge-aware reader. In: Proc. of the 57th Conf. of the Association for Computational Linguistics. Washington: ACL, 2019. 4258  4264. [doi: 10.18653/v1/p19-1417] [find](https://scholar.google.com/scholar?q=Xiong+WH%2C+Yu+M%2C+Chang+SY%2C+Guo+XX%2C+Wang+YW%2E+Improving+question+answering+over+incomplete+kbs+with+knowledge%2Daware+reader%2E+In%3A+Proc%2E+of+the+57th+Conf%2E+of+the+Association+for+Computational+Linguistics%2E+W)
33. Zhang YH, Qi P, Manning CD. Graph convolution over pruned dependency trees improves relation extraction. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing. Washington: ACL, 2018. 2205  2215. [doi: 10.18653/v1/d18-1244] [find](https://scholar.google.com/scholar?q=Zhang+YH%2C+Qi+P%2C+Manning+CD%2E+Graph+convolution+over+pruned+dependency+trees+improves+relation+extraction%2E+In%3A+Proc%2E+of+the+2018+Conf%2E+on+Empirical+Methods+in+Natural+Language+Processing%2E+Washington%3A+AC) [our card](https://md.research.elamur.ai/free/papers/zhang-2018-graph-convolution-over-pruned-dependency/)
34. Yasunaga M, Liang P. Graph-based, self-supervised program repair from diagnostic feedback. In: Proc. of the 37th Int’l Conf. on Machine Learning. Berlin: PMLR, 2020. 10799  10808. [find](https://scholar.google.com/scholar?q=Yasunaga+M%2C+Liang+P%2E+Graph%2Dbased%2C+self%2Dsupervised+program+repair+from+diagnostic+feedback%2E+In%3A+Proc%2E+of+the+37th+Int%E2%80%99l+Conf%2E+on+Machine+Learning%2E+Berlin%3A+PMLR%2C+2020%2E+10799+%EF%80%AD+10808%2E)
35. Lee J, Lee I, Kang J. Self-attention graph pooling. In: Proc. of the 36th Int’l Conf. on Machine Learning. PMLR, 2019. 3734  3743. [find](https://scholar.google.com/scholar?q=Lee+J%2C+Lee+I%2C+Kang+J%2E+Self%2Dattention+graph+pooling%2E+In%3A+Proc%2E+of+the+36th+Int%E2%80%99l+Conf%2E+on+Machine+Learning%2E+PMLR%2C+2019%2E+3734+%EF%80%AD+3743%2E)
36. Vaswani A, Shazeer N, Parmar N, Uszkoreit J, Jones L, Gomez AN, Kaiser L, Polosukhin I. Attention is all you need. In: Proc. of the Advances in Neural Information Processing Systems 30: Annual Conf. on Neural Information Processing Systems 2017. New York: ACM, 2017. 5998  6008. [find](https://scholar.google.com/scholar?q=Vaswani+A%2C+Shazeer+N%2C+Parmar+N%2C+Uszkoreit+J%2C+Jones+L%2C+Gomez+AN%2C+Kaiser+L%2C+Polosukhin+I%2E+Attention+is+all+you+need%2E+In%3A+Proc%2E+of+the+Advances+in+Neural+Information+Processing+Systems+30%3A+Annual+Conf%2E+o)
37. Velickovic P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y. Graph attention networks. In: Proc. of the 6th Int’l Conf. on Learning Representations. New York: ACM, 2018. 1  12. [find](https://scholar.google.com/scholar?q=Velickovic+P%2C+Cucurull+G%2C+Casanova+A%2C+Romero+A%2C+Lio+P%2C+Bengio+Y%2E+Graph+attention+networks%2E+In%3A+Proc%2E+of+the+6th+Int%E2%80%99l+Conf%2E+on+Learning+Representations%2E+New+York%3A+ACM%2C+2018%2E+1+%EF%80%AD+12%2E)
38. Talmor A, Herzig J, Lourie N, Berant J. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Washington: ACL, 2019. 4149  4158. [doi: 10… [find](https://scholar.google.com/scholar?q=Talmor+A%2C+Herzig+J%2C+Lourie+N%2C+Berant+J%2E+CommonsenseQA%3A+A+question+answering+challenge+targeting+commonsense+knowledge%2E+In%3A+Proc%2E+of+the+2019+Conf%2E+of+the+North+American+Chapter+of+the+Association+for+)
39. Mihaylov T, Clark P, Khot T, Sabharwal A. Can a suit of armor conduct electricity? A new dataset for open book question answering. In: Proc. of the 2018 Conf. on Empirical Methods in Natural Language Processing. Washington: ACL, 2018. 2381  2391. [doi: 10.18653/v1/d18-1260] [find](https://scholar.google.com/scholar?q=Mihaylov+T%2C+Clark+P%2C+Khot+T%2C+Sabharwal+A%2E+Can+a+suit+of+armor+conduct+electricity%3F+A+new+dataset+for+open+book+question+answering%2E+In%3A+Proc%2E+of+the+2018+Conf%2E+on+Empirical+Methods+in+Natural+Language+)
40. Jin D, Pan B, Oufattole N, Weng WH, Fang HY, Szolovits P. What disease does this patient have? A large-scale open domain question answering dataset from medical exams. Applied Sciences, 2021, 11(4): 1  17. [doi: 10.3390/app11146421] [find](https://scholar.google.com/scholar?q=Jin+D%2C+Pan+B%2C+Oufattole+N%2C+Weng+WH%2C+Fang+HY%2C+Szolovits+P%2E+What+disease+does+this+patient+have%3F+A+large%2Dscale+open+domain+question+answering+dataset+from+medical+exams%2E+Applied+Sciences%2C+2021%2C+11%284%29%3A+1) [our card](https://md.research.elamur.ai/free/papers/jin-2020-disease-this-patient-have-large/)
41. Bodenreider O. The unified medical language system (UMLS): Integrating biomedical terminology. Nucleic Acids Research, 2004, 32(4): 267  270. [doi: 10.1093/nar/gkh061] — [doi:10.1093/nar/gkh061](https://doi.org/10.1093/nar/gkh061) [find](https://scholar.google.com/scholar?q=Bodenreider+O%2E+The+unified+medical+language+system+%28UMLS%29%3A+Integrating+biomedical+terminology%2E+Nucleic+Acids+Research%2C+2004%2C+32%284%29%3A+267+%EF%80%AD+270%2E+%5Bdoi%3A+10%2E1093%2Fnar%2Fgkh061%5D+%E2%80%94+%5Bdoi%3A10%2E1093%2Fnar%2Fgkh061%5D%28http)
42. Wishart DS, Feunang YD, Guo AC, Lo EJ, Marcu A, Grant JR, Sajed T, Johnson D, Li C, Sayeeda Z, Assempour N, Iynkkaran I, Liu YF, Maciejewski A, Gale N, Wilson A, Chin L, Cummings R, Le D, Pon A, Knox C, Wilson M. DrugBank 5.0: A major update to the DrugBank database for 2018. Nucleic Acids Research,… — [doi:10.1093/nar/gkx1037](https://doi.org/10.1093/nar/gkx1037) [find](https://scholar.google.com/scholar?q=Wishart+DS%2C+Feunang+YD%2C+Guo+AC%2C+Lo+EJ%2C+Marcu+A%2C+Grant+JR%2C+Sajed+T%2C+Johnson+D%2C+Li+C%2C+Sayeeda+Z%2C+Assempour+N%2C+Iynkkaran+I%2C+Liu+YF%2C+Maciejewski+A%2C+Gale+N%2C+Wilson+A%2C+Chin+L%2C+Cummings+R%2C+Le+D%2C+Pon+A%2C+Knox+)
43. Liu FY, Shareghi E, Meng ZQ, Basaldella M, Collier N. Self-alignment pretraining for biomedical entity representations. In: Proc. of the 2021 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Washington: ACL, 2021. 4228  4238. [doi: 1… [find](https://scholar.google.com/scholar?q=Liu+FY%2C+Shareghi+E%2C+Meng+ZQ%2C+Basaldella+M%2C+Collier+N%2E+Self%2Dalignment+pretraining+for+biomedical+entity+representations%2E+In%3A+Proc%2E+of+the+2021+Conf%2E+of+the+North+American+Chapter+of+the+Association+for)
44. Liu YH, Ott M, Goyal M, Du JF, Joshi M, Chen DQ, Levy O, Lewis M, Zettlemoyer L, Stoyanov V. RoBERTa: A robustly optimized BERT pretraining approach. arXiv:1907.11692, 2019. [find](https://scholar.google.com/scholar?q=Liu+YH%2C+Ott+M%2C+Goyal+M%2C+Du+JF%2C+Joshi+M%2C+Chen+DQ%2C+Levy+O%2C+Lewis+M%2C+Zettlemoyer+L%2C+Stoyanov+V%2E+RoBERTa%3A+A+robustly+optimized+BERT+pretraining+approach%2E+arXiv%3A1907%2E11692%2C+2019%2E)
45. Clark P, Etzioni O, Khashabi D, Khot T, Mishra BD, Richardson K, Sabharwal A, Schoenick C, Tafjord O, Tandon N, Bhakthavatsalam S, Groeneveld D, Guerquin M, Schmitz M. From ‘f’ to ‘a’ on the NY regent science exams: An overview of the aristo project. arXiv:1909.01958, 2019. [find](https://scholar.google.com/scholar?q=Clark+P%2C+Etzioni+O%2C+Khashabi+D%2C+Khot+T%2C+Mishra+BD%2C+Richardson+K%2C+Sabharwal+A%2C+Schoenick+C%2C+Tafjord+O%2C+Tandon+N%2C+Bhakthavatsalam+S%2C+Groeneveld+D%2C+Guerquin+M%2C+Schmitz+M%2E+From+%E2%80%98f%E2%80%99+to+%E2%80%98a%E2%80%99+on+the+NY+regent)
46. Santoro A, Raposo D, Barrett DGT, Malinowski M, Pascanu R, Battaglia PW, Lillicrap T. A simple neural network module for relational reasoning. In: Proc. of the Advances in Neural Information Processing Systems 30: Annual Conf. on Neural Information Processing Systems. Washington: ACL, 2017. 4967  4… [find](https://scholar.google.com/scholar?q=Santoro+A%2C+Raposo+D%2C+Barrett+DGT%2C+Malinowski+M%2C+Pascanu+R%2C+Battaglia+PW%2C+Lillicrap+T%2E+A+simple+neural+network+module+for+relational+reasoning%2E+In%3A+Proc%2E+of+the+Advances+in+Neural+Information+Processin) [our card](https://md.research.elamur.ai/free/papers/santoro-2017-simple-neural-network-module-relational/)
47. Schlichtkrull MS, Kipf TN, Bloem P, Berg RVD, Titov I, Welling M. Modeling relational data with graph convolutional networks. In: Proc. of the 15th Int’l Conf. on Semantic Web. Berlin: Springer, 2018. 593  607. [doi: 10.1007/978-3-319-93417-4\\_38] [find](https://scholar.google.com/scholar?q=Schlichtkrull+MS%2C+Kipf+TN%2C+Bloem+P%2C+Berg+RVD%2C+Titov+I%2C+Welling+M%2E+Modeling+relational+data+with+graph+convolutional+networks%2E+In%3A+Proc%2E+of+the+15th+Int%E2%80%99l+Conf%2E+on+Semantic+Web%2E+Berlin%3A+Springer%2C+2018%2E)
48. Wang XY, Kapanipathi P, Musa R, Yu M, Talamadupula K, Abdelaziz I, Chang M, Fokoue A, Makni B, Mattei N, Witbrock M. Improving natural language inference using external knowledge in the science questions domain. In: Proc. of the 33rd AAAI Conf. on Artificial Intelligence. San Francisco: AAAI, 2019.… — [doi:10.1609/aaai.v33i01.33017208](https://doi.org/10.1609/aaai.v33i01.33017208) [find](https://scholar.google.com/scholar?q=Wang+XY%2C+Kapanipathi+P%2C+Musa+R%2C+Yu+M%2C+Talamadupula+K%2C+Abdelaziz+I%2C+Chang+M%2C+Fokoue+A%2C+Makni+B%2C+Mattei+N%2C+Witbrock+M%2E+Improving+natural+language+inference+using+external+knowledge+in+the+science+questi)
49. Kingma DP, Ba J. Adam: A method for stochastic optimization. In: Proc. of the 3rd Int’l Conf. on Learning Representations. Washington: ACL, 2015. 1  15. [find](https://scholar.google.com/scholar?q=Kingma+DP%2C+Ba+J%2E+Adam%3A+A+method+for+stochastic+optimization%2E+In%3A+Proc%2E+of+the+3rd+Int%E2%80%99l+Conf%2E+on+Learning+Representations%2E+Washington%3A+ACL%2C+2015%2E+1+%EF%80%AD+15%2E)
50. Devlin J, Chang MW, Lee KT, Toutanova K. BERT: Pre-training of deep bidirectional transformers for language understanding. In: Proc. of the 2019 Conf. of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Washington: ACL, 2019. 4171  4186. [doi… [find](https://scholar.google.com/scholar?q=Devlin+J%2C+Chang+MW%2C+Lee+KT%2C+Toutanova+K%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+In%3A+Proc%2E+of+the+2019+Conf%2E+of+the+North+American+Chapter+of+the+Association+)
51. Lee J, Yoon W, Kim SD, Kim D, Kim SY, So CH, Kang J. BioBERT: A pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 2020, 36(4): 1234  1240. [doi: 10.1093/bioinformatics/btz682] 附中文参考文献 : [6] 李涓子 , 侯磊 . 知识图谱研究综述 . 山西大学学报 ( 自然科学版 ), 2017, 40(3): 454  459… [find](https://scholar.google.com/scholar?q=Lee+J%2C+Yoon+W%2C+Kim+SD%2C+Kim+D%2C+Kim+SY%2C+So+CH%2C+Kang+J%2E+BioBERT%3A+A+pre%2Dtrained+biomedical+language+representation+model+for+biomedical+text+mining%2E+Bioinformatics%2C+2020%2C+36%284%29%3A+1234+%EF%80%AD+1240%2E+%5Bdoi%3A+10%2E1093) [our card](https://md.research.elamur.ai/free/papers/lee-2019-biobert-pre-trained-biomedical-language/)