---
title: "Identifying Implicit Research Data References in Paper Citations"
description: "Citation counts could reward researchers who release data. 29 references. Among them: 1."
authors: ["Koshi Motegi", "Koichiro Ito", "Shigeki Matsubara"]
year: 2026
venue: "Proceedings of Natural Scientific Language Processing (NSLP) @ LREC 2026"
source: "https://aclanthology.org/2026.nslp-1.18/"
doi: "10.63317/2g9fq97f2h2j"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/motegi-2026-identifying-implicit-research-data-references/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/motegi-2026-identifying-implicit-research-data-references/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/13/) / Identifying Implicit Research Data References in Paper Citations

# Identifying Implicit Research Data References in Paper Citations

**Authors:** Koshi Motegi, [Koichiro Ito](https://md.research.elamur.ai/free/en/authors/koichiro-ito/) and [Shigeki Matsubara](https://md.research.elamur.ai/free/en/authors/shigeki-matsubara/) · **Year:** 2026 · **Venue:** [Proceedings of Natural Scientific Language Processing (NSLP) @ LREC 2026](https://md.research.elamur.ai/free/en/venues/nslp/) ([Natural Scientific Language Processing (NSLP) @ LREC 2026](https://md.research.elamur.ai/free/en/venues/nslp/2026/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.nslp-1.18/) · [PDF](https://aclanthology.org/2026.nslp-1.18.pdf) · [DOI 10.63317/2g9fq97f2h2j](https://doi.org/10.63317/2g9fq97f2h2j) · [OpenAlex](https://openalex.org/W7166528822)

**Citations:** 1 citations (OpenAlex, 2026-10-14)

## Summary
Citation counts could reward researchers who release data. However, datasets, tools and models are often credited only indirectly: authors cite the paper that introduced a resource, not the resource itself. These citations are usually counted as ordinary paper citations. Formal reference-list entries and URL mentions of data are already tracked by systems such as Make Data Count, but these implicit "paper citations" have received little attention.

The authors frame the problem as binary classification. Given a citation tag, its surrounding paragraph and, optionally, the cited paper's bibliographic record, a model decides whether the citation points to tangible research data (datasets, software, trained models) rather than to methods, architectures or concepts. They annotated 783 citations from 10 machine-learning papers in GSAP-NER and ran a zero-shot preliminary experiment in which open and proprietary LLMs answer Yes or No.

The supplied excerpt omits the score tables, so the comparative results are not reported here. The study is an early step toward data citation metrics that work outside the medical domain.

## Key points
- Three ways of citing data are distinguished: formal citation, URL citation, and implicit paper citation. Prior work on the third type (Callahan et al., 2018) depended on MeSH terms and did not check why each paper was cited.
- Data comes from GSAP-NER (Otto et al., 2023), which covers 100 ML papers and 10 entity types, including citation tags. The authors used 10 of these papers, giving 783 citations, of which 191 (24.4%) refer to research data.
- The models tested were Llama3.1 8B instruct, Ministral-3-{3B, 8B, 14B}-Instruct-2512, the GPT-4.1 series (nano/mini/full), the GPT-5 series (nano/mini/full), and GPT-5.2.
- Two input settings were compared: context plus citation tag ("Con + Cite"), and the same with bibliographic information added ("Con + Cite + Bib").
- The prompt asks for a single word, Yes or No. It instructs the model to answer "No" when a work is cited only for its method or background, or when the citation intent is unclear.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Nagoya University](https://md.research.elamur.ai/free/en/universities/nagoya-university/) — University, Nagoya, Japan · 8 works · 8 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works · [🎯 Decision Sciences](https://md.research.elamur.ai/free/en/shelves/decision-sciences/) · 209 works · [🎯 Statistics, Probability and Uncertainty](https://md.research.elamur.ai/free/en/shelves/decision-sciences/statistics-probability-and-uncertainty/) · 61 works

## Cite

Cite as: Koshi Motegi, Koichiro Ito and Shigeki Matsubara. 2026. Identifying Implicit Research Data References in Paper Citations. Proceedings of Natural Scientific Language Processing (NSLP) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{motegi2026identifying,
  title = {{Identifying Implicit Research Data References in Paper Citations}},
  author = {Koshi Motegi, Koichiro Ito and Shigeki Matsubara},
  booktitle = {Proceedings of Natural Scientific Language Processing (NSLP) @ LREC 2026},
  year = {2026},
  doi = {10.63317/2g9fq97f2h2j},
  url = {https://doi.org/10.63317/2g9fq97f2h2j}
}
```

- [← previous Identifying Bias in Machine-generated Text Detection](https://md.research.elamur.ai/free/en/papers/stowe-2026-identifying-bias-machine-generated-text/)
- [next → IF-CRITIC: Towards a Fine-Grained LLM Critic for Instruction-Following Evaluation](https://md.research.elamur.ai/free/en/papers/wen-2026-if-critic-fine-grained-llm/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/13/)

## References

29 references, 5 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Helene N. Andreassen, Andrea L. Berez-Kroeker, Lauren Collister, Philipp Conzett, Christopher Cox, Koenraad De Smedt, Bradley McDonnell, and Research Data Alliance Linguistic Data Interest Group. 2019. Tromsø recommendations for citation of research data in linguistics. [find](https://scholar.google.com/scholar?q=Helene+N%2E+Andreassen%2C+Andrea+L%2E+Berez%2DKroeker%2C+Lauren+Collister%2C+Philipp+Conzett%2C+Christopher+Cox%2C+Koenraad+De+Smedt%2C+Bradley+McDonnell%2C+and+Research+Data+Alliance+Linguistic+Data+Interest+Group%2E+2019)
2. Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. SciB-ERT: A pretrained language model for scientific text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pages 3615–3620, Hong Kong, Chin… [find](https://scholar.google.com/scholar?q=Iz+Beltagy%2C+Kyle+Lo%2C+and+Arman+Cohan%2E+2019%2E+SciB%2DERT%3A+A+pretrained+language+model+for+scientific+text%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the)
3. Alison Callahan, Rainer Winnenburg, and Nigam H. Shah. 2018. U-index, a dataset and an impact metric for informatics tools and databases. Scientific Data, 5(1):180043. [find](https://scholar.google.com/scholar?q=Alison+Callahan%2C+Rainer+Winnenburg%2C+and+Nigam+H%2E+Shah%2E+2018%2E+U%2Dindex%2C+a+dataset+and+an+impact+metric+for+informatics+tools+and+databases%2E+Scientific+Data%2C+5%281%29%3A180043%2E)
4. Clarivate. 2023. Recommended practices to promote scholarly data citation and tracking: The G7. 2023. Annex 1: G7 open science working group (OSWG). (Last Accessed: 2025-12-22). [find](https://scholar.google.com/scholar?q=Clarivate%2E+2023%2E+Recommended+practices+to+promote+scholarly+data+citation+and+tracking%3A+The+G7%2E+2023%2E+Annex+1%3A+G7+open+science+working+group+%28OSWG%29%2E+%28Last+Accessed%3A+2025%2D12%2D22%29%2E)
5. FORCE11 Data Citation Synthesis Group. 2014. Joint declaration of data citation principles. [find](https://scholar.google.com/scholar?q=FORCE11+Data+Citation+Synthesis+Group%2E+2014%2E+Joint+declaration+of+data+citation+principles%2E)
6. Amelia S. C. Hood and William J. Sutherland. 2021. The data-index: An author-level metric that values impactful data and incentivizes data sharing. Ecology and Evolution, 11(21):14344– 14350. [find](https://scholar.google.com/scholar?q=Amelia+S%2E+C%2E+Hood+and+William+J%2E+Sutherland%2E+2021%2E+The+data%2Dindex%3A+An+author%2Dlevel+metric+that+values+impactful+data+and+incentivizes+data+sharing%2E+Ecology+and+Evolution%2C+11%2821%29%3A14344%E2%80%93+14350%2E)
7. Koichiro Ito, Masaki Murata, Tomohiro Ohno, and Shigeki Matsubara. 2022. Construction of responsive utterance corpus for attentive listening response production. In Proceedings of the 13th Language Resources and Evaluation Conference, pages 7244–7252, Marseille, France. European Language Resources A… [find](https://scholar.google.com/scholar?q=Koichiro+Ito%2C+Masaki+Murata%2C+Tomohiro+Ohno%2C+and+Shigeki+Matsubara%2E+2022%2E+Construction+of+responsive+utterance+corpus+for+attentive+listening+response+production%2E+In+Proceedings+of+the+13th+Language+Re)
8. Lasse M. Jantsch, Dong-Jae Koh, Seonghwan Yoon, Jisu Lee, Anne Lauscher, and Young-Kyoon Suh. 2025. FineCite: A novel approach for fine-grained citation context analysis. In Findings of the Association for Computational Linguistics, pages 24525–24542, Vienna, Austria. Association for Computational L… [find](https://scholar.google.com/scholar?q=Lasse+M%2E+Jantsch%2C+Dong%2DJae+Koh%2C+Seonghwan+Yoon%2C+Jisu+Lee%2C+Anne+Lauscher%2C+and+Young%2DKyoon+Suh%2E+2025%2E+FineCite%3A+A+novel+approach+for+fine%2Dgrained+citation+context+analysis%2E+In+Findings+of+the+Associatio)
9. Paris Koloveas, Serafeim Chatzopoulos, Thanasis Vergoulis, and Christos Tryfonopoulos. 2025. Can LLMs predict citation intent? An experimental analysis of in-context learning and fine-tuning on open LLMs. In Linking Theory and Practice of Digital Libraries: 29th International Conference on Theory an… [find](https://scholar.google.com/scholar?q=Paris+Koloveas%2C+Serafeim+Chatzopoulos%2C+Thanasis+Vergoulis%2C+and+Christos+Tryfonopoulos%2E+2025%2E+Can+LLMs+predict+citation+intent%3F+An+experimental+analysis+of+in%2Dcontext+learning+and+fine%2Dtuning+on+open+L)
10. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph Gonzalez, Hao Zhang, and Ion Stoica. 2023. [Efficient memory management for large language model serving with pagedattention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, pages 6…
11. Lorenzo Paolini, Sahar Vahdati, Angelo Di Iorio, Robert Wardenga, Ivan Heibi, and Silvio Peroni. 2025. CiteFusion: An ensemble framework for citation intent classification harnessing dual-model binary couples and shap analyses. Scientometrics, 130(11):5911–5981. [find](https://scholar.google.com/scholar?q=Lorenzo+Paolini%2C+Sahar+Vahdati%2C+Angelo+Di+Iorio%2C+Robert+Wardenga%2C+Ivan+Heibi%2C+and+Silvio+Peroni%2E+2025%2E+CiteFusion%3A+An+ensemble+framework+for+citation+intent+classification+harnessing+dual%2Dmodel+binary)
12. Yasset Perez-Riverol, Andrey Zorin, Gaurhari Dass, Manh-Tu Vu, Pan Xu, Mihai Glont, Juan Antonio Vizcaíno, Andrew F. Jarnuczak, Robert Petryszak, Peipei Ping, and Henning Hermjakob. 2019. Quantifying the impact of public omics data. 10(1):3512. 8. [find](https://scholar.google.com/scholar?q=Yasset+Perez%2DRiverol%2C+Andrey+Zorin%2C+Gaurhari+Dass%2C+Manh%2DTu+Vu%2C+Pan+Xu%2C+Mihai+Glont%2C+Juan+Antonio+Vizca%C3%ADno%2C+Andrew+F%2E+Jarnuczak%2C+Robert+Petryszak%2C+Peipei+Ping%2C+and+Henning+Hermjakob%2E+2019%2E+Quantifying+)
13. Nature Communications, Summer Rosonovski, Maria Levchenko, Rajat Bhatnagar, Umamageswari Chandrasekaran, Lynne Faulk, Islam Hassan, Matt Jeffryes, Syed Irtaza Mubashar, Maaly Nassar, Madhumi-ethaa Jayaprabha Palanisamy, Michael Parkin, Jagadeeswararao Poluru, Frances Rogers, Shyamasree Saha, Mohamed… [find](https://scholar.google.com/scholar?q=Nature+Communications%2C+Summer+Rosonovski%2C+Maria+Levchenko%2C+Rajat+Bhatnagar%2C+Umamageswari+Chandrasekaran%2C+Lynne+Faulk%2C+Islam+Hassan%2C+Matt+Jeffryes%2C+Syed+Irtaza+Mubashar%2C+Maaly+Nassar%2C+Madhumi%2Dethaa+Jay)
14. Zeren Shui, Petros Karypis, Daniel S. Karls, Mingjian Wen, Saurav Manchanda, Ellad B. Tadmor, and George Karypis. 2024. Fine-tuning language models on multiple datasets for citation intention classification. pages 16718– 16732, Miami, Florida, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zeren+Shui%2C+Petros+Karypis%2C+Daniel+S%2E+Karls%2C+Mingjian+Wen%2C+Saurav+Manchanda%2C+Ellad+B%2E+Tadmor%2C+and+George+Karypis%2E+2024%2E+Fine%2Dtuning+language+models+on+multiple+datasets+for+citation+intention+classifi)
15. Masaya Tsunokake and Shigeki Matsubara. 2022. Classification of URL citations in scholarly papers for promoting utilization of research artifacts. In Proceedings of the 1st Workshop on Information Extraction from Scientific Publications, pages 8–19, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Masaya+Tsunokake+and+Shigeki+Matsubara%2E+2022%2E+Classification+of+URL+citations+in+scholarly+papers+for+promoting+utilization+of+research+artifacts%2E+In+Proceedings+of+the+1st+Workshop+on+Information+Ext)
16. UNESCO. 2021. UNESCO recommendation on open science. [find](https://scholar.google.com/scholar?q=UNESCO%2E+2021%2E+UNESCO+recommendation+on+open+science%2E)
17. Kazuhiro Wada, Masaya Tsunokake, and Shigeki Matsubara. 2025. Classification of URL citations on scholarly papers using intermediate task training. IEICE Transactions on Information and Systems, E108-D(10):1183–1193. [find](https://scholar.google.com/scholar?q=Kazuhiro+Wada%2C+Masaya+Tsunokake%2C+and+Shigeki+Matsubara%2E+2025%2E+Classification+of+URL+citations+on+scholarly+papers+using+intermediate+task+training%2E+IEICE+Transactions+on+Information+and+Systems%2C+E108%2D)
18. Yasunori Yamamoto and Toshihisa Takagi. 2007. OReFiL: An online resource finder for life sciences. BMC Bioinformatics, 8:287. [find](https://scholar.google.com/scholar?q=Yasunori+Yamamoto+and+Toshihisa+Takagi%2E+2007%2E+OReFiL%3A+An+online+resource+finder+for+life+sciences%2E+BMC+Bioinformatics%2C+8%3A287%2E)
19. He Zhao, Zhunchen Luo, Chong Feng, Anqing Zheng, and Xiaopeng Liu. 2019. A context-based framework for modeling the role and function of on-line resource citations in scientific literature. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th Internat… [find](https://scholar.google.com/scholar?q=He+Zhao%2C+Zhunchen+Luo%2C+Chong+Feng%2C+Anqing+Zheng%2C+and+Xiaopeng+Liu%2E+2019%2E+A+context%2Dbased+framework+for+modeling+the+role+and+function+of+on%2Dline+resource+citations+in+scientific+literature%2E+In+Proceed)
20. Language Resource References Amjad Abu-Jbara, Jefferson Ezra, and Dragomir Radev. 2013. Purpose and polarity of citation: Towards NLP-based bibliometrics. In Proceedings of the 2013 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies… [find](https://scholar.google.com/scholar?q=Language+Resource+References+Amjad+Abu%2DJbara%2C+Jefferson+Ezra%2C+and+Dragomir+Radev%2E+2013%2E+Purpose+and+polarity+of+citation%3A+Towards+NLP%2Dbased+bibliometrics%2E+In+Proceedings+of+the+2013+Conference+of+the+)
21. Ministral3. (Last Accessed: Arman Cohan, Waleed Ammar, Madeleine van Zuylen, and Field Cady. 2019. [Structural scaffolds for citation intent classification in scientific publications](https://md.research.elamur.ai/free/en/papers/cohan-2019-structural-scaffolds-citation-intent-classification/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:…
22. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, et al. 2024. [The Llama 3 herd of models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/).
23. David Jurgens, Srijan Kumar, Raine Hoover, Dan McFarland, and Dan Jurafsky. 2018. [Measuring the evolution of a scientific field through citation frames](https://md.research.elamur.ai/free/en/papers/jurgens-2018-measuring-evolution-scientific-field-through/). Transactions of the Association for Computational Linguistics, 6:391–406.
24. Suchetha Nambanoor Kunnath, Valentin Stauber, Ronin Wu, David Pride, Viktor Botev, and Petr Knoth. 2022. ACT2: A multi-disciplinary semi-structured dataset for importance and purpose classification of citations. In Proceedings of the 13th Language Resources and Evaluation Conference, pages 3398–3406… [find](https://scholar.google.com/scholar?q=Suchetha+Nambanoor+Kunnath%2C+Valentin+Stauber%2C+Ronin+Wu%2C+David+Pride%2C+Viktor+Botev%2C+and+Petr+Knoth%2E+2022%2E+ACT2%3A+A+multi%2Ddisciplinary+semi%2Dstructured+dataset+for+importance+and+purpose+classification+of)
25. OpenAI. 2025a. Introducing GPT-4.1 in the api. (Last Accessed: 2025-12-14). [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025a%2E+Introducing+GPT%2D4%2E1+in+the+api%2E+%28Last+Accessed%3A+2025%2D12%2D14%29%2E)
26. OpenAI. 2025b. Introducing GPT-5. cessed: 2025-12-22). [find](https://scholar.google.com/scholar?q=OpenAI%2E+2025b%2E+Introducing+GPT%2D5%2E+cessed%3A+2025%2D12%2D22%29%2E)
27. (Last Ac-OpenAI. 2025c. Introducing GPT-5.2. (Last Accessed: 2025-12-22). [find](https://scholar.google.com/scholar?q=%28Last+Ac%2DOpenAI%2E+2025c%2E+Introducing+GPT%2D5%2E2%2E+%28Last+Accessed%3A+2025%2D12%2D22%29%2E)
28. Wolfgang Otto, Matthäus Zloch, Lu Gan, Saurav Karmakar, and Stefan Dietze. 2023. GSAP-NER: A novel task, corpus, and baseline for scholarly entity extraction focused on machine learning models and datasets. In Findings of the Association for Computational Linguistics, pages 8166–8176, Singapore. Ass… [find](https://scholar.google.com/scholar?q=Wolfgang+Otto%2C+Matth%C3%A4us+Zloch%2C+Lu+Gan%2C+Saurav+Karmakar%2C+and+Stefan+Dietze%2E+2023%2E+GSAP%2DNER%3A+A+novel+task%2C+corpus%2C+and+baseline+for+scholarly+entity+extraction+focused+on+machine+learning+models+and+dat)
29. Simone Teufel, Advaith Siddharthan, and Dan Tidhar. 2006. [Automatic classification of citation function](https://md.research.elamur.ai/free/en/papers/teufel-2006-automatic-classification-citation-function/). In Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, pages 103–110, Sydney, Australia. Association for Computational Linguistics. *Figure 6: User prompt (Con…

*Updated 1 November 2026*