---
title: "Generating Research Data Metadata from Their Accompanying"
description: "Datasets released on software platforms such as GitHub usually come with a README but seldom with structured metadata. 22 references."
authors: ["Kotaro Sekido", "Yu Watanabe", "Koichiro Ito", "Shigeki Matsubara"]
year: 2026
venue: "Proceedings of Natural Scientific Language Processing (NSLP) @ LREC 2026"
source: "https://aclanthology.org/2026.nslp-1.17/"
doi: "10.63317/27oe6uwv2fws"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sekido-2026-generating-research-data-metadata-their/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sekido-2026-generating-research-data-metadata-their/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/31/) / Generating Research Data Metadata from Their Accompanying README Files

# Generating Research Data Metadata from Their Accompanying README Files

**Authors:** Kotaro Sekido, Yu Watanabe, [Koichiro Ito](https://md.research.elamur.ai/free/en/authors/koichiro-ito/) and [Shigeki Matsubara](https://md.research.elamur.ai/free/en/authors/shigeki-matsubara/) · **Year:** 2026 · **Venue:** [Proceedings of Natural Scientific Language Processing (NSLP) @ LREC 2026](https://md.research.elamur.ai/free/en/venues/nslp/) ([Natural Scientific Language Processing (NSLP) @ LREC 2026](https://md.research.elamur.ai/free/en/venues/nslp/2026/)) · **Publisher:** [European Language Resources Association (ELRA)](https://md.research.elamur.ai/free/en/publishers/european-language-resources-association-elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.nslp-1.17/) · [PDF](https://aclanthology.org/2026.nslp-1.17.pdf) · [DOI 10.63317/27oe6uwv2fws](https://doi.org/10.63317/27oe6uwv2fws) · [OpenAlex](https://openalex.org/W7166502967)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Datasets released on software platforms such as GitHub usually come with a README but seldom with structured metadata. This makes them hard to discover through metadata repositories. Sekido et al. ask whether READMEs can supply that missing metadata. They paired dataset records from Papers With Code Datasets (PWCD) with English READMEs fetched via the GitHub API. They then measured how often values for five DataCite-aligned fields appear in the README text: Title, PublicationYear, Creator, Subject and Language. On average, about two-thirds of the values were present, so READMEs look like a usable source.

Next, the authors prompted four LLMs to fill a fixed JSON schema from each README, returning null when a value was absent. Three were open models and one was closed. GPT-5 performed best, and Llama-3.1-8B-Instruct came close behind. Title, year and creators were extracted reliably. Subject and Language were harder, mainly because precision was low. A recurring error was confusing the dataset with a model or another artifact in the same repository. The work suggests a cheap route to better findability of research data hosted on GitHub.

## Key points
- **Analysis data:** 2,464 PWCD metadata–README pairs from repositories created before 2023. Repositories with several READMEs were excluded.
- **Occurrence rates in READMEs:**
  - Title: 85.23%
  - Creator: 82.44%
  - PublicationYear: 71.79%
  - Subject: 54.23%
  - Language: 30.97%
  - Macro average: 64.93%
- **Models and setup:** Llama-3.1-8B-Instruct, Ministral-8B-Instruct-2410 and Qwen3-8B ran on vLLM at temperature 0. GPT-5 was accessed through the OpenAI API. All outputs were constrained with Structured Outputs. The data split was 2,189 train, 275 dev (used for prompt design) and 273 test samples.
- **Macro F1:** GPT-5 80.06, Llama 78.43, Ministral 74.88, Qwen 70.01. Creator F1 reached up to 97.07 (Llama), while Language F1 fell as low as 38.18 (Qwen).
- **Main error and future work:** Models sometimes pulled out names of non-target artifacts. For example, they returned the model "Sonnet" instead of the dataset "WEA". Future work will focus on identifying what type of research data a README describes.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Nagoya University](https://md.research.elamur.ai/free/en/universities/nagoya-university/) — University, Nagoya, Japan · 8 works · 8 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,437 works · [🎯 Decision Sciences](https://md.research.elamur.ai/free/en/shelves/decision-sciences/) · 209 works · [🎯 Information Systems and Management](https://md.research.elamur.ai/free/en/shelves/decision-sciences/information-systems-and-management/) · 41 works

## Cite

Cite as: Kotaro Sekido, Yu Watanabe, Koichiro Ito and Shigeki Matsubara. 2026. Generating Research Data Metadata from Their Accompanying README Files. Proceedings of Natural Scientific Language Processing (NSLP) @ LREC 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{sekido2026generating,
  title = {{Generating Research Data Metadata from Their Accompanying README Files}},
  author = {Kotaro Sekido, Yu Watanabe, Koichiro Ito and Shigeki Matsubara},
  booktitle = {Proceedings of Natural Scientific Language Processing (NSLP) @ LREC 2026},
  year = {2026},
  doi = {10.63317/27oe6uwv2fws},
  url = {https://doi.org/10.63317/27oe6uwv2fws}
}
```

- [← previous Generalizing Trust: Weak-to-Strong Trustworthiness in Language Models](https://md.research.elamur.ai/free/en/papers/sun-2026-generalizing-trust-weak-strong-trustworthiness/)
- [next → Generating Sign Language Poses from HamNoSys and Natural Language Descriptions](https://md.research.elamur.ai/free/en/papers/maximo-2026-generating-sign-language-poses-hamnosys/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/31/)

## References

22 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Zaid Alyafeai, Maged S. Al-shaibani, and Bernard Ghanem. 2025. [MOLE: Metadata Extraction and Validation in Scientific Papers Using LLMs](https://md.research.elamur.ai/free/en/papers/alyafeai-2025-mole-metadata-extraction-validation-scientific/). In Findings of the Association for Computational Linguistics: EMNLP 2025, pages 12236–12264.
2. DataCite Metadata Working Group. 2024. DataCite Metadata Schema Documentation for the Publication and Citation of Research Data and Other Research Outputs. Version 4.6. [find](https://scholar.google.com/scholar?q=DataCite+Metadata+Working+Group%2E+2024%2E+DataCite+Metadata+Schema+Documentation+for+the+Publication+and+Citation+of+Research+Data+and+Other+Research+Outputs%2E+Version+4%2E6%2E)
3. Genet Asefa Gesese, Zongxiong Chen, Oussama Zoubia, and others. 2025. A Survey on Metadata for Machine Learning Models and Datasets: Standards, Practices, and Harmonization Challenges. In Proceedings of 5th International Workshop on Scientific Knowledge: Representation, Discovery, and Assessment (Sc… [find](https://scholar.google.com/scholar?q=Genet+Asefa+Gesese%2C+Zongxiong+Chen%2C+Oussama+Zoubia%2C+and+others%2E+2025%2E+A+Survey+on+Metadata+for+Machine+Learning+Models+and+Datasets%3A+Standards%2C+Practices%2C+and+Harmonization+Challenges%2E+In+Proceedings+)
4. Aaron Grattafiori, Abhimanyu Dubey, Abhinav Jauhri, and others. 2024. [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/). arXiv preprint arXiv:2407.21783.
5. Laura Koesten, Pavlos Vougiouklis, Elena Simperl, and Paul Groth. 2020. Dataset Reuse: Toward Translating Principles to Practice. Patterns, 1(8). [find](https://scholar.google.com/scholar?q=Laura+Koesten%2C+Pavlos+Vougiouklis%2C+Elena+Simperl%2C+and+Paul+Groth%2E+2020%2E+Dataset+Reuse%3A+Toward+Translating+Principles+to+Practice%2E+Patterns%2C+1%288%29%2E)
6. Prince Kumar, Srikanth Tamilselvam, and Dinesh Garg. 2024. Read between the Lines - Functionality Extraction from READMEs. In Findings of the Association for Computational Linguistics: NAACL 2024, pages 3977–3990. [find](https://scholar.google.com/scholar?q=Prince+Kumar%2C+Srikanth+Tamilselvam%2C+and+Dinesh+Garg%2E+2024%2E+Read+between+the+Lines+%2D+Functionality+Extraction+from+READMEs%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+NAACL+2024%2C+pag)
7. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, and others. 2023. [Efficient Memory Management for Large Language Model Serving with PagedAttention](https://md.research.elamur.ai/free/en/papers/kwon-2023-efficient-memory-management-large-language/). In Proceedings of the 29th Symposium on Operating Systems Principles (SOSP 2023), page 611–626.
8. Mistral AI Team. 2024. Un Ministral, des Ministraux. [find](https://scholar.google.com/scholar?q=Mistral+AI+Team%2E+2024%2E+Un+Ministral%2C+des+Ministraux%2E)
9. Open AI. 2025. Introducing GPT-5. [find](https://scholar.google.com/scholar?q=Open+AI%2E+2025%2E+Introducing+GPT%2D5%2E)
10. G7 OSWG. 2023. Annex 1: G7 Open Science Working Group (OSWG). [find](https://scholar.google.com/scholar?q=G7+OSWG%2E+2023%2E+Annex+1%3A+G7+Open+Science+Working+Group+%28OSWG%29%2E)
11. Anthony Cintron Roman, Kevin Xu, Arfon Smith, and others. 2023. Open Data on GitHub: Unlocking the Potential of AI. arXiv preprint arXiv:2306.06191. — [arXiv:2306.06191](https://arxiv.org/abs/2306.06191) [find](https://scholar.google.com/scholar?q=Anthony+Cintron+Roman%2C+Kevin+Xu%2C+Arfon+Smith%2C+and+others%2E+2023%2E+Open+Data+on+GitHub%3A+Unlocking+the+Potential+of+AI%2E+arXiv+preprint+arXiv%3A2306%2E06191%2E+%E2%80%94+%5BarXiv%3A2306%2E06191%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2306%2E061)
12. David Schindler, Felix Bensmann, Stefan Dietze, and Frank Krüger. 2021. SoMeSci- A 5 Star Open Data Gold Standard Knowledge Graph of Software Mentions in Scientific Articles. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management (CIKM 2021), pages 4574–4583. [find](https://scholar.google.com/scholar?q=David+Schindler%2C+Felix+Bensmann%2C+Stefan+Dietze%2C+and+Frank+Kr%C3%BCger%2E+2021%2E+SoMeSci%2D+A+5+Star+Open+Data+Gold+Standard+Knowledge+Graph+of+Software+Mentions+in+Scientific+Articles%2E+In+Proceedings+of+the+30t)
13. Yuxuan Shu and Vasileios Lampos. 2025. Sonnet: Spectral Operator Neural Network for Multivariable Time Series Forecasting. arXiv preprint arXiv:2505.15312. — [arXiv:2505.15312](https://arxiv.org/abs/2505.15312) [find](https://scholar.google.com/scholar?q=Yuxuan+Shu+and+Vasileios+Lampos%2E+2025%2E+Sonnet%3A+Spectral+Operator+Neural+Network+for+Multivariable+Time+Series+Forecasting%2E+arXiv+preprint+arXiv%3A2505%2E15312%2E+%E2%80%94+%5BarXiv%3A2505%2E15312%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2)
14. Petros Stavropoulos, Ioannis Lyris, Natalia Manola, Ioanna Grypari, and Haris Papageorgiou. 2023. Empowering Knowledge Discovery from Scientific Literature: A Novel Approach to Research Artifact Analysis. In Proceedings of the 3rd Workshop for Natural Language Processing Open Source Software (NLP-OS… [find](https://scholar.google.com/scholar?q=Petros+Stavropoulos%2C+Ioannis+Lyris%2C+Natalia+Manola%2C+Ioanna+Grypari%2C+and+Haris+Papageorgiou%2E+2023%2E+Empowering+Knowledge+Discovery+from+Scientific+Literature%3A+A+Novel+Approach+to+Research+Artifact+Analy)
15. Aarne Talman, Antti Suni, Hande Celikkanat, and others. 2019. Predicting Prosodic Prominence from Text with Pre-trained Contextualized Word Representations. In Proceedings of the 22nd Nordic Conference on Computational Linguistics (NoDaLiDa 2019), pages 281–290. [find](https://scholar.google.com/scholar?q=Aarne+Talman%2C+Antti+Suni%2C+Hande+Celikkanat%2C+and+others%2E+2019%2E+Predicting+Prosodic+Prominence+from+Text+with+Pre%2Dtrained+Contextualized+Word+Representations%2E+In+Proceedings+of+the+22nd+Nordic+Conferenc)
16. UNESCO. 2021. UNESCO Recommendation on Open Science. [find](https://scholar.google.com/scholar?q=UNESCO%2E+2021%2E+UNESCO+Recommendation+on+Open+Science%2E)
17. Carlos Utrilla Guerrero, Oscar Corcho, and Daniel Garijo. 2024. Automated Extraction of Research Software Installation Instructions from README Files: An Initial Analysis. In Proceedings of the Natural Scientific Language Processing and Research Knowledge Graphs (NSLP 2024), pages 114–133. [find](https://scholar.google.com/scholar?q=Carlos+Utrilla+Guerrero%2C+Oscar+Corcho%2C+and+Daniel+Garijo%2E+2024%2E+Automated+Extraction+of+Research+Software+Installation+Instructions+from+README+Files%3A+An+Initial+Analysis%2E+In+Proceedings+of+the+Natura)
18. Yu Watanabe, Koichiro Ito, and Shigeki Matsubara. 2024. Capabilities and Challenges of LLMs in Metadata Extraction from Scholarly Papers. In Proceedings of the 26th International Conference on Asia-Pacific Digital Libraries (ICADL 2024), pages 280–287. [find](https://scholar.google.com/scholar?q=Yu+Watanabe%2C+Koichiro+Ito%2C+and+Shigeki+Matsubara%2E+2024%2E+Capabilities+and+Challenges+of+LLMs+in+Metadata+Extraction+from+Scholarly+Papers%2E+In+Proceedings+of+the+26th+International+Conference+on+Asia%2DPa)
19. Yu Watanabe, Koichiro Ito, and Shigeki Matsubara. 2025. Metadata Generation for Research Data from URL Citation Contexts in Scholarly Papers: Task Definition and Dataset Construction. In Proceedings of the Third Workshop for Artificial Intelligence for Scientific Publications (WASP 2025), pages 72–7… [find](https://scholar.google.com/scholar?q=Yu+Watanabe%2C+Koichiro+Ito%2C+and+Shigeki+Matsubara%2E+2025%2E+Metadata+Generation+for+Research+Data+from+URL+Citation+Contexts+in+Scholarly+Papers%3A+Task+Definition+and+Dataset+Construction%2E+In+Proceedings+o)
20. Mark Wilkinson, Dumontier Michel, IJsbrand Aalbersberg, and others. 2016. The FAIR Guiding Principles for Scientific Data Management and Stewardship. Sci Data, 3. [find](https://scholar.google.com/scholar?q=Mark+Wilkinson%2C+Dumontier+Michel%2C+IJsbrand+Aalbersberg%2C+and+others%2E+2016%2E+The+FAIR+Guiding+Principles+for+Scientific+Data+Management+and+Stewardship%2E+Sci+Data%2C+3%2E)
21. An Yang, Anfeng Li, Baosong Yang, and others. 2025. [Qwen3 Technical Report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/). arXiv preprint arXiv:2505.09388. 8.
22. Language Resource References Papers With Code. 2019–25. Papers With Code Datasets. PID [https://huggingface.co/pwc-archive/datasets](https://huggingface.co/pwc-archive/datasets). Originally obtained via the Papers With Code GitHub repository. The dataset is not currently directly distributed from the repository (Last Accessed: 2025-08-08). [↗](https://huggingface.co/pwc-archive/datasets)

*Updated 1 November 2026*