---
title: "DARWIN Series — paper by Xie et al. (2023)"
description: "To create literature-based instructions cheaply, they introduce a Scientific Instruction Generation… 48 references, cited by 22 papers here."
authors: ["Tong Xie", "Yuwei Wan", "Wei Huang", "Zhenyu Yin", "Yixuan Liu", "Shaozhou Wang", "Qingyuan Linghu", "Chunyu Kit", "Clara Grazian", "Wenjie Zhang", "Imran Razzak", "Bram Hoex"]
year: 2023
venue: "arXiv:2308.13565"
source: "https://arxiv.org/abs/2308.13565"
doi: "10.48550/arxiv.2308.13565"
arxiv: "2308.13565"
licence: "CC BY-SA 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by-sa/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xie-2023-darwin-series-domain-specific-large/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xie-2023-darwin-series-domain-specific-large/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/) / DARWIN Series: Domain Specific Large Language Models for Natural Science

# DARWIN Series: Domain Specific Large Language Models for Natural Science

**Authors:** Tong Xie, Yuwei Wan, [Wei Huang](https://md.research.elamur.ai/free/en/authors/wei-huang/), Zhenyu Yin, Yixuan Liu, Shaozhou Wang, Qingyuan Linghu, [Chunyu Kit](https://md.research.elamur.ai/free/en/authors/chunyu-kit/), Clara Grazian, [Wenjie Zhang](https://md.research.elamur.ai/free/en/authors/wenjie-zhang/), [Imran Razzak](https://md.research.elamur.ai/free/en/authors/imran-razzak/) and Bram Hoex · **Year:** 2023 · **Venue:** arXiv:2308.13565 · **Licence:** [CC BY-SA 4.0 (arXiv)](http://creativecommons.org/licenses/by-sa/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2308.13565) · [PDF](https://arxiv.org/pdf/2308.13565) · [DOI 10.48550/arxiv.2308.13565](https://doi.org/10.48550/arxiv.2308.13565) · [arXiv 2308.13565](https://arxiv.org/abs/2308.13565) · [OpenAlex](https://openalex.org/W4386269388)

**Citations:** 33 citations (OpenAlex, 2026-10-14) · [22](https://md.research.elamur.ai/free/en/papers/xie-2023-darwin-series-domain-specific-large/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary

General-purpose instruction-tuned LLMs perform poorly on natural science. Fine-tuned GPT models are closed and must be adapted on OpenAI's paid servers. Open alternatives are usually tuned on instructions written by GPT-3.5/GPT-4, which can reduce factual accuracy. The authors address this with DARWIN, a family of open models built on LLaMA-7B for physics, chemistry and materials science.

The training data combine three sources:
- the SciQ exam set,
- question–answer pairs drawn from scientific literature,
- public FAIR datasets of material and molecular properties.

To create literature-based instructions cheaply, they introduce a Scientific Instruction Generation (SIG) model. It turns paper text into instructions without manual extraction or knowledge graphs.

Training happens in stages. A knowledge-oriented model (DARWIN-BASE) comes first. A material and device prediction model (DARWIN-MDP), tuned on many tasks jointly, follows. DARWIN-BASE reaches 96.9% on SciQ. DARWIN-MDP wins most regression and classification benchmarks and is roughly on par with specialised ML tools.

The work shows that a compact open model grounded in curated science data can compete with closed or task-specific systems while keeping costs low.

## Key points

- **Model and data:** the base model is LLaMA-7B, tuned on more than 60,000 instruction data points. Code is released at github.com/MasterAI-EAM/Darwin.
- **Literature corpus:**
  - About 6 million English full-text papers from Web of Science in materials-science-related fields, from publishers such as Elsevier, Springer Nature, ACS and RSC.
  - Highly cited papers were fetched with SciCrawler. Papers with overly similar topics were filtered out using cosine similarity of MatBert embeddings.
  - Long inputs are split with "To be continued" tokens to work around LLaMA's input length limit.
- **SciQ (13,679 questions):** DARWIN-BASE scores 96.9%. For comparison, Pythia 6.9B scores 95.2% five-shot, and zero-shot scores are 89.2% for BLOOM-7B and 90.1% for OPT-6.7B.
- **DARWIN-MDP results:**
  - F1 of 97.2 on Pei (gptchem: 79.2) and 93.3 on waterStability.
  - MAE of 0.54 on OPV.
  - MODNet still does better on steels (MAE 88 vs 125.97) and matbench_glass (F1 97.5 vs 88.0).
- **Training findings:** joint multi-task tuning (LLaMA-all) beats per-task tuning (LLaMA-single), and an earlier science-knowledge stage adds further gains.
- **Stated limitations:** few QA pairs, use of several separate models instead of one, and an evaluation limited to multiple-choice questions.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [The University of Melbourne](https://md.research.elamur.ai/free/en/universities/the-university-of-melbourne/) — University, Melbourne, Australia · 102 works · 22,618 citations
- [The University of Sydney](https://md.research.elamur.ai/free/en/universities/the-university-of-sydney/) — University, Sydney, Australia · 49 works · 68,350 citations
- [City University of Hong Kong](https://md.research.elamur.ai/free/en/universities/city-university-of-hong-kong/) — University, Hong Kong, China · 59 works · 3,480 citations
- [UNSW Sydney](https://md.research.elamur.ai/free/en/universities/unsw-sydney/) — University, Sydney, Australia · 36 works · 1,509 citations

Also on the byline: [Cooperative Trials Group for Neuro-Oncology](https://ror.org/004gchq58).

📚 Shelves (7): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works · [🧱 Materials Science](https://md.research.elamur.ai/free/en/shelves/materials-science/) · 31 works · [🧱 Materials Chemistry](https://md.research.elamur.ai/free/en/shelves/materials-science/materials-chemistry/) · 31 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Tong Xie, Yuwei Wan, Wei Huang, Zhenyu Yin, Yixuan Liu, Shaozhou Wang, Qingyuan Linghu, Chunyu Kit, Clara Grazian, Wenjie Zhang, Imran Razzak and Bram Hoex. 2023. DARWIN Series: Domain Specific Large Language Models for Natural Science. arXiv:2308.13565.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{xie2023darwin,
  title = {{DARWIN Series: Domain Specific Large Language Models for Natural Science}},
  author = {Tong Xie, Yuwei Wan, Wei Huang, Zhenyu Yin, Yixuan Liu, Shaozhou Wang, Qingyuan Linghu, Chunyu Kit, Clara Grazian, Wenjie Zhang, Imran Razzak and Bram Hoex},
  year = {2023},
  doi = {10.48550/arxiv.2308.13565},
  eprint = {2308.13565},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2308.13565}
}
```

- [← previous Beyond Chain-of-Thought, Effective Graph-of-Thought Reasoning in Language Models](https://md.research.elamur.ai/free/en/papers/yao-2023-chain-thought-effective-graph-thought/)
- [next → DERA: Enhancing Large Language Model Completions with Dialog-Enabled Resolving Agents](https://md.research.elamur.ai/free/en/papers/nair-2023-dera-enhancing-large-language-model/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/10/)

## References

48 references, 17 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. L Weston, V Tshitoyan, J Dagdelen, O Kononova, A Trewartha, K A Persson, G Ceder, and A Jain. Named Entity Recognition and Normalization Applied to Large-Scale Information Extraction from the Materials Science Literature. Journal of Chemical Information and Modeling, 59(9):3692–3702, sep 2019. [find](https://scholar.google.com/scholar?q=L+Weston%2C+V+Tshitoyan%2C+J+Dagdelen%2C+O+Kononova%2C+A+Trewartha%2C+K+A+Persson%2C+G+Ceder%2C+and+A+Jain%2E+Named+Entity+Recognition+and+Normalization+Applied+to+Large%2DScale+Information+Extraction+from+the+Material)
2. Amalie Trewartha, Nicholas Walker, Haoyan Huo, Sanghoon Lee, Kevin Cruse, John Dagdelen, Alexander Dunn, Kristin A Persson, Gerbrand Ceder, and Anubhav Jain. Quantifying the advantage of domain-specific pre-training on named entity recognition tasks in materials science. Patterns, 3(4):100488, 2022. [find](https://scholar.google.com/scholar?q=Amalie+Trewartha%2C+Nicholas+Walker%2C+Haoyan+Huo%2C+Sanghoon+Lee%2C+Kevin+Cruse%2C+John+Dagdelen%2C+Alexander+Dunn%2C+Kristin+A+Persson%2C+Gerbrand+Ceder%2C+and+Anubhav+Jain%2E+Quantifying+the+advantage+of+domain%2Dspecif)
3. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, and Alex Ray. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). Advances in Neural Information Processing Systems, 35:27730–27744, 2022.
4. Tong Xie, Yuwei Wan, Wei Huang, Yufei Zhou, Yixuan Liu, Qingyuan Linghu, Shaozhou Wang, Chunyu Kit, Clara Grazian, Wenjie Zhang, and Bram Hoex. Large language models as master key: Unlocking the secrets of materials science with gpt, 2023. [find](https://scholar.google.com/scholar?q=Tong+Xie%2C+Yuwei+Wan%2C+Wei+Huang%2C+Yufei+Zhou%2C+Yixuan+Liu%2C+Qingyuan+Linghu%2C+Shaozhou+Wang%2C+Chunyu+Kit%2C+Clara+Grazian%2C+Wenjie+Zhang%2C+and+Bram+Hoex%2E+Large+language+models+as+master+key%3A+Unlocking+the+secre)
5. Kevin Maik Jablonka, Philippe Schwaller, and Andres Ortega-guerrero. Is GPT-3 all you need for low-data discovery in chemistry ? ChemRxiv, pages 1–32, 2023. [find](https://scholar.google.com/scholar?q=Kevin+Maik+Jablonka%2C+Philippe+Schwaller%2C+and+Andres+Ortega%2Dguerrero%2E+Is+GPT%2D3+all+you+need+for+low%2Ddata+discovery+in+chemistry+%3F+ChemRxiv%2C+pages+1%E2%80%9332%2C+2023%2E)
6. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, and Faisal Azhar. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). arXiv preprint arXiv:2302.13971, 2023.
7. Bo Peng, Eric Alcaide, Quentin Anthony, Alon Albalak, Samuel Arcadinho, Huanqi Cao, Xin Cheng, Michael Chung, Matteo Grella, and Kranthi Kiran GV. RWKV: Reinventing RNNs for the Transformer Era. arXiv preprint arXiv:2305.13048, 2023. [find](https://scholar.google.com/scholar?q=Bo+Peng%2C+Eric+Alcaide%2C+Quentin+Anthony%2C+Alon+Albalak%2C+Samuel+Arcadinho%2C+Huanqi+Cao%2C+Xin+Cheng%2C+Michael+Chung%2C+Matteo+Grella%2C+and+Kranthi+Kiran+GV%2E+RWKV%3A+Reinventing+RNNs+for+the+Transformer+Era%2E+arXiv)
8. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-following llama model. [https://github.com/](https://github.com/) tatsu-lab/stanford_alpaca, 2023. [↗](https://github.com/)
9. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. [find](https://scholar.google.com/scholar?q=Wei%2DLin+Chiang%2C+Zhuohan+Li%2C+Zi+Lin%2C+Ying+Sheng%2C+Zhanghao+Wu%2C+Hao+Zhang%2C+Lianmin+Zheng%2C+Siyuan+Zhuang%2C+Yonghao+Zhuang%2C+Joseph+E%2E+Gonzalez%2C+Ion+Stoica%2C+and+Eric+P%2E+Xing%2E+Vicuna%3A+An+open%2Dsource+chatbot+i)
10. OpenAI. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/), 2023.
11. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-Instruct: Aligning Language Model with Self Generated Instructions. arXiv preprint arXiv:2212.10560, 2022. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+Self%2DInstruct%3A+Aligning+Language+Model+with+Self+Generated+Instructions%2E+arXiv+preprint+)
12. Haochun Wang, Chi Liu, Nuwa Xi, Zewen Qiang, Sendong Zhao, Bing Qin, and Ting Liu. [Huatuo: Tuning llama model with chinese medical knowledge](https://md.research.elamur.ai/free/en/papers/wang-2023-huatuo-tuning-llama-model-chinese/). arXiv preprint arXiv:2304.06975, 2023.
13. Mark D Wilkinson, Michel Dumontier, IJsbrand Jan Aalbersberg, Gabrielle Appleton, Myles Axton, Arie Baak, Niklas Blomberg, Jan-Willem Boiten, Luiz Bonino da Silva Santos, Philip E Bourne, Jildau Bouwman, Anthony J Brookes, Tim Clark, Mercè Crosas, Ingrid Dillo, Olivier Dumon, Scott Edmunds, Chris T… [find](https://scholar.google.com/scholar?q=Mark+D+Wilkinson%2C+Michel+Dumontier%2C+IJsbrand+Jan+Aalbersberg%2C+Gabrielle+Appleton%2C+Myles+Axton%2C+Arie+Baak%2C+Niklas+Blomberg%2C+Jan%2DWillem+Boiten%2C+Luiz+Bonino+da+Silva+Santos%2C+Philip+E+Bourne%2C+Jildau+Bouwm)
14. Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. [Distributed representations of words and phrases and their compositionality](https://md.research.elamur.ai/free/en/papers/mikolov-2013-distributed-representations-words-phrases-their/). Advances in neural information processing systems, 26, 2013.
15. Jeffrey Pennington, Richard Socher, and Christopher D Manning. [Glove: Global vectors for word representation](https://md.research.elamur.ai/free/en/papers/pennington-2014-glove-global-vectors-word-representation/). In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP), pages 1532–1543, 2014.
16. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805, 2018.
17. Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. Benchmarking generalization via in-context instructions on 1,600+ language tasks. arXiv e-prints, pages arXiv–2204, 2022. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Swaroop+Mishra%2C+Pegah+Alipoormolabashi%2C+Yeganeh+Kordi%2C+Amirreza+Mirzaei%2C+Anjana+Arunkumar%2C+Arjun+Ashok%2C+Arut+Selvan+Dhanasekaran%2C+Atharva+Naik%2C+David+Stap%2C+et+al%2E+Benchmarking+generaliza)
18. Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. Super-naturalinstructions: Generalization via declarative instructions on 1600+ nlp tasks. arXiv preprint arXiv:2204.07705,… [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Swaroop+Mishra%2C+Pegah+Alipoormolabashi%2C+Yeganeh+Kordi%2C+Amirreza+Mirzaei%2C+Anjana+Arunkumar%2C+Arjun+Ashok%2C+Arut+Selvan+Dhanasekaran%2C+Atharva+Naik%2C+David+Stap%2C+et+al%2E+Super%2Dnaturalinstructio)
19. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416, 2022. [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+Shayne+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+Eric+Li%2C+Xuezhi+Wang%2C+Mostafa+Dehghani%2C+Siddhartha+Brahma%2C+et+al%2E+Scaling+instruction%2Dfinetuned+language+models%2E+arXiv+prep)
20. Ilia Shumailov, Zakhar Shumaylov, Yiren Zhao, Yarin Gal, Nicolas Papernot, and Ross Anderson. [The curse of recursion: Training on generated data makes models forget](https://md.research.elamur.ai/free/en/papers/shumailov-2023-curse-recursion-training-generated-data/). arXiv preprint arxiv:2305.17493, 2023.
21. Iz Beltagy, Kyle Lo, and Arman Cohan. [Scibert: A pretrained language model for scientific text](https://md.research.elamur.ai/free/en/papers/beltagy-2019-scibert-pretrained-language-model-scientific/). arXiv preprint arXiv:1903.10676, 2019.
22. Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. [Domain-specific language model pretraining for biomedical natural language processing](https://md.research.elamur.ai/free/en/papers/gu-2020-domain-specific-language-model-pretraining/). ACM Transactions on Computing for Healthcare (HEALTH), 3(1):1–23, 2021.
23. Amalie Trewartha, Nicholas Walker, Haoyan Huo, Sanghoon Lee, Kevin Cruse, John Dagdelen, Alexander Dunn, Kristin A Persson, Gerbrand Ceder, and Anubhav Jain. Quantifying the advantage of domain-specific pre-training on named entity recognition tasks in materials science. Patterns, 3(4), 2022. [find](https://scholar.google.com/scholar?q=Amalie+Trewartha%2C+Nicholas+Walker%2C+Haoyan+Huo%2C+Sanghoon+Lee%2C+Kevin+Cruse%2C+John+Dagdelen%2C+Alexander+Dunn%2C+Kristin+A+Persson%2C+Gerbrand+Ceder%2C+and+Anubhav+Jain%2E+Quantifying+the+advantage+of+domain%2Dspecif)
24. Yanrong Ji, Zhihan Zhou, Han Liu, and Ramana V Davuluri. Dnabert: pre-trained bidirectional encoder representations from transformers model for dna-language in genome. Bioinformatics, 37(15):2112–2120, 2021. [find](https://scholar.google.com/scholar?q=Yanrong+Ji%2C+Zhihan+Zhou%2C+Han+Liu%2C+and+Ramana+V+Davuluri%2E+Dnabert%3A+pre%2Dtrained+bidirectional+encoder+representations+from+transformers+model+for+dna%2Dlanguage+in+genome%2E+Bioinformatics%2C+37%2815%29%3A2112%E2%80%932120)
25. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, and Amanda Askell. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). Advances in neural information processing systems, 33:1877–1901, 2020.
26. Chaoyi Wu, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. Pmc-llama: Further finetuning llama on medical papers. arXiv preprint arXiv:2304.[14454, 2023](https://md.research.elamur.ai/free/en/papers/wu-2023-pmc-llama-towards-building-open/).
27. Karan Singhal, Shekoofeh Azizi, Tao Tu, S Sara Mahdavi, Jason Wei, Hyung Won Chung, Nathan Scales, Ajay Tanwani, Heather Cole-Lewis, Stephen Pfohl, et al. [Large language models encode clinical knowledge](https://md.research.elamur.ai/free/en/papers/singhal-2022-large-language-models-encode-clinical/). arXiv preprint arXiv:2212.13138, 2022.
28. Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. [Galactica: A large language model for science](https://md.research.elamur.ai/free/en/papers/taylor-2022-galactica-large-language-model-science/). arXiv preprint arXiv:2211.09085, 2022.
29. Andres M Bran, Sam Cox, Andrew D White, and Philippe Schwaller. [Chemcrow: Augmenting large-language models with chemistry tools](https://md.research.elamur.ai/free/en/papers/bran-2023-chemcrow-augmenting-large-language/). arXiv preprint arXiv:2304.05376, 2023.
30. Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu. Biogpt: generative pre-trained transformer for biomedical text generation and mining. Briefings in Bioinformatics, 23(6), 2022. [find](https://scholar.google.com/scholar?q=Renqian+Luo%2C+Liai+Sun%2C+Yingce+Xia%2C+Tao+Qin%2C+Sheng+Zhang%2C+Hoifung+Poon%2C+and+Tie%2DYan+Liu%2E+Biogpt%3A+generative+pre%2Dtrained+transformer+for+biomedical+text+generation+and+mining%2E+Briefings+in+Bioinformatic)
31. Peter Clark. Elementary school science and math tests as a driver for ai: Take the aristo challenge! In Proceedings of the AAAI Conference on Artificial Intelligence, volume 29, pages 4019–4021, 2015. [find](https://scholar.google.com/scholar?q=Peter+Clark%2E+Elementary+school+science+and+math+tests+as+a+driver+for+ai%3A+Take+the+aristo+challenge%21+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+29%2C+pages+4019%E2%80%934021%2C+2015%2E)
32. Johannes Welbl, Nelson F Liu, and Matt Gardner. [Crowdsourcing multiple choice science questions](https://md.research.elamur.ai/free/en/papers/welbl-2017-crowdsourcing-multiple-choice-science-questions/). arXiv preprint arXiv:1707.06209, 2017.
33. Matthias Scheffler, Martin Aeschlimann, Martin Albrecht, Tristan Bereau, Hans-Joachim Bungartz, Claudia Felser, Mark Greiner, Axel Groß, Christoph T Koch, Kurt Kremer, et al. Fair data enabling new horizons for materials research. Nature, 604(7907):635–642, 2022. [find](https://scholar.google.com/scholar?q=Matthias+Scheffler%2C+Martin+Aeschlimann%2C+Martin+Albrecht%2C+Tristan+Bereau%2C+Hans%2DJoachim+Bungartz%2C+Claudia+Felser%2C+Mark+Greiner%2C+Axel+Gro%C3%9F%2C+Christoph+T+Koch%2C+Kurt+Kremer%2C+et+al%2E+Fair+data+enabling+new+ho)
34. Yin Fang, Xiaozhuan Liang, Ningyu Zhang, Kangwei Liu, Rui Huang, Zhuo Chen, Xiaohui Fan, and Huajun Chen. [Mol-instructions: A large-scale biomolecular instruction dataset for large language models](https://md.research.elamur.ai/free/en/papers/fang-2023-mol-instructions-large-scale-biomolecular/). arXiv preprint arXiv:2306.08018, 2023.
35. Alexander Dunn, Qi Wang, Alex Ganose, Daniel Dopp, and Anubhav Jain. Benchmarking materials property prediction methods: the Matbench test set and Automatminer reference algorithm. npj Computational Materials, 6(1):138, 2020. [find](https://scholar.google.com/scholar?q=Alexander+Dunn%2C+Qi+Wang%2C+Alex+Ganose%2C+Daniel+Dopp%2C+and+Anubhav+Jain%2E+Benchmarking+materials+property+prediction+methods%3A+the+Matbench+test+set+and+Automatminer+reference+algorithm%2E+npj+Computational+M)
36. Kevin Maik Jablonka, Philippe Schwaller, Andres Ortega-Guerrero, and Berend Smit. Is gpt-3 all you need for low-data discovery in chemistry? 2023. [find](https://scholar.google.com/scholar?q=Kevin+Maik+Jablonka%2C+Philippe+Schwaller%2C+Andres+Ortega%2DGuerrero%2C+and+Berend+Smit%2E+Is+gpt%2D3+all+you+need+for+low%2Ddata+discovery+in+chemistry%3F+2023%2E)
37. Anna Gaulton, Louisa J Bellis, A Patricia Bento, Jon Chambers, Mark Davies, Anne Hersey, Yvonne Light, Shaun McGlinchey, David Michalovich, and Bissan Al-Lazikani. ChEMBL: a large-scale bioactivity database for drug discovery. Nucleic acids research, 40(D1):D1100–D1107, 2012. [find](https://scholar.google.com/scholar?q=Anna+Gaulton%2C+Louisa+J+Bellis%2C+A+Patricia+Bento%2C+Jon+Chambers%2C+Mark+Davies%2C+Anne+Hersey%2C+Yvonne+Light%2C+Shaun+McGlinchey%2C+David+Michalovich%2C+and+Bissan+Al%2DLazikani%2E+ChEMBL%3A+a+large%2Dscale+bioactivity+da)
38. Xixi Qin, Xiaochen Du, Sampreeti Bhattacharya, Connor Clayton, Jun Hu, Manoj Jana, Svenja Janke, Rebecca Lau, Raul Laasner, and Andrew Levin. HybriD 3: Materials Property Database for Hybrid Organic-Inorganic Perovskites. In APS March Meeting Abstracts, volume 2021, pages X54–007, 2021. [find](https://scholar.google.com/scholar?q=Xixi+Qin%2C+Xiaochen+Du%2C+Sampreeti+Bhattacharya%2C+Connor+Clayton%2C+Jun+Hu%2C+Manoj+Jana%2C+Svenja+Janke%2C+Rebecca+Lau%2C+Raul+Laasner%2C+and+Andrew+Levin%2E+HybriD+3%3A+Materials+Property+Database+for+Hybrid+Organic%2DI)
39. John S Delaney. ESOL: estimating aqueous solubility directly from molecular structure. Journal of chemical information and computer sciences, 44(3):1000–1005, 2004. [find](https://scholar.google.com/scholar?q=John+S+Delaney%2E+ESOL%3A+estimating+aqueous+solubility+directly+from+molecular+structure%2E+Journal+of+chemical+information+and+computer+sciences%2C+44%283%29%3A1000%E2%80%931005%2C+2004%2E)
40. Osbel Almora, Derya Baran, Guillermo C Bazan, Carlos I Cabrera, Sule Erten-Ela, Karen Forberich, Fei Guo, Jens Hauch, Anita W Y Ho-Baillie, and T Jesper Jacobsson. Device performance of emerging photovoltaic materials (Version 3). Advanced Energy Materials, 13(1):2203313, 2023. [find](https://scholar.google.com/scholar?q=Osbel+Almora%2C+Derya+Baran%2C+Guillermo+C+Bazan%2C+Carlos+I+Cabrera%2C+Sule+Erten%2DEla%2C+Karen+Forberich%2C+Fei+Guo%2C+Jens+Hauch%2C+Anita+W+Y+Ho%2DBaillie%2C+and+T+Jesper+Jacobsson%2E+Device+performance+of+emerging+photo)
41. Masahiro Sakurai, Renhai Wang, Timothy Liao, Chao Zhang, Huaijun Sun, Yang Sun, Haidi Wang, Xin Zhao, Songyou Wang, and Balamurugan Balasubramanian. Discovering rare-earth-free magnetic materials through the development of a database. Physical review materials, 4(11):114408, 2020. [find](https://scholar.google.com/scholar?q=Masahiro+Sakurai%2C+Renhai+Wang%2C+Timothy+Liao%2C+Chao+Zhang%2C+Huaijun+Sun%2C+Yang+Sun%2C+Haidi+Wang%2C+Xin+Zhao%2C+Songyou+Wang%2C+and+Balamurugan+Balasubramanian%2E+Discovering+rare%2Dearth%2Dfree+magnetic+materials+thro)
42. Ya Zhuo, Aria Mansouri Tehrani, and Jakoah Brgoch. Predicting the band gaps of inorganic solids by machine learning. The journal of physical chemistry letters, 9(7):1668–1673, 2018. [find](https://scholar.google.com/scholar?q=Ya+Zhuo%2C+Aria+Mansouri+Tehrani%2C+and+Jakoah+Brgoch%2E+Predicting+the+band+gaps+of+inorganic+solids+by+machine+learning%2E+The+journal+of+physical+chemistry+letters%2C+9%287%29%3A1668%E2%80%931673%2C+2018%2E)
43. Yoshiyuki Kawazoe, J-Z Yu, A-P Tsai, and T Masumoto. Nonequilibrium phase diagrams of ternary amorphous alloys, 1997. [find](https://scholar.google.com/scholar?q=Yoshiyuki+Kawazoe%2C+J%2DZ+Yu%2C+A%2DP+Tsai%2C+and+T+Masumoto%2E+Nonequilibrium+phase+diagrams+of+ternary+amorphous+alloys%2C+1997%2E)
44. Seyed Mohamad Moosavi, Aditya Nandy, Kevin Maik Jablonka, Daniele Ongari, Jon Paul Janet, Peter G Boyd, Yongjin Lee, Berend Smit, and Heather J Kulik. Understanding the diversity of the metal-organic framework ecosystem. Nature communications, 11(1):1–10, 2020. [find](https://scholar.google.com/scholar?q=Seyed+Mohamad+Moosavi%2C+Aditya+Nandy%2C+Kevin+Maik+Jablonka%2C+Daniele+Ongari%2C+Jon+Paul+Janet%2C+Peter+G+Boyd%2C+Yongjin+Lee%2C+Berend+Smit%2C+and+Heather+J+Kulik%2E+Understanding+the+diversity+of+the+metal%2Dorganic+)
45. Shinji Nagasawa, Eman Al-Naamani, and Akinori Saeki. Computer-aided screening of conjugated polymers for organic solar cell: classification by random forest. The Journal of Physical Chemistry Letters, 9(10):2639–2646, 2018. [find](https://scholar.google.com/scholar?q=Shinji+Nagasawa%2C+Eman+Al%2DNaamani%2C+and+Akinori+Saeki%2E+Computer%2Daided+screening+of+conjugated+polymers+for+organic+solar+cell%3A+classification+by+random+forest%2E+The+Journal+of+Physical+Chemistry+Letters%2C)
46. Zongrui Pei, Junqi Yin, Jeffrey A Hawk, David E Alman, and Michael C Gao. Machine-learning informed prediction of high-entropy solid solution formation: Beyond the Hume-Rothery rules. npj Computational Materials, 6(1):50, 2020. [find](https://scholar.google.com/scholar?q=Zongrui+Pei%2C+Junqi+Yin%2C+Jeffrey+A+Hawk%2C+David+E+Alman%2C+and+Michael+C+Gao%2E+Machine%2Dlearning+informed+prediction+of+high%2Dentropy+solid+solution+formation%3A+Beyond+the+Hume%2DRothery+rules%2E+npj+Computationa)
47. Daniel Hickox-Young, Danilo Puggioni, and James M Rondinelli. Polar metals taxonomy for materials classification and discovery. Physical Review Materials, 7(1):10301, 2023. [find](https://scholar.google.com/scholar?q=Daniel+Hickox%2DYoung%2C+Danilo+Puggioni%2C+and+James+M+Rondinelli%2E+Polar+metals+taxonomy+for+materials+classification+and+discovery%2E+Physical+Review+Materials%2C+7%281%29%3A10301%2C+2023%2E)
48. Rohit Batra, Carmen Chen, Tania G Evans, Krista S Walton, and Rampi Ramprasad. Prediction of water stability of metal–organic frameworks using machine learning. Nature Machine Intelligence, 2(11):704–710, 2020. [find](https://scholar.google.com/scholar?q=Rohit+Batra%2C+Carmen+Chen%2C+Tania+G+Evans%2C+Krista+S+Walton%2C+and+Rampi+Ramprasad%2E+Prediction+of+water+stability+of+metal%E2%80%93organic+frameworks+using+machine+learning%2E+Nature+Machine+Intelligence%2C+2%2811%29%3A704%E2%80%93)

*Updated 2 November 2026*