--- title: "Leveraging large language models for predictive chemistry (2024)" description: "For inverse design, the question is reversed so that the model proposes a structure with a target… 5 references, cited by 25 papers here." authors: ["Kevin Maik Jablonka", "Philippe Schwaller", "Andres Ortega‐Guerrero", "Berend Smit"] year: 2024 venue: "Nature Machine Intelligence" source: "https://doi.org/10.1038/s42256-023-00788-1" doi: "10.1038/s42256-023-00788-1" licence: "CC BY 4.0 (publisher, via OpenAlex)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/jablonka-2024-leveraging-large-language-models-predictive/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/jablonka-2024-leveraging-large-language-models-predictive/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/) / Leveraging large language models for predictive chemistry # Leveraging large language models for predictive chemistry **Authors:** Kevin Maik Jablonka, [Philippe Schwaller](https://md.research.elamur.ai/free/en/authors/philippe-schwaller/), Andres Ortega‐Guerrero and Berend Smit · **Year:** 2024 · **Venue:** [Nature Machine Intelligence](https://md.research.elamur.ai/free/en/venues/nature-machine-intelligence/) ([Nature Machine Intelligence 2024](https://md.research.elamur.ai/free/en/venues/nature-machine-intelligence/2024/)) · **Publisher:** [Nature Portfolio](https://md.research.elamur.ai/free/en/publishers/nature-portfolio/) · **Licence:** [CC BY 4.0 (publisher, via OpenAlex)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://doi.org/10.1038/s42256-023-00788-1) · [PDF](https://www.nature.com/articles/s42256-023-00788-1.pdf) · [DOI 10.1038/s42256-023-00788-1](https://doi.org/10.1038/s42256-023-00788-1) · [OpenAlex](https://openalex.org/W4391561379) **Citations:** 367 citations (OpenAlex, 2026-10-14) · [25](https://md.research.elamur.ai/free/en/papers/jablonka-2024-leveraging-large-language-models-predictive/cited-by/) in our library, of the 149,439 papers whose bibliographies we read ## Summary Chemistry datasets are usually small, so researchers tend to build bespoke machine learning models with hand-crafted chemical descriptors. This requires specialist skill. Jablonka and colleagues test a simpler route. They take GPT-3, a general-purpose LLM not trained specifically on chemistry, and fine-tune it on chemical questions written as plain text and paired with their answers. They benchmark this approach on three kinds of data: - **molecules:** HOMO–LUMO gaps, water solubility, photovoltaic performance - **materials:** alloys, metal–organic frameworks, polymers - **reactions:** yields of cross-coupling reactions The tasks include classification, regression and inverse design. For inverse design, the question is reversed so that the model proposes a structure with a target property. With little training data, the fine-tuned model usually equals or beats dedicated baselines and often needs fewer examples. Conventional models tend to catch up as datasets grow. Results stay strong across different molecular representations. The authors argue that querying a fine-tuned LLM could become a cheap default baseline and a quick way to start new chemistry projects. Future specialized models would then need to outperform it. ## Key points - On high-entropy alloy phase prediction (single vs. multi-phase), GPT-3 trained on about 50 examples performed on par with a random forest from prior work that used roughly 1,126 training points. Automatminer and CrabNet were also compared. - All main results use the smallest GPT-3 variant (ada) via the OpenAI API with one fixed setting: 8 epochs and a learning rate multiplier of 0.02. Prompts and hyperparameters were not optimized. - IUPAC names, SMILES and SELFIES all gave good results, and IUPAC names were frequently the best. This makes the approach accessible to non-specialists. - For inverse design, iterative self-fine-tuning on generated molecules (evaluated with xTB, starting from QMugs) shifted HOMO–LUMO gaps toward a 4.0 eV target within four rounds. - In-context learning without fine-tuning also worked, especially with the largest GPT-3 models and GPT-4. The open-source GPT-J-6B could be fine-tuned on consumer hardware using 8-bit quantization and LoRA. The code is released in the gptchem and chemlift repositories. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [École Polytechnique Fédérale de Lausanne](https://md.research.elamur.ai/free/en/universities/ecole-polytechnique-federale-de-lausanne/) — University, Lausanne, Switzerland · 56 works · 8,189 citations - [Friedrich Schiller University Jena](https://md.research.elamur.ai/free/en/universities/friedrich-schiller-university-jena/) — University, Jena, Germany · 4 works · 579 citations Also on the byline: [Helmholtz Institute Jena](https://ror.org/02rzw6h69). 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🧮 Computational Theory and Mathematics](https://md.research.elamur.ai/free/en/shelves/computer-science/computational-theory-and-mathematics/) · 496 works · [🧱 Materials Science](https://md.research.elamur.ai/free/en/shelves/materials-science/) · 58 works · [🧱 Materials Chemistry](https://md.research.elamur.ai/free/en/shelves/materials-science/materials-chemistry/) · 58 works · [🧪 Chemistry](https://md.research.elamur.ai/free/en/shelves/chemistry/) · 6 works · [🧪 Physical and Theoretical Chemistry](https://md.research.elamur.ai/free/en/shelves/chemistry/physical-and-theoretical-chemistry/) · 5 works 🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 4,145 works ## Cite Cite as: Kevin Maik Jablonka, Philippe Schwaller, Andres Ortega‐Guerrero and Berend Smit. 2024. Leveraging large language models for predictive chemistry. Nature Machine Intelligence. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{jablonka2024leveraging, title = {{Leveraging large language models for predictive chemistry}}, author = {Kevin Maik Jablonka, Philippe Schwaller, Andres Ortega‐Guerrero and Berend Smit}, booktitle = {Nature Machine Intelligence}, year = {2024}, doi = {10.1038/s42256-023-00788-1}, url = {https://doi.org/10.1038/s42256-023-00788-1} } ``` - [← previous RULER: What's the Real Context Size of Your Long-Context Language Models?](https://md.research.elamur.ai/free/en/papers/hsieh-2024-ruler-s-real-context-size/) - [next → Adaptive-RAG: Learning to Adapt Retrieval-Augmented Large Language Models through Question Complexity](https://md.research.elamur.ai/free/en/papers/jeong-2024-adaptive-rag-learning-adapt-retrieval/) - [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/) ## References 5 references, 3 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. 1. 2. 3. Bommasani, R. et al. [On the opportunities and risks of foundation models](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/). Preprint at https://arxiv.org/abs/2108.07258 (2021). Vaswani, A. et al. Attention is all you need. Adv. Neural Inf. Process. Syst. https://proceedings.neurips.cc/paper/2017/file/ 3f5ee243547dee91fbd053c1c4a845aa-Paper… [↗](https://arxiv.org/abs/2108.07258) 2. Hoffmann, J. et al. An empirical analysis of compute-optimal large language model training. Adv. Neural Inf. Process. Syst. 35, 30016–30030 (2022). 5. Brown, T. et al. Language models are few-shot learners. Adv. Neural Inf. Process. Syst. 33, 1877–1901 (2020). 6. Edwards, C. N., Lai, T., Ros, K., Ho… — [Galactica: A Large Language Model for Science](https://md.research.elamur.ai/free/en/papers/taylor-2022-galactica-large-language-model-science/) 3. ICLR 2024 [Conference, OpenReview](https://md.research.elamur.ai/free/en/papers/dai-2023-why-can-gpt-learn-context/) [https://openreview.net/forum?id=X7nz6ljg9Y](https://openreview.net/forum?id=X7nz6ljg9Y) (2023). 27. Schwaller, P. et al. Molecular transformer: a model for uncertainty-calibrated chemical reaction prediction. ACS Cent. Sci. 5, 1572–1583 (2019). 28. Winter, B., Winter, C., Schilling, J. & Bardow, A. A smile is a… [↗](https://openreview.net/forum?id=X7nz6ljg9Y) 4. RDKit: Open-source Cheminformatics; (2023) [http://www.rdkit.org](http://www.rdkit.org) 64. Preuer, K., Renz, P., Unterthiner, T., Hochreiter, S. & Klambauer, G. Fréchet ChemNet distance: a metric for generative models for molecules in drug discovery. J. Chem. Inf. Model. 58, 1736–1741 (2018). 65. Probst, D. & Reymond, J.-… [↗](http://www.rdkit.org) 5. Author contributions K.M.J. developed the machine learning approach with feedback from P.S. and B.S. K.M.J. and B.S. wrote the article. A.O.-G. contributed to the density functional theory calculations. Funding Open access funding provided by EPFL Lausanne. Competing interests The authors declare no… — [doi:10.1038/s42256-023-00788-1](https://doi.org/10.1038/s42256-023-00788-1) [find](https://scholar.google.com/scholar?q=Author+contributions+K%2EM%2EJ%2E+developed+the+machine+learning+approach+with+feedback+from+P%2ES%2E+and+B%2ES%2E+K%2EM%2EJ%2E+and+B%2ES%2E+wrote+the+article%2E+A%2EO%2E%2DG%2E+contributed+to+the+density+functional+theory+calculation) *Updated 3 November 2026*