---
title: "GIT-Mol — paper by Liu et al. (2023) — summary and ideas"
description: "Most language models for chemistry handle molecules only as text, such as SMILES strings or… 35 references, cited by 24 papers here."
authors: ["Pengfei Liu", "Yiming Ren", "Jun Tao", "Zhixiang Ren"]
year: 2023
venue: "arXiv:2308.06911 · Computers in Biology and Medicine, 108073, 2024, ISSN 0010-4825"
source: "https://arxiv.org/abs/2308.06911"
doi: "10.1016/j.compbiomed.2024.108073"
arxiv: "2308.06911"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2023-git-mol-multi-modal-large/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2023-git-mol-multi-modal-large/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/) / GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text

# GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text

**Authors:** [Pengfei Liu](https://md.research.elamur.ai/free/en/authors/pengfei-liu/), Yiming Ren, Jun Tao and Zhixiang Ren · **Year:** 2023 · **Venue:** arXiv:2308.06911 · Computers in Biology and Medicine, 108,073, 2024, ISSN 0010-4825 · **Publisher:** [Elsevier BV](https://md.research.elamur.ai/free/en/publishers/elsevier-bv/) · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://doi.org/10.1016/j.compbiomed.2024.108073) · [PDF](https://arxiv.org/pdf/2308.06911) · [DOI 10.1016/j.compbiomed.2024.108073](https://doi.org/10.1016/j.compbiomed.2024.108073) · [arXiv 2308.06911](https://arxiv.org/abs/2308.06911) · [OpenAlex](https://openalex.org/W4391345293)

**Citations:** 89 citations (OpenAlex, 2026-10-14) · [24](https://md.research.elamur.ai/free/en/papers/liu-2023-git-mol-multi-modal-large/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Most language models for chemistry handle molecules only as text, such as SMILES strings or captions. They miss information carried by molecular graphs and by depictions of molecules in images. The authors address this with GIT-Mol, a multi-modal large language model of about 700M parameters that combines graph, image, and text inputs.

Its central component is GIT-Former, a fusion module inspired by the Q-Former from BLIP2. It uses learnable queries with self-attention and cross-attention to project inputs of different lengths and modalities into one fixed-size shared space, aligned at the level of whole molecules. GIT-Former is pre-trained with two self-supervised objectives (XTM and XTC) while the graph and image encoders stay frozen. A MolT5 decoder then turns the fused embeddings into SMILES or captions, and an MLP head handles property prediction.

The model covers molecule captioning, text-based molecule generation, image-to-SMILES recognition, and property prediction. It reports clear gains over baselines, which suggests that combining several modalities can benefit AI-aided drug discovery.

## Key points
- **Results:** GIT-Mol improves property prediction accuracy by roughly 5–10% and the validity of generated molecules by 20.2% over baselines.
- **Ablation:** fusing several modalities outperforms single-modality variants by about 10–15%.
- **Pre-training:** the two objectives are XTM, which learns whether a cross-modal input and a text belong together, and XTC, which contrasts cross-modal representations against text representations. Each run pairs one to three modalities with a target text modality, uses learning rates between 1e-4 and 5e-5, and trains for fewer than 20 epochs.
- **Design and limitations:** an "any-to-language" translation strategy is meant to extend the model to compound name recognition and reaction prediction. Training speed remains a limitation, and the authors plan to address it with LoRA, Adapters, or PEFT.
- **Availability:** code and data are at github.com/AI-HPC-Research-Team/GIT-Mol for non-commercial use only.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Sun Yat-sen University](https://md.research.elamur.ai/free/en/universities/sun-yat-sen-university/) — University, Guangzhou, China · 80 works · 9,799 citations
- [Peng Cheng Laboratory](https://md.research.elamur.ai/free/en/labs/peng-cheng-laboratory/) — Research institute, Shenzhen, China · 48 works · 2,812 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,621 works · [🧮 Computational Theory and Mathematics](https://md.research.elamur.ai/free/en/shelves/computer-science/computational-theory-and-mathematics/) · 496 works · [🧱 Materials Science](https://md.research.elamur.ai/free/en/shelves/materials-science/) · 58 works · [🧱 Materials Chemistry](https://md.research.elamur.ai/free/en/shelves/materials-science/materials-chemistry/) · 58 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 4,145 works · [🖼️ Vision & language](https://md.research.elamur.ai/free/en/topics/multimodal/) · 827 works

## Cite

Cite as: Pengfei Liu, Yiming Ren, Jun Tao and Zhixiang Ren. 2023. GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text. arXiv:2308.06911 · Computers in Biology and Medicine, 108,073, 2024, ISSN 0010-4825.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{liu2023git,
  title = {{GIT-Mol: A Multi-modal Large Language Model for Molecular Science with Graph, Image, and Text}},
  author = {Pengfei Liu, Yiming Ren, Jun Tao and Zhixiang Ren},
  year = {2023},
  doi = {10.1016/j.compbiomed.2024.108073},
  eprint = {2308.06911},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1016/j.compbiomed.2024.108073}
}
```

- [← previous Exploring the Use of Large Language Models for Reference-Free Text Quality Evaluation: An Empirical Study](https://md.research.elamur.ai/free/en/papers/chen-2023-exploring-use-large-language-models/)
- [next → WiCE: Real-World Entailment for Claims in Wikipedia](https://md.research.elamur.ai/free/en/papers/kamoi-2023-wice-real-world-entailment-claims/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/)

## References

35 references, 14 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alayrac, J.B., Donahue, J., Luc, P., Miech, A., Barr, I., Hasson, Y., Lenc, K., Mensch, A., Millican, K., Reynolds, M., et al., 2022. [Flamingo: a visual language model for few-shot learning](https://md.research.elamur.ai/free/en/papers/alayrac-2022-flamingo-visual-language-model-few/). Advances in Neural Information Processing Systems 35, 23716–23736.
2. Bao, F., Nie, S., Xue, K., Li, C., Pu, S., Wang, Y., Yue, G., Cao, Y., Su, H., Zhu, J., 2023. One transformer fits all distributions in multi-modal diffusion at scale. arXiv preprint arXiv:2303.06555 . Bao, H., Wang, W., Dong, L., Liu, Q., Mohammed, O.K., Aggarwal, K., Som, S., Piao, S., Wei, F., 20… [find](https://scholar.google.com/scholar?q=Bao%2C+F%2E%2C+Nie%2C+S%2E%2C+Xue%2C+K%2E%2C+Li%2C+C%2E%2C+Pu%2C+S%2E%2C+Wang%2C+Y%2E%2C+Yue%2C+G%2E%2C+Cao%2C+Y%2E%2C+Su%2C+H%2E%2C+Zhu%2C+J%2E%2C+2023%2E+One+transformer+fits+all+distributions+in+multi%2Dmodal+diffusion+at+scale%2E+arXiv+preprint+arXiv%3A2303%2E06555+)
3. Beltagy, I., Lo, K., Cohan, A., 2019. [Scibert: A pretrained language model for scientific text](https://md.research.elamur.ai/free/en/papers/beltagy-2019-scibert-pretrained-language-model-scientific/). arXiv preprint arXiv:1903.10676 . Bento, A.P., Hersey, A., Félix, E., Landrum, G., Gaulton, A., Atkinson, F., Bellis, L.J., De Veij, M., Leach, A.R., 2020. An open source chemical structure curation pipel…
4. Bilodeau, C., Jin, W., Jaakkola, T., Barzilay, R., Jensen, K.F., 2022. Generative models for molecular discovery: Recent advances and challenges. Wiley Interdisciplinary Reviews: Computational Molecular Science 12, e1608. [find](https://scholar.google.com/scholar?q=Bilodeau%2C+C%2E%2C+Jin%2C+W%2E%2C+Jaakkola%2C+T%2E%2C+Barzilay%2C+R%2E%2C+Jensen%2C+K%2EF%2E%2C+2022%2E+Generative+models+for+molecular+discovery%3A+Recent+advances+and+challenges%2E+Wiley+Interdisciplinary+Reviews%3A+Computational+Molecul)
5. Devlin, J., Chang, M.W., Lee, K., Toutanova, K., 2018. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). arXiv preprint arXiv:1810.04805 . Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., Dehghani, M., Minderer, M., Heigold, G.,…
6. Edwards, C., Zhai, C., Ji, H., 2021. [Text2mol: Cross-modal molecule retrieval with natural language queries](https://md.research.elamur.ai/free/en/papers/edwards-2021-text2mol-cross-modal-molecule-retrieval/), in: Proceedings of the 2021
7. Conference on Empirical Methods in Natural Language Processing, pp. 595–607. Filippov, I.V., Nicklaus, M.C., 2009. Optical structure recognition software to recover chemical information: Osra, an open source solution. [find](https://scholar.google.com/scholar?q=Conference+on+Empirical+Methods+in+Natural+Language+Processing%2C+pp%2E+595%E2%80%93607%2E+Filippov%2C+I%2EV%2E%2C+Nicklaus%2C+M%2EC%2E%2C+2009%2E+Optical+structure+recognition+software+to+recover+chemical+information%3A+Osra%2C+an+open)
8. Floridi, L., Chiriatti, M., 2020. Gpt-3: Its nature, scope, limits, and consequences. Minds and Machines 30, 681–694. [find](https://scholar.google.com/scholar?q=Floridi%2C+L%2E%2C+Chiriatti%2C+M%2E%2C+2020%2E+Gpt%2D3%3A+Its+nature%2C+scope%2C+limits%2C+and+consequences%2E+Minds+and+Machines+30%2C+681%E2%80%93694%2E)
9. Hastings, J., Owen, G., Dekker, A., Ennis, M., Kale, N., Muthukrishnan, V., Turner, S., Swainston, N., Mendes, P., Steinbeck, C., 2016. Chebi in 2016: Improved services and an expanding collection of metabolites. Nucleic Acids Research 44, D1214–D1219. [find](https://scholar.google.com/scholar?q=Hastings%2C+J%2E%2C+Owen%2C+G%2E%2C+Dekker%2C+A%2E%2C+Ennis%2C+M%2E%2C+Kale%2C+N%2E%2C+Muthukrishnan%2C+V%2E%2C+Turner%2C+S%2E%2C+Swainston%2C+N%2E%2C+Mendes%2C+P%2E%2C+Steinbeck%2C+C%2E%2C+2016%2E+Chebi+in+2016%3A+Improved+services+and+an+expanding+collection+of+)
10. He, K., Zhang, X., Ren, S., Sun, J., 2016. [Deep residual learning for image recognition](https://md.research.elamur.ai/free/en/papers/he-2015-deep-residual-learning-image-recognition/), in: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778.
11. Hou, Z., Liu, X., Cen, Y., Dong, Y., Yang, H., Wang, C., Tang, J., 2022. Graphmae: Self-supervised masked graph autoencoders, in: Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, pp. 594–604. [find](https://scholar.google.com/scholar?q=Hou%2C+Z%2E%2C+Liu%2C+X%2E%2C+Cen%2C+Y%2E%2C+Dong%2C+Y%2E%2C+Yang%2C+H%2E%2C+Wang%2C+C%2E%2C+Tang%2C+J%2E%2C+2022%2E+Graphmae%3A+Self%2Dsupervised+masked+graph+autoencoders%2C+in%3A+Proceedings+of+the+28th+ACM+SIGKDD+Conference+on+Knowledge+Discovery+a)
12. Huang, S., Dong, L., Wang, W., Hao, Y., Singhal, S., Ma, S., Lv, T., Cui, L., Mohammed, O.K., Liu, Q., et al., 2023. [Language is not all you need: Aligning perception with language models](https://md.research.elamur.ai/free/en/papers/huang-2023-language-all-need-aligning-perception/). arXiv preprint arXiv:2302.14045 . Jia, C., Yang, Y., Xia, Y., Chen, Y.T., Parekh, Z., Pham, H., Le, Q., Sung, Y…
13. Kim, S., Chen, J., Cheng, T., Gindulyte, A., He, J., He, S., Li, Q., Shoemaker, B.A., Thiessen, P.A., Yu, B., et al., 2019. [find](https://scholar.google.com/scholar?q=Kim%2C+S%2E%2C+Chen%2C+J%2E%2C+Cheng%2C+T%2E%2C+Gindulyte%2C+A%2E%2C+He%2C+J%2E%2C+He%2C+S%2E%2C+Li%2C+Q%2E%2C+Shoemaker%2C+B%2EA%2E%2C+Thiessen%2C+P%2EA%2E%2C+Yu%2C+B%2E%2C+et+al%2E%2C+2019%2E)
14. Pubchem 2019 update: improved access to chemical data. Nucleic Acids Research 47, D1102–D1109. [find](https://scholar.google.com/scholar?q=Pubchem+2019+update%3A+improved+access+to+chemical+data%2E+Nucleic+Acids+Research+47%2C+D1102%E2%80%93D1109%2E)
15. Li, J., Li, D., Savarese, S., Hoi, S., 2023a. [Blip-2: Bootstrapping language-image pre-training with frozen image encoders and large language models](https://md.research.elamur.ai/free/en/papers/li-2023-blip-2-bootstrapping-language-image/). arXiv preprint arXiv:2301.12597 . Li, J., Liu, Y., Fan, W., Wei, X.Y., Liu, H., Tang, J., Li, Q., 2023b. Empowering molecule discovery for molecule-ca…
16. Liu, S., Nie, W., Wang, C., Lu, J., Qiao, Z., Liu, L., Tang, J., Xiao, C., Anandkumar, A., 2022. Multi-modal molecule structure-text model for text-based retrieval and editing. arXiv preprint arXiv:2212.10789 . Liu, S., Wang, H., Liu, W., Lasenby, J., Guo, H., Tang, J., 2021a. Pre-training molecular… [find](https://scholar.google.com/scholar?q=Liu%2C+S%2E%2C+Nie%2C+W%2E%2C+Wang%2C+C%2E%2C+Lu%2C+J%2E%2C+Qiao%2C+Z%2E%2C+Liu%2C+L%2E%2C+Tang%2C+J%2E%2C+Xiao%2C+C%2E%2C+Anandkumar%2C+A%2E%2C+2022%2E+Multi%2Dmodal+molecule+structure%2Dtext+model+for+text%2Dbased+retrieval+and+editing%2E+arXiv+preprint+arXiv%3A22)
17. Luo, R., Sun, L., Xia, Y., Qin, T., Zhang, S., Poon, H., Liu, T.Y., 2022. Biogpt: generative pre-trained transformer for biomedical text generation and mining. Briefings in Bioinformatics 23, bbac409. [find](https://scholar.google.com/scholar?q=Luo%2C+R%2E%2C+Sun%2C+L%2E%2C+Xia%2C+Y%2E%2C+Qin%2C+T%2E%2C+Zhang%2C+S%2E%2C+Poon%2C+H%2E%2C+Liu%2C+T%2EY%2E%2C+2022%2E+Biogpt%3A+generative+pre%2Dtrained+transformer+for+biomedical+text+generation+and+mining%2E+Briefings+in+Bioinformatics+23%2C+bbac409%2E)
18. OpenAI, 2023. [GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). CoRR abs/2303.08774.
19. Peryea, T., Katzel, D., Zhao, T., Southall, N., Nguyen, D.T., 2019. Molvec: Open source library for chemical structure recognition, in: Abstracts of Papers of the American Chemical Society, Amer Chemical Soc 1155 16TH ST, NW, WASHINGTON, DC 20036 USA. [find](https://scholar.google.com/scholar?q=Peryea%2C+T%2E%2C+Katzel%2C+D%2E%2C+Zhao%2C+T%2E%2C+Southall%2C+N%2E%2C+Nguyen%2C+D%2ET%2E%2C+2019%2E+Molvec%3A+Open+source+library+for+chemical+structure+recognition%2C+in%3A+Abstracts+of+Papers+of+the+American+Chemical+Society%2C+Amer+Chemi)
20. Radford, A., Kim, J.W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al., 2021. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/), in: International Conference on Machine Learning, PMLR. pp. 8748–8763.
21. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., Sutskever, I., et al., 2019. Language models are unsupervised multitask learners. OpenAI blog 1, 9. [find](https://scholar.google.com/scholar?q=Radford%2C+A%2E%2C+Wu%2C+J%2E%2C+Child%2C+R%2E%2C+Luan%2C+D%2E%2C+Amodei%2C+D%2E%2C+Sutskever%2C+I%2E%2C+et+al%2E%2C+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog+1%2C+9%2E)
22. Raffel, C., Shazeer, N., Roberts, A., Lee, K., Narang, S., Matena, M., Zhou, Y., Li, W., Liu, P.J., 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). The Journal of Machine Learning Research 21, 5485–5551.
23. Rajan, K., Zielesny, A., Steinbeck, C., 2021. Decimer 1.0: deep learning for chemical image recognition using transformers. Journal of Cheminformatics 13, 1–16. [find](https://scholar.google.com/scholar?q=Rajan%2C+K%2E%2C+Zielesny%2C+A%2E%2C+Steinbeck%2C+C%2E%2C+2021%2E+Decimer+1%2E0%3A+deep+learning+for+chemical+image+recognition+using+transformers%2E+Journal+of+Cheminformatics+13%2C+1%E2%80%9316%2E)
24. Ramesh, A., Dhariwal, P., Nichol, A., Chu, C., Chen, M., 2022. [Hierarchical text-conditional image generation with clip latents](https://md.research.elamur.ai/free/en/papers/ramesh-2022-hierarchical-text-conditional-image-generation/). arXiv preprint arXiv:2204.06125 . Reed, S., Zolna, K., Parisotto, E., Colmenarejo, S.G., Novikov, A., Barth-Maron, G., Gimenez, M., Sulsky, Y., Kay, J., Springenberg, J.T.…
25. Stefanini, M., Cornia, M., Baraldi, L., Cascianelli, S., Fiameni, G., Cucchiara, R., 2022. From show to tell: A survey on deep learning-based image captioning. IEEE Transactions on Pattern Analysis and Machine Intelligence 45, 539–559. [find](https://scholar.google.com/scholar?q=Stefanini%2C+M%2E%2C+Cornia%2C+M%2E%2C+Baraldi%2C+L%2E%2C+Cascianelli%2C+S%2E%2C+Fiameni%2C+G%2E%2C+Cucchiara%2C+R%2E%2C+2022%2E+From+show+to+tell%3A+A+survey+on+deep+learning%2Dbased+image+captioning%2E+IEEE+Transactions+on+Pattern+Analysis+an)
26. Su, B., Du, D., Yang, Z., Zhou, Y., Li, J., Rao, A., Sun, H., Lu, Z., Wen, J.R., 2022. A molecular multimodal foundation model associating molecule graphs with natural language. arXiv preprint arXiv:2209.05481 . Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł.,… — [arXiv:2209.05481](https://arxiv.org/abs/2209.05481) [find](https://scholar.google.com/scholar?q=Su%2C+B%2E%2C+Du%2C+D%2E%2C+Yang%2C+Z%2E%2C+Zhou%2C+Y%2E%2C+Li%2C+J%2E%2C+Rao%2C+A%2E%2C+Sun%2C+H%2E%2C+Lu%2C+Z%2E%2C+Wen%2C+J%2ER%2E%2C+2022%2E+A+molecular+multimodal+foundation+model+associating+molecule+graphs+with+natural+language%2E+arXiv+preprint+arXiv%3A2)
27. Wang, H., Ma, S., Huang, S., Dong, L., Wang, W., Peng, Z., Wu, Y., Bajaj, P., Singhal, S., Benhaim, A., et al., 2022a. Foundation transformers. arXiv preprint arXiv:2210.06423 . Wang, J., Shen, Z., Liao, Y., Yuan, Z., Li, S., He, G., Lan, M., Qian, X., Zhang, K., Li, H., 2022b. Multi-modal chemical… — [arXiv:2210.06423](https://arxiv.org/abs/2210.06423) [find](https://scholar.google.com/scholar?q=Wang%2C+H%2E%2C+Ma%2C+S%2E%2C+Huang%2C+S%2E%2C+Dong%2C+L%2E%2C+Wang%2C+W%2E%2C+Peng%2C+Z%2E%2C+Wu%2C+Y%2E%2C+Bajaj%2C+P%2E%2C+Singhal%2C+S%2E%2C+Benhaim%2C+A%2E%2C+et+al%2E%2C+2022a%2E+Foundation+transformers%2E+arXiv+preprint+arXiv%3A2210%2E06423+%2E+Wang%2C+J%2E%2C+Shen%2C+Z%2E%2C+Li)
28. Wang, W., Bao, H., Dong, L., Bjorck, J., Peng, Z., Liu, Q., Aggarwal, K., Mohammed, O.K., Singhal, S., Som, S., et al., 2022c. Image as a foreign language: Beit pretraining for all vision and vision-language tasks. arXiv preprint arXiv:2208.10442 . Wang, Y., Wang, J., Cao, Z., Barati Farimani, A., 2… [find](https://scholar.google.com/scholar?q=Wang%2C+W%2E%2C+Bao%2C+H%2E%2C+Dong%2C+L%2E%2C+Bjorck%2C+J%2E%2C+Peng%2C+Z%2E%2C+Liu%2C+Q%2E%2C+Aggarwal%2C+K%2E%2C+Mohammed%2C+O%2EK%2E%2C+Singhal%2C+S%2E%2C+Som%2C+S%2E%2C+et+al%2E%2C+2022c%2E+Image+as+a+foreign+language%3A+Beit+pretraining+for+all+vision+and+vision%2Dl)
29. Wei, J., Wang, X., Schuurmans, D., Bosma, M., Xia, F., Chi, E., Le, Q.V., Zhou, D., et al., 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). Advances in Neural Information Processing Systems 35, 24824–24837.
30. Weininger, D., 1988. Smiles, a chemical language and information system. 1. introduction to methodology and encoding rules. Journal of Chemical Information and Computer Sciences 28, 31–36. [find](https://scholar.google.com/scholar?q=Weininger%2C+D%2E%2C+1988%2E+Smiles%2C+a+chemical+language+and+information+system%2E+1%2E+introduction+to+methodology+and+encoding+rules%2E+Journal+of+Chemical+Information+and+Computer+Sciences+28%2C+31%E2%80%9336%2E)
31. Wu, C., Yin, S., Qi, W., Wang, X., Tang, Z., Duan, N., 2023. [Visual chatgpt: Talking, drawing and editing with visual foundation models](https://md.research.elamur.ai/free/en/papers/wu-2023-visual-chatgpt-talking-drawing-editing/). arXiv preprint arXiv:2303.04671 . Wu, Z., Ramsundar, B., Feinberg, E.N., Gomes, J., Geniesse, C., Pappu, A.S., Leswing, K., Pande, V., 2018. Moleculenet: a benchma…
32. Xu, Z., Li, J., Yang, Z., Li, S., Li, H., 2022. Swinocsr: end-to-end optical chemical structure recognition using a swin transformer. Journal of Cheminformatics 14, 1–13. [find](https://scholar.google.com/scholar?q=Xu%2C+Z%2E%2C+Li%2C+J%2E%2C+Yang%2C+Z%2E%2C+Li%2C+S%2E%2C+Li%2C+H%2E%2C+2022%2E+Swinocsr%3A+end%2Dto%2Dend+optical+chemical+structure+recognition+using+a+swin+transformer%2E+Journal+of+Cheminformatics+14%2C+1%E2%80%9313%2E)
33. Yang, L., Zhang, Z., Song, Y., Hong, S., Xu, R., Zhao, Y., Shao, Y., Zhang, W., Cui, B., Yang, M.H., 2022. Diffusion models: A comprehensive survey of methods and applications. arXiv preprint arXiv:2209.00796 . Zeng, Z., Yao, Y., Liu, Z., Sun, M., 2022. A deep-learning system bridging molecule struc… — [arXiv:2209.00796](https://arxiv.org/abs/2209.00796) [find](https://scholar.google.com/scholar?q=Yang%2C+L%2E%2C+Zhang%2C+Z%2E%2C+Song%2C+Y%2E%2C+Hong%2C+S%2E%2C+Xu%2C+R%2E%2C+Zhao%2C+Y%2E%2C+Shao%2C+Y%2E%2C+Zhang%2C+W%2E%2C+Cui%2C+B%2E%2C+Yang%2C+M%2EH%2E%2C+2022%2E+Diffusion+models%3A+A+comprehensive+survey+of+methods+and+applications%2E+arXiv+preprint+arXiv%3A22)
34. Zhou, G., Gao, Z., Ding, Q., Zheng, H., Xu, H., Wei, Z., Zhang, L., Ke, G., 2023. Uni-mol: A universal 3d molecular representation learning framework, in: The Eleventh International Conference on Learning Representations. URL: [https://openreview.net/](https://openreview.net/) forum?id=6K2RM6wVqKu. [↗](https://openreview.net/)
35. Zhu, D., Chen, J., Shen, X., Li, X., Elhoseiny, M., 2023. [Minigpt-4: Enhancing vision-language understanding with advanced large language models](https://md.research.elamur.ai/free/en/papers/zhu-2023-minigpt-4-enhancing-vision-language/). arXiv preprint arXiv:2304.10592 .

*Updated 3 November 2026*