---
title: "UniversalNER — paper by Zhou et al. (2023)"
description: "The authors propose \"targeted distillation\" through mission-focused instruction tuning and test it on… 75 references, cited by 96 papers here."
authors: ["Wenxuan Zhou", "Sheng Zhang", "Yu Gu", "Muhao Chen", "Hoifung Poon"]
year: 2023
venue: "arXiv:2308.03279"
source: "https://arxiv.org/abs/2308.03279"
doi: "10.48550/arxiv.2308.03279"
arxiv: "2308.03279"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/zhou-2023-universalner-targeted-distillation-large-language/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/zhou-2023-universalner-targeted-distillation-large-language/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/) / UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition

# UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition

**Authors:** [Wenxuan Zhou](https://md.research.elamur.ai/free/en/authors/wenxuan-zhou/), [Sheng Zhang](https://md.research.elamur.ai/free/en/authors/sheng-zhang/), [Yu Gu](https://md.research.elamur.ai/free/en/authors/yu-gu/), [Muhao Chen](https://md.research.elamur.ai/free/en/authors/muhao-chen/) and [Hoifung Poon](https://md.research.elamur.ai/free/en/authors/hoifung-poon/) · **Year:** 2023 · **Venue:** arXiv:2308.03279 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2308.03279) · [PDF](https://arxiv.org/pdf/2308.03279) · [DOI 10.48550/arxiv.2308.03279](https://doi.org/10.48550/arxiv.2308.03279) · [arXiv 2308.03279](https://arxiv.org/abs/2308.03279) · [OpenAlex](https://openalex.org/W4385681676)

**Citations:** 24 citations (OpenAlex, 2026-10-14) · [96](https://md.research.elamur.ai/free/en/papers/zhou-2023-universalner-targeted-distillation-large-language/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper asks whether a compact open model can match a proprietary LLM on one broad task family instead of copying it everywhere. General distilled assistants like Alpaca and Vicuna fall well short of ChatGPT on specific downstream jobs. The authors propose "targeted distillation" through mission-focused instruction tuning and test it on open named entity recognition (NER).

They sampled 50K passages from the Pile and had ChatGPT label every entity it found along with a free-form type. The resulting data was used to fine-tune LLaMA into the UniversalNER (UniNER) models. Diversity comes from varied inputs and entity labels, not from varied instructions.

To evaluate, they assembled the UniversalNER benchmark, the largest NER collection so far. In zero-shot tests, UniNER beats general instruction-tuned models by a wide margin and outperforms ChatGPT itself, despite being far smaller. It also beats the supervised multi-task system InstructUIE. The recipe, data and models are publicly released, which makes this a practical basis for affordable, transparent entity extraction.

## Key points
- **Training data:**
  - Built from Pile passages of up to 256 tokens, labeled by gpt-3.5-turbo-0301 at temperature 0.
  - After filtering, it contains 45,889 examples, 240,725 entity mentions and 13,020 distinct entity types.
  - The type distribution is heavily skewed: the top 1% of types account for about 74% of all occurrences.
- **UniversalNER benchmark:** 43 datasets across 9 domains, spanning areas such as transportation, finance, legal text, social media posts, programming, STEM, clinical text and biomedicine. Examples include CrossNER, FewNERD, Ontonotes, i2b2 and Stackoverflow-NER.
- **Comparison with general models:** In zero-shot NER, LLaMA and Alpaca score near zero F1, and Vicuna trails ChatGPT by more than 20 points. UniNER exceeds Vicuna by over 30 average F1 points.
- **Comparison with ChatGPT and InstructUIE:** UniNER (e.g., UniNER-7B) beats ChatGPT by 7–9 average F1 points. It also clearly outperforms InstructUIE, which was trained on supervised NER data.
- **Ablations and variants:** Ablation studies examine instruction prompt design and negative sampling. A variant, UniNER-7B w/ sup, is additionally trained on supervised datasets.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Microsoft](https://md.research.elamur.ai/free/en/companies/microsoft/) — Company, Redmond, United States · 378 works · 161,630 citations
- [University of Southern California](https://md.research.elamur.ai/free/en/universities/university-of-southern-california/) — University, Los Angeles, United States · 175 works · 37,050 citations
- [University of California, Davis](https://md.research.elamur.ai/free/en/universities/university-of-california-davis/) — University, Davis, United States · 34 works · 42,940 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works · [👁️ Computer Vision and Pattern Recognition](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-vision-and-pattern-recognition/) · 3,486 works

🧭 Topics (3): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works · [🏷️ Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/) · 214 works · [🪶 Efficient models & compression](https://md.research.elamur.ai/free/en/topics/efficient-models/) · 297 works

## Cite

Cite as: Wenxuan Zhou, Sheng Zhang, Yu Gu, Muhao Chen and Hoifung Poon. 2023. UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition. arXiv:2308.03279.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{zhou2023universalner,
  title = {{UniversalNER: Targeted Distillation from Large Language Models for Open Named Entity Recognition}},
  author = {Wenxuan Zhou, Sheng Zhang, Yu Gu, Muhao Chen and Hoifung Poon},
  year = {2023},
  doi = {10.48550/arxiv.2308.03279},
  eprint = {2308.03279},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2308.03279}
}
```

- [← previous Knowledge of cultural moral norms in large language models](https://md.research.elamur.ai/free/en/papers/ramezani-2023-knowledge-cultural-moral-norms-large/)
- [next → XLM-V: Overcoming the Vocabulary Bottleneck in Multilingual Masked Language Models](https://md.research.elamur.ai/free/en/papers/liang-2023-xlm-v-overcoming-vocabulary-bottleneck/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/)

## References

75 references, 25 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rami Al-Rfou, Vivek Kulkarni, Bryan Perozzi, and Steven Skiena. Polyglot-ner: Massive multilingual named entity recognition. In Proceedings of the 2015 SIAM International Conference on Data Mining, pp. 586–594. [find](https://scholar.google.com/scholar?q=Rami+Al%2DRfou%2C+Vivek+Kulkarni%2C+Bryan+Perozzi%2C+and+Steven+Skiena%2E+Polyglot%2Dner%3A+Massive+multilingual+named+entity+recognition%2E+In+Proceedings+of+the+2015+SIAM+International+Conference+on+Data+Mining%2C+pp)
2. Victoria Arranz, Khalid Choukri, Montse Cuadros, Aitor García Pablos, Lucie Gianola, Cyril Grouin, Manuel Herranz, Patrick Paroubek, and Pierre Zweigenbaum. MAPA project: Ready-to-go open-source datasets and deep learning technology to remove identifying information from text documents. In Proceedin… [find](https://scholar.google.com/scholar?q=Victoria+Arranz%2C+Khalid+Choukri%2C+Montse+Cuadros%2C+Aitor+Garc%C3%ADa+Pablos%2C+Lucie+Gianola%2C+Cyril+Grouin%2C+Manuel+Herranz%2C+Patrick+Paroubek%2C+and+Pierre+Zweigenbaum%2E+MAPA+project%3A+Ready%2Dto%2Dgo+open%2Dsource+datas)
3. Ting Wai Terence Au, Vasileios Lampos, and Ingemar Cox. E-NER — an annotated named entity recognition corpus of legal text. In Nikolaos Aletras, Ilias Chalkidis, Leslie Barrett, Cătălina Goant , ă, and Daniel Preot , iuc-Pietro (eds.), Proceedings of the Natural Legal Language Processing Workshop… [find](https://scholar.google.com/scholar?q=Ting+Wai+Terence+Au%2C+Vasileios+Lampos%2C+and+Ingemar+Cox%2E+E%2DNER+%E2%80%94+an+annotated+named+entity+recognition+corpus+of+legal+text%2E+In+Nikolaos+Aletras%2C+Ilias+Chalkidis%2C+Leslie+Barrett%2C+Ca%CC%86ta%CC%86lina+Goant+%2C+a%CC%86%2C)
4. Pei Chen, Haotian Xu, Cheng Zhang, and Ruihong Huang. Crossroads, buildings and neighborhoods: A dataset for fine-grained location recognition. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 3329–… [find](https://scholar.google.com/scholar?q=Pei+Chen%2C+Haotian+Xu%2C+Cheng+Zhang%2C+and+Ruihong+Huang%2E+Crossroads%2C+buildings+and+neighborhoods%3A+A+dataset+for+fine%2Dgrained+location+recognition%2E+In+Proceedings+of+the+2022+Conference+of+the+North+Ameri)
5. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing gpt-4 with 90%* chatgpt quality, March 2023. URL [https://vicuna.lmsys.org](https://vicuna.lmsys.org). [↗](https://vicuna.lmsys.org)
6. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416, 2022. [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+Shayne+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+Eric+Li%2C+Xuezhi+Wang%2C+Mostafa+Dehghani%2C+Siddhartha+Brahma%2C+et+al%2E+Scaling+instruction%2Dfinetuned+language+models%2E+arXiv+prep)
7. Nigel Collier and Jin-Dong Kim. [Introduction to the bio-entity recognition task at jnlpba](https://md.research.elamur.ai/free/en/papers/collier-2004-introduction-bio-entity-recognition-task/). In Proceedings of the International Joint Workshop on Natural Language Processing in Biomedicine and its Applications (NLPBA/BioNLP), pp. 73–78, 2004.
8. Leon Derczynski, Kalina Bontcheva, and Ian Roberts. Broad Twitter corpus: A diverse named entity recognition resource. In Proceedings of COLING 2016, the 26th International Conference on Computational Linguistics: Technical Papers, pp. 1169–1179, Osaka, Japan, December 2016. [find](https://scholar.google.com/scholar?q=Leon+Derczynski%2C+Kalina+Bontcheva%2C+and+Ian+Roberts%2E+Broad+Twitter+corpus%3A+A+diverse+named+entity+recognition+resource%2E+In+Proceedings+of+COLING+2016%2C+the+26th+International+Conference+on+Computational)
9. The COLING 2016 Organizing Committee. URL https://aclanthology.org/C16-1111. [↗](https://aclanthology.org/C16-1111)
10. Ning Ding, Guangwei Xu, Yulin Chen, Xiaobin Wang, Xu Han, Pengjun Xie, Haitao Zheng, and Zhiyuan Liu. [Few-NERD: A few-shot named entity recognition dataset](https://md.research.elamur.ai/free/en/papers/ding-2021-few-nerd-few-shot-named/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natur…
11. Rezarta Islamaj Doğan, Robert Leaman, and Zhiyong Lu. Ncbi disease corpus: a resource for disease name recognition and concept normalization. Journal of biomedical informatics, 47:1–10, 2014. [find](https://scholar.google.com/scholar?q=Rezarta+Islamaj+Dog%CC%86an%2C+Robert+Leaman%2C+and+Zhiyong+Lu%2E+Ncbi+disease+corpus%3A+a+resource+for+disease+name+recognition+and+concept+normalization%2E+Journal+of+biomedical+informatics%2C+47%3A1%E2%80%9310%2C+2014%2E)
12. Oren Etzioni, Michele Banko, Stephen Soderland, and Daniel S Weld. [Open information extraction from the web](https://md.research.elamur.ai/free/en/papers/banko-2007-open-information-extraction-web/). Communications of the ACM, 51(12):68–74, 2008.
13. Annemarie Friedrich, Heike Adel, Federico Tomazic, Johannes Hingerl, Renou Benteau, Anika Maruscyk, and Lukas Lange. The sofc-exp corpus and neural approaches to information extraction in the materials science domain, 2020. [find](https://scholar.google.com/scholar?q=Annemarie+Friedrich%2C+Heike+Adel%2C+Federico+Tomazic%2C+Johannes+Hingerl%2C+Renou+Benteau%2C+Anika+Maruscyk%2C+and+Lukas+Lange%2E+The+sofc%2Dexp+corpus+and+neural+approaches+to+information+extraction+in+the+material)
14. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, et al. The pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027, 2020. [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Stella+Biderman%2C+Sid+Black%2C+Laurence+Golding%2C+Travis+Hoppe%2C+Charles+Foster%2C+Jason+Phang%2C+Horace+He%2C+Anish+Thite%2C+Noa+Nabeshima%2C+et+al%2E+The+pile%3A+An+800gb+dataset+of+diverse+text+for+language+)
15. Felix Grezes, Sergi Blanco-Cuaresma, Thomas Allen, and Tirthankar Ghosal. Overview of the first shared task on detecting entities in the astrophysics literature (DEAL). In Proceedings of the first Workshop on Information Extraction from Scientific Publications, pp. 1–7, Online, November 2022. Associ… [find](https://scholar.google.com/scholar?q=Felix+Grezes%2C+Sergi+Blanco%2DCuaresma%2C+Thomas+Allen%2C+and+Tirthankar+Ghosal%2E+Overview+of+the+first+shared+task+on+detecting+entities+in+the+astrophysics+literature+%28DEAL%29%2E+In+Proceedings+of+the+first+Wor)
16. Yu Gu, Sheng Zhang, Naoto Usuyama, Yonas Woldesenbet, Cliff Wong, Praneeth Sanapathi, Mu Wei, Naveen Valluri, Erika Strandberg, Tristan Naumann, and Hoifung Poon. Distilling large language models for biomedical knowledge extraction: A case study on adverse drug events, 2023. [find](https://scholar.google.com/scholar?q=Yu+Gu%2C+Sheng+Zhang%2C+Naoto+Usuyama%2C+Yonas+Woldesenbet%2C+Cliff+Wong%2C+Praneeth+Sanapathi%2C+Mu+Wei%2C+Naveen+Valluri%2C+Erika+Strandberg%2C+Tristan+Naumann%2C+and+Hoifung+Poon%2E+Distilling+large+language+models+for+)
17. Runwei Guan, Ka Lok Man, Feifan Chen, Shanliang Yao, Rongsheng Hu, Xiaohui Zhu, Jeremy Smith, Eng Gee Lim, and Yutao Yue. Findvehicle and vehiclefinder: A ner dataset for natural language-based vehicle retrieval and a keyword-based cross-modal vehicle retrieval system. arXiv preprint arXiv:2304.1089… [find](https://scholar.google.com/scholar?q=Runwei+Guan%2C+Ka+Lok+Man%2C+Feifan+Chen%2C+Shanliang+Yao%2C+Rongsheng+Hu%2C+Xiaohui+Zhu%2C+Jeremy+Smith%2C+Eng+Gee+Lim%2C+and+Yutao+Yue%2E+Findvehicle+and+vehiclefinder%3A+A+ner+dataset+for+natural+language%2Dbased+vehicl)
18. Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. [The false promise of imitating proprietary llms](https://md.research.elamur.ai/free/en/papers/gudibande-2023-false-promise-imitating-proprietary-llms/), 2023.
19. Sam Henry, Kevin Buchan, Michele Filannino, Amber Stubbs, and Ozlem Uzuner. 2018 n2c2 shared task on adverse drug events and medication extraction in electronic health records. Journal of the American Medical Informatics Association, 27(1):3–12, 2020. [find](https://scholar.google.com/scholar?q=Sam+Henry%2C+Kevin+Buchan%2C+Michele+Filannino%2C+Amber+Stubbs%2C+and+Ozlem+Uzuner%2E+2018+n2c2+shared+task+on+adverse+drug+events+and+medication+extraction+in+electronic+health+records%2E+Journal+of+the+American)
20. Geoffrey Hinton, Oriol Vinyals, and Jeff Dean. [Distilling the knowledge in a neural network](https://md.research.elamur.ai/free/en/papers/hinton-2015-distilling-knowledge-neural-network/). arXiv preprint arXiv:1503.02531, 2015.
21. Cheng-Yu Hsieh, Chun-Liang Li, Chih-kuan Yeh, Hootan Nakhost, Yasuhisa Fujii, Alex Ratner, Ranjay Krishna, Chen-Yu Lee, and Tomas Pfister. [Distilling step-by-step! outperforming larger language models with less training data and smaller model sizes](https://md.research.elamur.ai/free/en/papers/hsieh-2023-distilling-step-step-outperforming-larger/). In Findings of the Association for Computational L…
22. Yu-Ming Hsieh, Yueh-Yin Shih, and Wei-Yun Ma. Converting the Sinica Treebank of Mandarin Chinese to Universal Dependencies. In Proceedings of the 16th Linguistic Annotation Workshop (LAW-XVI) within LREC2022, pp. 23–30, Marseille, France, June 2022. European Language Resources Association. URL https… [find](https://scholar.google.com/scholar?q=Yu%2DMing+Hsieh%2C+Yueh%2DYin+Shih%2C+and+Wei%2DYun+Ma%2E+Converting+the+Sinica+Treebank+of+Mandarin+Chinese+to+Universal+Dependencies%2E+In+Proceedings+of+the+16th+Linguistic+Annotation+Workshop+%28LAW%2DXVI%29+within+L)
23. Sarthak Jain, Madeleine van Zuylen, Hannaneh Hajishirzi, and Iz Beltagy. [SciREX: A challenge dataset for document-level information extraction](https://md.research.elamur.ai/free/en/papers/jain-2020-scirex-challenge-dataset-document-level/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 7506–7516, Online, July 2020. Association for Computational…
24. Jaehun Jung, Peter West, Liwei Jiang, Faeze Brahman, Ximing Lu, Jillian Fisher, Taylor Sorensen, and Yejin Choi. [Impossible distillation: from low-quality model to high-quality dataset & model for summarization and paraphrasing](https://md.research.elamur.ai/free/en/papers/jung-2023-impossible-distillation-low-quality-model/), 2023.
25. J-D Kim, Tomoko Ohta, Yuka Tateisi, and Jun’ichi Tsujii. Genia corpus—a semantically annotated corpus for bio-textmining. Bioinformatics, 19(suppl_1):i180–i182, 2003. [find](https://scholar.google.com/scholar?q=J%2DD+Kim%2C+Tomoko+Ohta%2C+Yuka+Tateisi%2C+and+Jun%E2%80%99ichi+Tsujii%2E+Genia+corpus%E2%80%94a+semantically+annotated+corpus+for+bio%2Dtextmining%2E+Bioinformatics%2C+19%28suppl%5F1%29%3Ai180%E2%80%93i182%2C+2003%2E)
26. Martin Krallinger, Obdulia Rabal, Florian Leitner, Miguel Vazquez, David Salgado, Zhiyong Lu, Robert Leaman, Yanan Lu, Donghong Ji, Daniel M Lowe, et al. The chemdner corpus of chemicals and drugs and its annotation principles. Journal of cheminformatics, 7(1):1–17, 2015. [find](https://scholar.google.com/scholar?q=Martin+Krallinger%2C+Obdulia+Rabal%2C+Florian+Leitner%2C+Miguel+Vazquez%2C+David+Salgado%2C+Zhiyong+Lu%2C+Robert+Leaman%2C+Yanan+Lu%2C+Donghong+Ji%2C+Daniel+M+Lowe%2C+et+al%2E+The+chemdner+corpus+of+chemicals+and+drugs+and)
27. Chaitanya Kulkarni, Wei Xu, Alan Ritter, and Raghu Machiraju. An annotated corpus for machine reading of instructions in wet lab protocols. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 2 (Sho… [find](https://scholar.google.com/scholar?q=Chaitanya+Kulkarni%2C+Wei+Xu%2C+Alan+Ritter%2C+and+Raghu+Machiraju%2E+An+annotated+corpus+for+machine+reading+of+instructions+in+wet+lab+protocols%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+)
28. Aman Kumar and Binil Starly. “fabner”: information extraction from manufacturing process science domain literature using named entity recognition. Journal of Intelligent Manufacturing, 33(8): 2393–2407, 2022. [find](https://scholar.google.com/scholar?q=Aman+Kumar+and+Binil+Starly%2E+%E2%80%9Cfabner%E2%80%9D%3A+information+extraction+from+manufacturing+process+science+domain+literature+using+named+entity+recognition%2E+Journal+of+Intelligent+Manufacturing%2C+33%288%29%3A+2393%E2%80%93240)
29. Bo Li, Gexiang Fang, Yang Yang, Quansen Wang, Wei Ye, Wen Zhao, and Shikun Zhang. Evaluating chatgpt’s information extraction capabilities: An assessment of performance, explainability, calibration, and faithfulness. arXiv preprint arXiv:2304.[11633, 2023](https://md.research.elamur.ai/free/en/papers/li-2023-evaluating-chatgpt-s-information-extraction/).
30. Jiao Li, Yueping Sun, Robin J Johnson, Daniela Sciaky, Chih-Hsuan Wei, Robert Leaman, Allan Peter Davis, Carolyn J Mattingly, Thomas C Wiegers, and Zhiyong Lu. Biocreative v cdr task corpus: a resource for chemical disease relation extraction. Database, 2016, 2016. [find](https://scholar.google.com/scholar?q=Jiao+Li%2C+Yueping+Sun%2C+Robin+J+Johnson%2C+Daniela+Sciaky%2C+Chih%2DHsuan+Wei%2C+Robert+Leaman%2C+Allan+Peter+Davis%2C+Carolyn+J+Mattingly%2C+Thomas+C+Wiegers%2C+and+Zhiyong+Lu%2E+Biocreative+v+cdr+task+corpus%3A+a+resourc)
31. Jingjing Liu, Panupong Pasupat, Scott Cyphers, and Jim Glass. Asgard: A portable architecture for multilingual dialogue systems. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing, pp. 8386–8390. [find](https://scholar.google.com/scholar?q=Jingjing+Liu%2C+Panupong+Pasupat%2C+Scott+Cyphers%2C+and+Jim+Glass%2E+Asgard%3A+A+portable+architecture+for+multilingual+dialogue+systems%2E+In+2013+IEEE+International+Conference+on+Acoustics%2C+Speech+and+Signal+P)
32. Zihan Liu, Yan Xu, Tiezheng Yu, Wenliang Dai, Ziwei Ji, Samuel Cahyawijaya, Andrea Madotto, and Pascale Fung. Crossner: Evaluating cross-domain named entity recognition. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 35, pp. 13452–13460, 2021. [find](https://scholar.google.com/scholar?q=Zihan+Liu%2C+Yan+Xu%2C+Tiezheng+Yu%2C+Wenliang+Dai%2C+Ziwei+Ji%2C+Samuel+Cahyawijaya%2C+Andrea+Madotto%2C+and+Pascale+Fung%2E+Crossner%3A+Evaluating+cross%2Ddomain+named+entity+recognition%2E+In+Proceedings+of+the+AAAI+Con)
33. Yi Luan, Luheng He, Mari Ostendorf, and Hannaneh Hajishirzi. [Multi-task identification of entities, relations, and coreference for scientific knowledge graph construction](https://md.research.elamur.ai/free/en/papers/luan-2018-multi-task-identification-entities-relations/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 3219–3232, Brussels, Belgium, Oct…
34. Ling Luo, Po-Ting Lai, Chih-Hsuan Wei, Cecilia N Arighi, and Zhiyong Lu. Biored: a rich biomedical relation extraction dataset. Briefings in Bioinformatics, 23(5):bbac282, 2022. [find](https://scholar.google.com/scholar?q=Ling+Luo%2C+Po%2DTing+Lai%2C+Chih%2DHsuan+Wei%2C+Cecilia+N+Arighi%2C+and+Zhiyong+Lu%2E+Biored%3A+a+rich+biomedical+relation+extraction+dataset%2E+Briefings+in+Bioinformatics%2C+23%285%29%3Abbac282%2C+2022%2E)
35. Alexis Mitchell, Stephanie Strassel, Shudong Huang, and Ramez Zakhary. Ace 2004 multilingual training corpus. Linguistic Data Consortium, Philadelphia, 1:1–1, 2005. [find](https://scholar.google.com/scholar?q=Alexis+Mitchell%2C+Stephanie+Strassel%2C+Shudong+Huang%2C+and+Ramez+Zakhary%2E+Ace+2004+multilingual+training+corpus%2E+Linguistic+Data+Consortium%2C+Philadelphia%2C+1%3A1%E2%80%931%2C+2005%2E)
36. Danielle L Mowery, Sumithra Velupillai, Brett R South, Lee Christensen, David Martinez, Liadh Kelly, Lorraine Goeuriot, Noemie Elhadad, Sameer Pradhan, Guergana Savova, et al. Task 2: Share/clef ehealth evaluation lab 2014. In Proceedings of CLEF 2014, 2014. [find](https://scholar.google.com/scholar?q=Danielle+L+Mowery%2C+Sumithra+Velupillai%2C+Brett+R+South%2C+Lee+Christensen%2C+David+Martinez%2C+Liadh+Kelly%2C+Lorraine+Goeuriot%2C+Noemie+Elhadad%2C+Sameer+Pradhan%2C+Guergana+Savova%2C+et+al%2E+Task+2%3A+Share%2Fclef+eheal)
37. Benjamin Nye, Junyi Jessy Li, Roma Patel, Yinfei Yang, Iain J Marshall, Ani Nenkova, and Byron C Wallace. [A corpus with multi-level annotations of patients, interventions and outcomes to support language processing for medical literature](https://md.research.elamur.ai/free/en/papers/nye-2018-corpus-multi-level-annotations-patients/). In Proceedings of the conference. Association for Computation…
38. NIH Public Access, 2018. [find](https://scholar.google.com/scholar?q=NIH+Public+Access%2C+2018%2E)
39. OpenAI. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/), 2023.
40. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Gray, et al. [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/). In Advances in Neural Information Processing Systems, 2022.
41. Xiaoman Pan, Boliang Zhang, Jonathan May, Joel Nothman, Kevin Knight, and Heng Ji. [Cross-lingual name tagging and linking for 282 languages](https://md.research.elamur.ai/free/en/papers/pan-2017-cross-lingual-name-tagging-linking/). In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1946–1958, Vancouver, Canada, July 2017…
42. Baolin Peng, Chunyuan Li, Pengcheng He, Michel Galley, and Jianfeng Gao. [Instruction tuning with gpt-4](https://md.research.elamur.ai/free/en/papers/peng-2023-instruction-tuning-gpt-4/). arXiv preprint arXiv:2304.03277, 2023.
43. Nadeesha Perera, Matthias Dehmer, and Frank Emmert-Streib. Named entity recognition and relation detection for biomedical information extraction. Frontiers in cell and developmental biology, pp. 673, 2020. [find](https://scholar.google.com/scholar?q=Nadeesha+Perera%2C+Matthias+Dehmer%2C+and+Frank+Emmert%2DStreib%2E+Named+entity+recognition+and+relation+detection+for+biomedical+information+extraction%2E+Frontiers+in+cell+and+developmental+biology%2C+pp%2E+673%2C+)
44. Sampo Pyysalo and Sophia Ananiadou. Anatomical entity mention recognition at literature scale. Bioinformatics, 30(6):868–875, 2014. [find](https://scholar.google.com/scholar?q=Sampo+Pyysalo+and+Sophia+Ananiadou%2E+Anatomical+entity+mention+recognition+at+literature+scale%2E+Bioinformatics%2C+30%286%29%3A868%E2%80%93875%2C+2014%2E)
45. Victor Sanh, Albert Webson, Colin Raffel, Stephen H. Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, Manan Dey, M Saiful Bari, Canwen Xu, Urmish Thakker, Shanya Sharma Sharma, Eliza Szczechla, Taewoon Kim, Gunjan Chhablani, Nihal Nayak, Debajyoti Da… — [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)
46. David Schindler, Felix Bensmann, Stefan Dietze, and Frank Krüger. Somesci-a 5 star open data gold standard knowledge graph of software mentions in scientific articles. In Proceedings of the 30th ACM International Conference on Information & Knowledge Management, pp. 4574–4583, 2021. [find](https://scholar.google.com/scholar?q=David+Schindler%2C+Felix+Bensmann%2C+Stefan+Dietze%2C+and+Frank+Kr%C3%BCger%2E+Somesci%2Da+5+star+open+data+gold+standard+knowledge+graph+of+software+mentions+in+scientific+articles%2E+In+Proceedings+of+the+30th+ACM+I)
47. Isabel Segura-Bedmar, Paloma Martínez Fernández, and María Herrero Zazo. Semeval-2013 [task 9: Extraction of drug-drug interactions from biomedical texts (ddiextraction 2013)](https://md.research.elamur.ai/free/en/papers/bedmar-2013-semeval-2013-task-9-extraction/). Association for Computational Linguistics, 2013.
48. Agam Shah, Ruchit Vithani, Abhinav Gullapalli, and Sudheer Chava. Finer: Financial named entity recognition dataset and weak-supervision model. arXiv preprint arXiv:2302.11157, 2023. — [arXiv:2302.11157](https://arxiv.org/abs/2302.11157) [find](https://scholar.google.com/scholar?q=Agam+Shah%2C+Ruchit+Vithani%2C+Abhinav+Gullapalli%2C+and+Sudheer+Chava%2E+Finer%3A+Financial+named+entity+recognition+dataset+and+weak%2Dsupervision+model%2E+arXiv+preprint+arXiv%3A2302%2E11157%2C+2023%2E+%E2%80%94+%5BarXiv%3A2302%2E111)
49. Larry Smith, Lorraine K Tanabe, Cheng-Ju Kuo, I Chung, Chun-Nan Hsu, Yu-Shi Lin, Roman Klinger, Christoph M Friedrich, Kuzman Ganchev, Manabu Torii, et al. Overview of biocreative ii gene mention recognition. Genome biology, 9(2):1–19, 2008. [find](https://scholar.google.com/scholar?q=Larry+Smith%2C+Lorraine+K+Tanabe%2C+Cheng%2DJu+Kuo%2C+I+Chung%2C+Chun%2DNan+Hsu%2C+Yu%2DShi+Lin%2C+Roman+Klinger%2C+Christoph+M+Friedrich%2C+Kuzman+Ganchev%2C+Manabu+Torii%2C+et+al%2E+Overview+of+biocreative+ii+gene+mention+reco)
50. Ryan Smith, Jason A Fries, Braden Hancock, and Stephen H Bach. Language models in the loop: Incorporating prompting into weak supervision. arXiv preprint arXiv:2205.02318, 2022. — [arXiv:2205.02318](https://arxiv.org/abs/2205.02318) [find](https://scholar.google.com/scholar?q=Ryan+Smith%2C+Jason+A+Fries%2C+Braden+Hancock%2C+and+Stephen+H+Bach%2E+Language+models+in+the+loop%3A+Incorporating+prompting+into+weak+supervision%2E+arXiv+preprint+arXiv%3A2205%2E02318%2C+2022%2E+%E2%80%94+%5BarXiv%3A2205%2E02318%5D%28h)
51. Amber Stubbs, Christopher Kotfila, and Özlem Uzuner. Automated systems for the de-identification of longitudinal clinical narratives: Overview of 2014 i2b2/uthealth shared task track 1. Journal of biomedical informatics, 58:S11–S19, 2015. [find](https://scholar.google.com/scholar?q=Amber+Stubbs%2C+Christopher+Kotfila%2C+and+%C3%96zlem+Uzuner%2E+Automated+systems+for+the+de%2Didentification+of+longitudinal+clinical+narratives%3A+Overview+of+2014+i2b2%2Futhealth+shared+task+track+1%2E+Journal+of+bio)
52. Weiyi Sun, Anna Rumshisky, and Ozlem Uzuner. Evaluating temporal relations in clinical text: 2012 i2b2 challenge. Journal of the American Medical Informatics Association, 20(5):806–813, 2013. [find](https://scholar.google.com/scholar?q=Weiyi+Sun%2C+Anna+Rumshisky%2C+and+Ozlem+Uzuner%2E+Evaluating+temporal+relations+in+clinical+text%3A+2012+i2b2+challenge%2E+Journal+of+the+American+Medical+Informatics+Association%2C+20%285%29%3A806%E2%80%93813%2C+2013%2E)
53. Xiaofei Sun, Linfeng Dong, Xiaoya Li, Zhen Wan, Shuhe Wang, Tianwei Zhang, Jiwei Li, Fei Cheng, Lingjuan Lyu, Fei Wu, and Guoyin Wang. [Pushing the limits of chatgpt on nlp tasks](https://md.research.elamur.ai/free/en/papers/sun-2023-pushing-limits-chatgpt-nlp-tasks/), 2023.
54. Jeniya Tabassum, Mounica Maddela, Wei Xu, and Alan Ritter. Code and named entity recognition in StackOverflow. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 4913–4926, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/202… [find](https://scholar.google.com/scholar?q=Jeniya+Tabassum%2C+Mounica+Maddela%2C+Wei+Xu%2C+and+Alan+Ritter%2E+Code+and+named+entity+recognition+in+StackOverflow%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for+Computational+Linguistic)
55. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford alpaca: An instruction-following llama model. [https://github.com/tatsu-lab/stanford_alpaca](https://github.com/tatsu-lab/stanford_alpaca), 2023. [↗](https://github.com/tatsu-lab/stanford_alpaca)
56. Simone Tedeschi and Roberto Navigli. MultiNERD: A multilingual, multi-genre and fine-grained dataset for named entity recognition (and disambiguation). In Findings of the Association for Computational Linguistics: NAACL 2022, pp. 801–812, Seattle, United States, July 2022. Association for Computatio… [find](https://scholar.google.com/scholar?q=Simone+Tedeschi+and+Roberto+Navigli%2E+MultiNERD%3A+A+multilingual%2C+multi%2Dgenre+and+fine%2Dgrained+dataset+for+named+entity+recognition+%28and+disambiguation%29%2E+In+Findings+of+the+Association+for+Computational)
57. Simone Tedeschi, Valentino Maiorca, Niccolò Campolungo, Francesco Cecconi, and Roberto Navigli. WikiNEuRal: Combined neural and knowledge-based silver data creation for multilingual NER. In Findings of the Association for Computational Linguistics: EMNLP 2021, pp. 2521–2533, Punta Cana, Dominican Re… [find](https://scholar.google.com/scholar?q=Simone+Tedeschi%2C+Valentino+Maiorca%2C+Niccol%C3%B2+Campolungo%2C+Francesco+Cecconi%2C+and+Roberto+Navigli%2E+WikiNEuRal%3A+Combined+neural+and+knowledge%2Dbased+silver+data+creation+for+multilingual+NER%2E+In+Findings+o)
58. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/), 2023a.
59. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
60. Asahi Ushio, Leonardo Neves, Vitor Silva, Francesco. Barbieri, and Jose Camacho-Collados. Named Entity Recognition in Twitter: A Dataset and Analysis on Short-Term Temporal Shifts. In The 2nd Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 12th Interna… [find](https://scholar.google.com/scholar?q=Asahi+Ushio%2C+Leonardo+Neves%2C+Vitor+Silva%2C+Francesco%2E+Barbieri%2C+and+Jose+Camacho%2DCollados%2E+Named+Entity+Recognition+in+Twitter%3A+A+Dataset+and+Analysis+on+Short%2DTerm+Temporal+Shifts%2E+In+The+2nd+Conferen)
61. Özlem Uzuner, Yuan Luo, and Peter Szolovits. Evaluating the state-of-the-art in automatic de-identification. Journal of the American Medical Informatics Association, 14(5):550–563, 2007. [find](https://scholar.google.com/scholar?q=%C3%96zlem+Uzuner%2C+Yuan+Luo%2C+and+Peter+Szolovits%2E+Evaluating+the+state%2Dof%2Dthe%2Dart+in+automatic+de%2Didentification%2E+Journal+of+the+American+Medical+Informatics+Association%2C+14%285%29%3A550%E2%80%93563%2C+2007%2E)
62. Özlem Uzuner, Brett R South, Shuying Shen, and Scott L DuVall. 2010 i2b2/va challenge on concepts, assertions, and relations in clinical text. Journal of the American Medical Informatics Association, 18(5):552–556, 2011. [find](https://scholar.google.com/scholar?q=%C3%96zlem+Uzuner%2C+Brett+R+South%2C+Shuying+Shen%2C+and+Scott+L+DuVall%2E+2010+i2b2%2Fva+challenge+on+concepts%2C+assertions%2C+and+relations+in+clinical+text%2E+Journal+of+the+American+Medical+Informatics+Association%2C+)
63. Christopher Walker, Stephanie Strassel, Julie Medero, and Kazuaki Maeda. Ace 2005 multilingual training corpus. Linguistic Data Consortium, Philadelphia, 57:45, 2006. [find](https://scholar.google.com/scholar?q=Christopher+Walker%2C+Stephanie+Strassel%2C+Julie+Medero%2C+and+Kazuaki+Maeda%2E+Ace+2005+multilingual+training+corpus%2E+Linguistic+Data+Consortium%2C+Philadelphia%2C+57%3A45%2C+2006%2E)
64. Xiao Wang, Weikang Zhou, Can Zu, Han Xia, Tianze Chen, Yuansen Zhang, Rui Zheng, Junjie Ye, Qi Zhang, Tao Gui, Jihua Kang, Jingsheng Yang, Siyuan Li, and Chunsai Du. [Instructuie: Multi-task instruction tuning for unified information extraction](https://md.research.elamur.ai/free/en/papers/wang-2023-instructuie-multi-task-instruction-tuning/), 2023a.
65. Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Atharva Naik, Arjun Ashok, Arut Selvan Dhanasekaran, Anjana Arunkumar, David Stap, Eshaan Pathak, Giannis Karamanolakis, Haizhi Lai, Ishan Purohit, Ishani Mondal, Jacob Anderson, Kirby Kuznia, Krima Doshi, Kuntal… [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Swaroop+Mishra%2C+Pegah+Alipoormolabashi%2C+Yeganeh+Kordi%2C+Amirreza+Mirzaei%2C+Atharva+Naik%2C+Arjun+Ashok%2C+Arut+Selvan+Dhanasekaran%2C+Anjana+Arunkumar%2C+David+Stap%2C+Eshaan+Pathak%2C+Giannis+Karaman)
66. Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. [How far can camels go? exploring the state of instruction tuning on open resources](https://md.research.elamur.ai/free/en/papers/wang-2023-far-camels-go-exploring-state/), 2023b.
67. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-instruct: Aligning language models with self-generated instructions. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Asso… [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A%2E+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+Self%2Dinstruct%3A+Aligning+language+models+with+self%2Dgenerated+instructions%2E+In+Anna+Roger)
68. Zihan Wang, Jingbo Shang, Liyuan Liu, Lihao Lu, Jiacheng Liu, and Jiawei Han. Crossweigh: Training named entity tagger from imperfect annotations. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language… [find](https://scholar.google.com/scholar?q=Zihan+Wang%2C+Jingbo+Shang%2C+Liyuan+Liu%2C+Lihao+Lu%2C+Jiacheng+Liu%2C+and+Jiawei+Han%2E+Crossweigh%3A+Training+named+entity+tagger+from+imperfect+annotations%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Me)
69. Jason Wei, Maarten Bosma, Vincent Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. [Finetuned language models are zero-shot learners](https://md.research.elamur.ai/free/en/papers/wei-2021-finetuned-language-models-zero-shot/). In International Conference on Learning Representations, 2021.
70. Xiang Wei, Xingyu Cui, Ning Cheng, Xiaobin Wang, Xin Zhang, Shen Huang, Pengjun Xie, Jinan Xu, Yufeng Chen, Meishan Zhang, et al. Zero-shot information extraction via chatting with chatgpt. arXiv preprint arXiv:2302.[10205, 2023](https://md.research.elamur.ai/free/en/papers/wei-2023-chatie-zero-shot-information-extraction/).
71. Ralph Weischedel, Martha Palmer, Mitchell Marcus, Eduard Hovy, Sameer Pradhan, Lance Ramshaw, Nianwen Xue, Ann Taylor, Jeff Kaufman, Michelle Franchini, et al. Ontonotes release 5.0 ldc2013t19. Linguistic Data Consortium, Philadelphia, PA, 23:170, 2013. [find](https://scholar.google.com/scholar?q=Ralph+Weischedel%2C+Martha+Palmer%2C+Mitchell+Marcus%2C+Eduard+Hovy%2C+Sameer+Pradhan%2C+Lance+Ramshaw%2C+Nianwen+Xue%2C+Ann+Taylor%2C+Jeff+Kaufman%2C+Michelle+Franchini%2C+et+al%2E+Ontonotes+release+5%2E0+ldc2013t19%2E+Lingui)
72. Ania Wróblewska, Agnieszka Kaliska, Maciej Pawłowski, Dawid Wiśniewski, Witold Sosnowski, and Agnieszka Ławrynowicz. Tasteset–recipe dataset and food entities recognition benchmark. arXiv preprint arXiv:2204.07775, 2022. — [arXiv:2204.07775](https://arxiv.org/abs/2204.07775) [find](https://scholar.google.com/scholar?q=Ania+Wr%C3%B3blewska%2C+Agnieszka+Kaliska%2C+Maciej+Paw%C5%82owski%2C+Dawid+Wis%CC%81niewski%2C+Witold+Sosnowski%2C+and+Agnieszka+%C5%81awrynowicz%2E+Tasteset%E2%80%93recipe+dataset+and+food+entities+recognition+benchmark%2E+arXiv+preprint+ar)
73. Yonghui Wu, Min Jiang, Jun Xu, Degui Zhi, and Hua Xu. Clinical named entity recognition using deep learning models. In AMIA annual symposium proceedings, volume 2017, pp. 1812. [find](https://scholar.google.com/scholar?q=Yonghui+Wu%2C+Min+Jiang%2C+Jun+Xu%2C+Degui+Zhi%2C+and+Hua+Xu%2E+Clinical+named+entity+recognition+using+deep+learning+models%2E+In+AMIA+annual+symposium+proceedings%2C+volume+2017%2C+pp%2E+1812%2E)
74. American Medical Informatics Association, 2017. [find](https://scholar.google.com/scholar?q=American+Medical+Informatics+Association%2C+2017%2E)
75. Junjie Ye, Xuanting Chen, Nuo Xu, Can Zu, Zekai Shao, Shichun Liu, Yuhan Cui, Zeyang Zhou, Chao Gong, Yang Shen, et al. [A comprehensive capability analysis of gpt-3 and gpt-3.5 series models](https://md.research.elamur.ai/free/en/papers/ye-2023-comprehensive-capability-analysis-gpt-3/). arXiv preprint arXiv:2303.10420, 2023.

*Updated 2 November 2026*