---
title: "A Systematic Study and Comprehensive — Laskar 2023"
description: "When ChatGPT appeared, nobody had tested it systematically on standard academic NLP benchmarks."
authors: ["Md Tahmid Rahman Laskar", "M Saiful Bari", "Mizanur Rahman", "Md Amran Hossen Bhuiyan", "Shafiq Joty", "Jimmy Xiangji Huang"]
year: 2023
venue: "Findings of the Association for Computational Linguistics: ACL 2023"
source: "https://aclanthology.org/2023.findings-acl.29/"
doi: "10.18653/v1/2023.findings-acl.29"
arxiv: "2305.18486"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/laskar-2023-systematic-study-comprehensive-evaluation-chatgpt/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/laskar-2023-systematic-study-comprehensive-evaluation-chatgpt/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/) / A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets

# A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets

**Authors:** Md Tahmid Rahman Laskar, [M Saiful Bari](https://md.research.elamur.ai/free/en/authors/m-saiful-bari/), Mizanur Rahman, Md Amran Hossen Bhuiyan, [Shafiq Joty](https://md.research.elamur.ai/free/en/authors/shafiq-joty/), Jimmy Xiangji Huang · **Year:** 2023 · **Venue:** Findings of the Association for Computational Linguistics: ACL 2023 · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.findings-acl.29/) · [PDF](https://aclanthology.org/2023.findings-acl.29.pdf) · [DOI 10.18653/v1/2023.findings-acl.29](https://doi.org/10.18653/v1/2023.findings-acl.29) · [arXiv 2305.18486](https://arxiv.org/abs/2305.18486) · [OpenAlex](https://openalex.org/W4385570371)

**Citations:** 91 citations (openalex-snapshot · 2026-10-14) · [107](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
When ChatGPT appeared, nobody had tested it systematically on standard academic NLP benchmarks. Its free-form conversational answers are hard to score automatically against gold labels. The authors ran a broad zero-shot evaluation covering 140 benchmark datasets and about 255K generated responses. The tasks included question answering, summarization, code generation, commonsense and mathematical reasoning, machine translation, bias, and ethics. Responses came first from the web interface and later from the gpt-3.5-turbo API. Classification-style outputs were checked by at least two human annotators, with majority voting, or by parsing scripts whose failures went to humans. Generative tasks were scored with ROUGE or BLEU.

ChatGPT is versatile and strong at open-domain knowledge, math, and coding, but it generally trails fine-tuned single-task SOTA models. It is also weak on several commonsense tasks and on underrepresented languages. The study also identifies a new ability, answering several unrelated questions in one prompt. This matters because it shows where ChatGPT-like models can be deployed reliably and argues that chat models need human-assisted evaluation.

## Key points
- Five leaderboards were covered: SuperGLUE, Big-Bench Hard, MMLU, the Ethics Benchmark, and Inverse Scaling tasks.
- **SuperGLUE (dev split):**
  - Zero-shot ChatGPT reached 87.0 on RTE, beating 1-shot PaLM-540B (78.7) and PaLM 2-L (79.3).
  - It outperformed zero-shot PaLM-540B on 5 of 8 datasets.
  - It scored 100 parity on AX-g and an MCC of 56.7 on AX-b.
- **Multi-query ability:** the authors call it "PolyQuery Synthesis." It appears mainly in instruction-tuned models (ChatGPT and the davinci-001/002/003 series), though accuracy usually drops slightly compared with single-question prompts.
- **Commonsense and summarization:**
  - On PIQA, SIQA, HellaSwag, and WinoGrande, ChatGPT lags PaLM 540B and LLaMA 65B.
  - Its summaries get lower ROUGE than SOTA models, yet human annotators preferred them, suggesting current metrics fall short.
- **Released resources:** prompts and responses are available at github.com/ntunlp/ChatGPT_Eval.
- **Caveats:** results may change between ChatGPT versions, and benchmark data may have leaked into OpenAI's undisclosed instruction-tuning data.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

107 papers of our library cite it (of the 140884 whose bibliographies we read); 9 of them have a page here, 98 not yet.

- [Étude des facteurs de complexité des modèles de langage dans une tâche de compréhension de lecture à l'aide d'une expérience contrôlée sémantiquement](https://md.research.elamur.ai/free/en/papers/antoine-2024-etude-des-facteurs-de-complexite/)
- [A Multitask, Multilingual, Multimodal Evaluation of ChatGPT on Reasoning, Hallucination, and Interactivity](https://md.research.elamur.ai/free/en/papers/bang-2023-multitask-multilingual-multimodal-evaluation-chatgpt/)
- [ALLaM: Large Language Models for Arabic and English](https://md.research.elamur.ai/free/en/papers/bari-2024-allam-large-language-models-arabic/)
- [A Survey on Evaluation of Large Language Models](https://md.research.elamur.ai/free/en/papers/chang-2023-survey-evaluation-large-language-models/)
- [TrustLLM: Trustworthiness in Large Language Models](https://md.research.elamur.ai/free/en/papers/huang-2024-trustllm-trustworthiness-large-language-models/)
- [TARJAMAT: Evaluation of Bard and ChatGPT on Machine Translation of Ten Arabic Varieties](https://md.research.elamur.ai/free/en/papers/kadaoui-2023-tarjamat-evaluation-bard-chatgpt-machine/)
- [New Trends for Modern Machine Translation with Large Reasoning Models](https://md.research.elamur.ai/free/en/papers/liu-2026-new-trends-modern-machine-translation/)
- [Orca 2: Teaching Small Language Models How to Reason](https://md.research.elamur.ai/free/en/papers/mitra-2023-orca-2-teaching-small-language/)
- [Who Generates More Empathetic Responses–-Humans or LLMs? A Comparative Evaluation with Human and LLM Judges](https://md.research.elamur.ai/free/en/papers/welivita-2026-who-generates-more-empathetic-responses/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{laskar2023systematic,
  title = {{A Systematic Study and Comprehensive Evaluation of ChatGPT on Benchmark Datasets}},
  author = {Laskar, Md Tahmid Rahman and Bari, M Saiful and Rahman, Mizanur and Bhuiyan, Md Amran Hossen and Joty, Shafiq and Huang, Jimmy Xiangji},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2023},
  year = {2023},
  doi = {10.18653/v1/2023.findings-acl.29},
  eprint = {2305.18486},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.18653/v1/2023.findings-acl.29}
}
```

- [← previous A Survey on Multimodal Large Language Models](https://md.research.elamur.ai/free/en/papers/yin-2023-survey-multimodal-large-language-models/)
- [next → A Transformer-based parser for Syriac morphology](https://md.research.elamur.ai/free/en/papers/aclbulk-2023-naaijer-transformer-based-parser-syriac-morphology/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/)

## References

195 references, 73 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Daniel Adiwardana, Minh-Thang Luong, David R So, Jamie Hall, Noah Fiedel, Romal Thoppilan, Zi Yang, Apoorv Kulshreshtha, Gaurav Nemade, Yifeng Lu, et al. 2020. Towards a human-like open-domain chatbot. arXiv preprint arXiv:2001.09977. [find](https://scholar.google.com/scholar?q=Daniel+Adiwardana%2C+Minh%2DThang+Luong%2C+David+R+So%2C+Jamie+Hall%2C+Noah+Fiedel%2C+Romal+Thoppilan%2C+Zi+Yang%2C+Apoorv+Kulshreshtha%2C+Gaurav+Nemade%2C+Yifeng+Lu%2C+et+al%2E+2020%2E+Towards+a+human%2Dlike+open%2Ddomain+chatbot) [our card](https://md.research.elamur.ai/free/en/papers/adiwardana-2020-towards-human-like-open-domain/)
2. Kabir Ahuja, Rishav Hada, Millicent Ochieng, Prachi Jain, Harshita Diddee, Samuel Maina, Tanuja Ganu, Sameer Segal, Maxamed Axmed, Kalika Bali, and Sunayana Sitaram. 2023. Mega: Multilingual evaluation of generative ai. [find](https://scholar.google.com/scholar?q=Kabir+Ahuja%2C+Rishav+Hada%2C+Millicent+Ochieng%2C+Prachi+Jain%2C+Harshita+Diddee%2C+Samuel+Maina%2C+Tanuja+Ganu%2C+Sameer+Segal%2C+Maxamed+Axmed%2C+Kalika+Bali%2C+and+Sunayana+Sitaram%2E+2023%2E+Mega%3A+Multilingual+evaluatio) [our card](https://md.research.elamur.ai/free/en/papers/ahuja-2023-mega-multilingual-evaluation-generative-ai/)
3. Rachith Aiyappa, Jisun An, Haewoon Kwak, and Yong-Yeol Ahn. 2023. Can we trust the evaluation on chatgpt? [find](https://scholar.google.com/scholar?q=Rachith+Aiyappa%2C+Jisun+An%2C+Haewoon+Kwak%2C+and+Yong%2DYeol+Ahn%2E+2023%2E+Can+we+trust+the+evaluation+on+chatgpt%3F)
4. Emily Alsentzer, John R Murphy, Willie Boag, Wei-Hung Weng, Di Jin, Tristan Naumann, and Matthew McDermott. 2019. Publicly available clinical bert embeddings. arXiv preprint arXiv:1904.03323. [find](https://scholar.google.com/scholar?q=Emily+Alsentzer%2C+John+R+Murphy%2C+Willie+Boag%2C+Wei%2DHung+Weng%2C+Di+Jin%2C+Tristan+Naumann%2C+and+Matthew+McDermott%2E+2019%2E+Publicly+available+clinical+bert+embeddings%2E+arXiv+preprint+arXiv%3A1904%2E03323%2E) [our card](https://md.research.elamur.ai/free/en/papers/alsentzer-2019-publicly-available-clinical-bert-embeddings/)
5. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. 2021. Program synthesis with large language models. arXiv preprint arXiv:2108.07732. [find](https://scholar.google.com/scholar?q=Jacob+Austin%2C+Augustus+Odena%2C+Maxwell+Nye%2C+Maarten+Bosma%2C+Henryk+Michalewski%2C+David+Dohan%2C+Ellen+Jiang%2C+Carrie+Cai%2C+Michael+Terry%2C+Quoc+Le%2C+et+al%2E+2021%2E+Program+synthesis+with+large+language+models%2E+a) [our card](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/)
6. Tom Ayoola, Shubhi Tyagi, Joseph Fisher, Christos Christodoulopoulos, and Andrea Pierleoni. 2022. Refined: An efficient zero-shot-capable approach to end-to-end entity linking. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human… [find](https://scholar.google.com/scholar?q=Tom+Ayoola%2C+Shubhi+Tyagi%2C+Joseph+Fisher%2C+Christos+Christodoulopoulos%2C+and+Andrea+Pierleoni%2E+2022%2E+Refined%3A+An+efficient+zero%2Dshot%2Dcapable+approach+to+end%2Dto%2Dend+entity+linking%2E+In+Proceedings+of+the+2)
7. Stephen H Bach, Victor Sanh, Zheng-Xin Yong, Albert Webson, Colin Raffel, Nihal V Nayak, Abheesht Sharma, Taewoon Kim, M Saiful Bari, Thibault Fevry, et al. 2022. Promptsource: An integrated development environment and repository for natural language prompts. arXiv preprint arXiv:2202.01279. [find](https://scholar.google.com/scholar?q=Stephen+H+Bach%2C+Victor+Sanh%2C+Zheng%2DXin+Yong%2C+Albert+Webson%2C+Colin+Raffel%2C+Nihal+V+Nayak%2C+Abheesht+Sharma%2C+Taewoon+Kim%2C+M+Saiful+Bari%2C+Thibault+Fevry%2C+et+al%2E+2022%2E+Promptsource%3A+An+integrated+developme) [our card](https://md.research.elamur.ai/free/en/papers/bach-2022-promptsource-integrated-development-environment-repository/)
8. Aram Bahrini, Mohammadsadra Khamoshifar, Hossein Abbasimehr, Robert J. Riggs, Maryam Esmaeili, Rastin Mastali Majdabadkohne, and Morteza Pase-hvar. 2023. Chatgpt: Applications, opportunities, and threats. [find](https://scholar.google.com/scholar?q=Aram+Bahrini%2C+Mohammadsadra+Khamoshifar%2C+Hossein+Abbasimehr%2C+Robert+J%2E+Riggs%2C+Maryam+Esmaeili%2C+Rastin+Mastali+Majdabadkohne%2C+and+Morteza+Pase%2Dhvar%2E+2023%2E+Chatgpt%3A+Applications%2C+opportunities%2C+and+thre)
9. Yejin Bang, Samuel Cahyawijaya, Nayeon Lee, Wenliang Dai, Dan Su, Bryan Wilie, Holy Lovenia, Ziwei Ji, Tiezheng Yu, Willy Chung, Quyet V. Do, Yan Xu, and Pascale Fung. 2023. A multitask, multilingual, multimodal evaluation of chatgpt on reasoning, hallucination, and interactivity. [find](https://scholar.google.com/scholar?q=Yejin+Bang%2C+Samuel+Cahyawijaya%2C+Nayeon+Lee%2C+Wenliang+Dai%2C+Dan+Su%2C+Bryan+Wilie%2C+Holy+Lovenia%2C+Ziwei+Ji%2C+Tiezheng+Yu%2C+Willy+Chung%2C+Quyet+V%2E+Do%2C+Yan+Xu%2C+and+Pascale+Fung%2E+2023%2E+A+multitask%2C+multilingual%2C) [our card](https://md.research.elamur.ai/free/en/papers/bang-2023-multitask-multilingual-multimodal-evaluation-chatgpt/)
10. Roy Bar Haim, Ido Dagan, Bill Dolan, Lisa Ferro, Danilo Giampiccolo, Bernardo Magnini, and Idan Szpektor. 2006. The second PASCAL recognising textual entailment challenge. [find](https://scholar.google.com/scholar?q=Roy+Bar+Haim%2C+Ido+Dagan%2C+Bill+Dolan%2C+Lisa+Ferro%2C+Danilo+Giampiccolo%2C+Bernardo+Magnini%2C+and+Idan+Szpektor%2E+2006%2E+The+second+PASCAL+recognising+textual+entailment+challenge%2E)
11. Loïc Barrault, Ondřej Bojar, Marta R Costa-jussà, Christian Federmann, Mark Fishel, Yvette Graham, Barry Haddow, Matthias Huck, Philipp Koehn, Shervin Malmasi, et al. 2019. Findings of the 2019 conference on machine translation (wmt19). In Proceedings of the Fourth Conference on Machine Translation… [find](https://scholar.google.com/scholar?q=Lo%C3%AFc+Barrault%2C+Ondr%CC%8Cej+Bojar%2C+Marta+R+Costa%2Djuss%C3%A0%2C+Christian+Federmann%2C+Mark+Fishel%2C+Yvette+Graham%2C+Barry+Haddow%2C+Matthias+Huck%2C+Philipp+Koehn%2C+Shervin+Malmasi%2C+et+al%2E+2019%2E+Findings+of+the+2019+confe) [our card](https://md.research.elamur.ai/free/en/papers/barrault-2019-findings-2019-conference-machine-translation/)
12. Sara Di Bartolomeo, Giorgio Severi, Victor Schetinger, and Cody Dunne. 2023. Ask and you shall receive (a graph drawing): Testing chatgpt’s potential to apply graph layout algorithms. [find](https://scholar.google.com/scholar?q=Sara+Di+Bartolomeo%2C+Giorgio+Severi%2C+Victor+Schetinger%2C+and+Cody+Dunne%2E+2023%2E+Ask+and+you+shall+receive+%28a+graph+drawing%29%3A+Testing+chatgpt%E2%80%99s+potential+to+apply+graph+layout+algorithms%2E)
13. Zeljana Basic, Ana Banovac, Ivana Kruzic, and Ivan Jerkovic. 2023. Better by you, better than me, chatgpt3 as writing assistance in students essays. [find](https://scholar.google.com/scholar?q=Zeljana+Basic%2C+Ana+Banovac%2C+Ivana+Kruzic%2C+and+Ivan+Jerkovic%2E+2023%2E+Better+by+you%2C+better+than+me%2C+chatgpt3+as+writing+assistance+in+students+essays%2E)
14. Iz Beltagy, Kyle Lo, and Arman Cohan. 2019. Scibert: A pretrained language model for scientific text. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3615–3620. [find](https://scholar.google.com/scholar?q=Iz+Beltagy%2C+Kyle+Lo%2C+and+Arman+Cohan%2E+2019%2E+Scibert%3A+A+pretrained+language+model+for+scientific+text%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natural+Language+Processing+and+the+) [our card](https://md.research.elamur.ai/free/en/papers/beltagy-2019-scibert-pretrained-language-model-scientific/)
15. Luisa Bentivogli, Ido Dagan, Hoa Trang Dang, Danilo Giampiccolo, and Bernardo Magnini. 2009. The fifth PASCAL recognizing textual entailment challenge. [find](https://scholar.google.com/scholar?q=Luisa+Bentivogli%2C+Ido+Dagan%2C+Hoa+Trang+Dang%2C+Danilo+Giampiccolo%2C+and+Bernardo+Magnini%2E+2009%2E+The+fifth+PASCAL+recognizing+textual+entailment+challenge%2E)
16. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. Semantic parsing on Freebase from question-answer pairs. In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pages 1533–1544, Seattle, Washington, USA. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Jonathan+Berant%2C+Andrew+Chou%2C+Roy+Frostig%2C+and+Percy+Liang%2E+2013%2E+Semantic+parsing+on+Freebase+from+question%2Danswer+pairs%2E+In+Proceedings+of+the+2013+Conference+on+Empirical+Methods+in+Natural+Languag)
17. Yonatan Bisk, Rowan Zellers, Jianfeng Gao, Yejin Choi, et al. 2020. Piqa: Reasoning about physical commonsense in natural language. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7432–7439. [find](https://scholar.google.com/scholar?q=Yonatan+Bisk%2C+Rowan+Zellers%2C+Jianfeng+Gao%2C+Yejin+Choi%2C+et+al%2E+2020%2E+Piqa%3A+Reasoning+about+physical+commonsense+in+natural+language%2E+In+Proceedings+of+the+AAAI+conference+on+artificial+intelligence%2C+vo) [our card](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/)
18. Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, Michael Pieler, USVSN Sai Prashanth, Shivanshu Purohit, Laria Reynolds, Jonathan Tow, Ben Wang, and Samuel Weinbach. 2022. Gpt-neox-20b: An open-source autoregr… [find](https://scholar.google.com/scholar?q=Sid+Black%2C+Stella+Biderman%2C+Eric+Hallahan%2C+Quentin+Anthony%2C+Leo+Gao%2C+Laurence+Golding%2C+Horace+He%2C+Connor+Leahy%2C+Kyle+McDonell%2C+Jason+Phang%2C+Michael+Pieler%2C+USVSN+Sai+Prashanth%2C+Shivanshu+Purohit%2C+Lari) [our card](https://md.research.elamur.ai/free/en/papers/black-2022-gpt-neox-20b-open-source/)
19. Ondřej Bojar, Christian Buck, Christian Federmann, Barry Haddow, Philipp Koehn, Johannes Leveling, Christof Monz, Pavel Pecina, Matt Post, Herve Saint-Amand, et al. 2014. Findings of the 2014 workshop on statistical machine translation. In Proceedings of the ninth workshop on statistical machine tr… [find](https://scholar.google.com/scholar?q=Ondr%CC%8Cej+Bojar%2C+Christian+Buck%2C+Christian+Federmann%2C+Barry+Haddow%2C+Philipp+Koehn%2C+Johannes+Leveling%2C+Christof+Monz%2C+Pavel+Pecina%2C+Matt+Post%2C+Herve+Saint%2DAmand%2C+et+al%2E+2014%2E+Findings+of+the+2014+worksho) [our card](https://md.research.elamur.ai/free/en/papers/bojar-2014-findings-2014-workshop-statistical-machine/)
20. Ondřej Bojar, Rajen Chatterjee, Christian Federmann, Yvette Graham, Barry Haddow, Matthias Huck, Antonio Jimeno Yepes, Philipp Koehn, Varvara Logacheva, Christof Monz, et al. 2016. Findings of the 2016 conference on machine translation. In Proceedings of the First Conference on Machine Translation:… [find](https://scholar.google.com/scholar?q=Ondr%CC%8Cej+Bojar%2C+Rajen+Chatterjee%2C+Christian+Federmann%2C+Yvette+Graham%2C+Barry+Haddow%2C+Matthias+Huck%2C+Antonio+Jimeno+Yepes%2C+Philipp+Koehn%2C+Varvara+Logacheva%2C+Christof+Monz%2C+et+al%2E+2016%2E+Findings+of+the+20) [our card](https://md.research.elamur.ai/free/en/papers/bojar-2016-findings-2016-conference-machine-translation/)
21. Ali Borji. 2023. A categorical archive of chatgpt failures. [find](https://scholar.google.com/scholar?q=Ali+Borji%2E+2023%2E+A+categorical+archive+of+chatgpt+failures%2E)
22. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. [find](https://scholar.google.com/scholar?q=Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+et+al%2E+2020%2E+Language+models+are+few%2Dshot+lear)
23. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006. The PASCAL recognising textual entailment challenge. In Machine learning challenges. evaluating predictive uncertainty, visual object classification, and recognising tectual entailment, pages 177–190. Springer. [find](https://scholar.google.com/scholar?q=Ido+Dagan%2C+Oren+Glickman%2C+and+Bernardo+Magnini%2E+2006%2E+The+PASCAL+recognising+textual+entailment+challenge%2E+In+Machine+learning+challenges%2E+evaluating+predictive+uncertainty%2C+visual+object+classificati)
24. Zhenguang G. Cai, David A. Haslett, Xufeng Duan, Shuqi Wang, and Martin J. Pickering. 2023. Does chatgpt resemble humans in language use? [find](https://scholar.google.com/scholar?q=Zhenguang+G%2E+Cai%2C+David+A%2E+Haslett%2C+Xufeng+Duan%2C+Shuqi+Wang%2C+and+Martin+J%2E+Pickering%2E+2023%2E+Does+chatgpt+resemble+humans+in+language+use%3F)
25. Haixing Dai, Zhengliang Liu, Wenxiong Liao, Xiaoke Huang, Yihan Cao, Zihao Wu, Lin Zhao, Shaochen Xu, Wei Liu, Ninghao Liu, Sheng Li, Dajiang Zhu, Hongmin Cai, Lichao Sun, Quanzheng Li, Dinggang Shen, Tianming Liu, and Xiang Li. 2023. Auggpt: Leveraging chatgpt for text data augmentation. [find](https://scholar.google.com/scholar?q=Haixing+Dai%2C+Zhengliang+Liu%2C+Wenxiong+Liao%2C+Xiaoke+Huang%2C+Yihan+Cao%2C+Zihao+Wu%2C+Lin+Zhao%2C+Shaochen+Xu%2C+Wei+Liu%2C+Ninghao+Liu%2C+Sheng+Li%2C+Dajiang+Zhu%2C+Hongmin+Cai%2C+Lichao+Sun%2C+Quanzheng+Li%2C+Dinggang+Shen%2C) [our card](https://md.research.elamur.ai/free/en/papers/dai-2023-auggpt-leveraging-chatgpt-text-data/)
26. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021a. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. [find](https://scholar.google.com/scholar?q=Mark+Chen%2C+Jerry+Tworek%2C+Heewoo+Jun%2C+Qiming+Yuan%2C+Henrique+Ponde+de+Oliveira+Pinto%2C+Jared+Kaplan%2C+Harri+Edwards%2C+Yuri+Burda%2C+Nicholas+Joseph%2C+Greg+Brockman%2C+et+al%2E+2021a%2E+Evaluating+large+language+mod) [our card](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
27. Yulong Chen, Yang Liu, Liang Chen, and Yue Zhang. 2021b. Dialogsum: A real-life scenario dialogue summarization dataset. arXiv preprint arXiv:2105.06762. [find](https://scholar.google.com/scholar?q=Yulong+Chen%2C+Yang+Liu%2C+Liang+Chen%2C+and+Yue+Zhang%2E+2021b%2E+Dialogsum%3A+A+real%2Dlife+scenario+dialogue+summarization+dataset%2E+arXiv+preprint+arXiv%3A2105%2E06762%2E) [our card](https://md.research.elamur.ai/free/en/papers/chen-2021-dialogsum-real-life-scenario-dialogue/)
28. Leon Derczynski, Eric Nichols, Marieke van Erp, and Nut Limsopatham. 2017. Results of the WNUT2017 shared task on novel and emerging entity recognition. In Proceedings of the 3rd Workshop on Noisy User-generated Text, pages 140–147, Copenhagen, Denmark. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Leon+Derczynski%2C+Eric+Nichols%2C+Marieke+van+Erp%2C+and+Nut+Limsopatham%2E+2017%2E+Results+of+the+WNUT2017+shared+task+on+novel+and+emerging+entity+recognition%2E+In+Proceedings+of+the+3rd+Workshop+on+Noisy+Use) [our card](https://md.research.elamur.ai/free/en/papers/derczynski-2017-results-wnut2017-shared-task-novel/)
29. Anton Cheshkov, Pavel Zadorozhny, and Rodion Levichev. 2023. Evaluation of chatgpt model for vulnerability detection. [find](https://scholar.google.com/scholar?q=Anton+Cheshkov%2C+Pavel+Zadorozhny%2C+and+Rodion+Levichev%2E+2023%2E+Evaluation+of+chatgpt+model+for+vulnerability+detection%2E)
30. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311. [find](https://scholar.google.com/scholar?q=Aakanksha+Chowdhery%2C+Sharan+Narang%2C+Jacob+Devlin%2C+Maarten+Bosma%2C+Gaurav+Mishra%2C+Adam+Roberts%2C+Paul+Barham%2C+Hyung+Won+Chung%2C+Charles+Sutton%2C+Sebastian+Gehrmann%2C+et+al%2E+2022%2E+Palm%3A+Scaling+language+mode)
31. Sanjay Deshpande and Jakub Szefer. 2023. Analyzing chatgpt’s aptitude in an introductory computer engineering course. [find](https://scholar.google.com/scholar?q=Sanjay+Deshpande+and+Jakub+Szefer%2E+2023%2E+Analyzing+chatgpt%E2%80%99s+aptitude+in+an+introductory+computer+engineering+course%2E)
32. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. 2022. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416. [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+Shayne+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+Eric+Li%2C+Xuezhi+Wang%2C+Mostafa+Dehghani%2C+Siddhartha+Brahma%2C+et+al%2E+2022%2E+Scaling+instruction%2Dfinetuned+language+models%2E+arXi)
33. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. 2019. Boolq: Exploring the surprising difficulty of natural yes/no questions. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics:… [find](https://scholar.google.com/scholar?q=Christopher+Clark%2C+Kenton+Lee%2C+Ming%2DWei+Chang%2C+Tom+Kwiatkowski%2C+Michael+Collins%2C+and+Kristina+Toutanova%2E+2019%2E+Boolq%3A+Exploring+the+surprising+difficulty+of+natural+yes%2Fno+questions%2E+In+Proceedings+of) [our card](https://md.research.elamur.ai/free/en/papers/clark-2019-boolq-exploring-surprising-difficulty-natural/)
34. Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. 2020. Electra: Pre-training text encoders as discriminators rather than generators. arXiv preprint arXiv:2003.10555. [find](https://scholar.google.com/scholar?q=Kevin+Clark%2C+Minh%2DThang+Luong%2C+Quoc+V+Le%2C+and+Christopher+D+Manning%2E+2020%2E+Electra%3A+Pre%2Dtraining+text+encoders+as+discriminators+rather+than+generators%2E+arXiv+preprint+arXiv%3A2003%2E10555%2E)
35. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. 2018. Think you have solved question answering? try arc, the ai2 reasoning challenge. arXiv preprint arXiv:1803.05457. [find](https://scholar.google.com/scholar?q=Peter+Clark%2C+Isaac+Cowhey%2C+Oren+Etzioni%2C+Tushar+Khot%2C+Ashish+Sabharwal%2C+Carissa+Schoenick%2C+and+Oyvind+Tafjord%2E+2018%2E+Think+you+have+solved+question+answering%3F+try+arc%2C+the+ai2+reasoning+challenge%2E+arX)
36. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168. [find](https://scholar.google.com/scholar?q=Karl+Cobbe%2C+Vineet+Kosaraju%2C+Mohammad+Bavarian%2C+Mark+Chen%2C+Heewoo+Jun%2C+Lukasz+Kaiser%2C+Matthias+Plappert%2C+Jerry+Tworek%2C+Jacob+Hilton%2C+Reiichiro+Nakano%2C+et+al%2E+2021%2E+Training+verifiers+to+solve+math+wor) [our card](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/)
37. Marie-Catherine De Marneffe, Mandy Simons, and Judith Tonhauser. 2019. The Commitment-Bank: Investigating projection in naturally occurring discourse. To appear in proceedings of Sinn und Bedeutung 23. Data can be found at [https://github.com/mcdm/CommitmentBank/](https://github.com/mcdm/CommitmentBank/). [↗](https://github.com/mcdm/CommitmentBank/)
38. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. Proceedings of the Annual Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+2018%2E+Bert%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+Proceedings+of+the+Annual+Conference+of+the+No)
39. Qingxiu Dong, Lei Li, Damai Dai, Ce Zheng, Zhiyong Wu, Baobao Chang, Xu Sun, Jingjing Xu, Lei Li, and Zhifang Sui. 2023. A survey for in-context learning. CoRR, abs/2301.00234. [find](https://scholar.google.com/scholar?q=Qingxiu+Dong%2C+Lei+Li%2C+Damai+Dai%2C+Ce+Zheng%2C+Zhiyong+Wu%2C+Baobao+Chang%2C+Xu+Sun%2C+Jingjing+Xu%2C+Lei+Li%2C+and+Zhifang+Sui%2E+2023%2E+A+survey+for+in%2Dcontext+learning%2E+CoRR%2C+abs%2F2301%2E00234%2E) [our card](https://md.research.elamur.ai/free/en/papers/dong-2023-survey-context-learning/)
40. Nan Du, Yanping Huang, Andrew M Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, et al. 2022. Glam: Efficient scaling of language models with mixture-of-experts. In International Conference on Machine Learning, pages 5547–5569. PMLR. [find](https://scholar.google.com/scholar?q=Nan+Du%2C+Yanping+Huang%2C+Andrew+M+Dai%2C+Simon+Tong%2C+Dmitry+Lepikhin%2C+Yuanzhong+Xu%2C+Maxim+Krikun%2C+Yanqi+Zhou%2C+Adams+Wei+Yu%2C+Orhan+Firat%2C+et+al%2E+2022%2E+Glam%3A+Efficient+scaling+of+language+models+with+mixtur) [our card](https://md.research.elamur.ai/free/en/papers/du-2021-glam-efficient-scaling-language-models/)
41. William Fedus, Barret Zoph, and Noam Shazeer. 2021. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. arXiv preprint arXiv:2101.03961. [find](https://scholar.google.com/scholar?q=William+Fedus%2C+Barret+Zoph%2C+and+Noam+Shazeer%2E+2021%2E+Switch+transformers%3A+Scaling+to+trillion+parameter+models+with+simple+and+efficient+sparsity%2E+arXiv+preprint+arXiv%3A2101%2E03961%2E)
42. Ronald Fischer, Markus Luczak-Roesch, and Johannes A Karl. 2023. What does chatgpt return about human values? exploring value bias in chatgpt using a descriptive value theory. [find](https://scholar.google.com/scholar?q=Ronald+Fischer%2C+Markus+Luczak%2DRoesch%2C+and+Johannes+A+Karl%2E+2023%2E+What+does+chatgpt+return+about+human+values%3F+exploring+value+bias+in+chatgpt+using+a+descriptive+value+theory%2E)
43. Simon Frieder, Luca Pinchetti, Ryan-Rhys Griffiths, Tommaso Salvatori, Thomas Lukasiewicz, Philipp Christian Petersen, Alexis Chevalier, and Julius Berner. 2023. Mathematical capabilities of chatgpt. [find](https://scholar.google.com/scholar?q=Simon+Frieder%2C+Luca+Pinchetti%2C+Ryan%2DRhys+Griffiths%2C+Tommaso+Salvatori%2C+Thomas+Lukasiewicz%2C+Philipp+Christian+Petersen%2C+Alexis+Chevalier%2C+and+Julius+Berner%2E+2023%2E+Mathematical+capabilities+of+chatgpt%2E) [our card](https://md.research.elamur.ai/free/en/papers/frieder-2023-mathematical-capabilities-chatgpt/)
44. Xue-Yong Fu, Cheng Chen, Md Tahmid Rahman Laskar, Shashi Bhushan Tn, and Simon Corston-Oliver. 2022. An effective, performant named entity recognition system for noisy business telephone conversation transcripts. In Proceedings of the Eighth Workshop on Noisy User-generated Text (W-NUT 2022), pages… [find](https://scholar.google.com/scholar?q=Xue%2DYong+Fu%2C+Cheng+Chen%2C+Md+Tahmid+Rahman+Laskar%2C+Shashi+Bhushan+Tn%2C+and+Simon+Corston%2DOliver%2E+2022%2E+An+effective%2C+performant+named+entity+recognition+system+for+noisy+business+telephone+conversation+)
45. Leo Gao, Stella Biderman, Sid Black, Laurence Golding, Travis Hoppe, Charles Foster, Jason Phang, Horace He, Anish Thite, Noa Nabeshima, Shawn Presser, and Connor Leahy. 2020. The Pile: An 800gb dataset of diverse text for language modeling. arXiv preprint arXiv:2101.00027. [find](https://scholar.google.com/scholar?q=Leo+Gao%2C+Stella+Biderman%2C+Sid+Black%2C+Laurence+Golding%2C+Travis+Hoppe%2C+Charles+Foster%2C+Jason+Phang%2C+Horace+He%2C+Anish+Thite%2C+Noa+Nabeshima%2C+Shawn+Presser%2C+and+Connor+Leahy%2E+2020%2E+The+Pile%3A+An+800gb+datas)
46. Mingqi Gao, Jie Ruan, Renliang Sun, Xunjian Yin, Shiping Yang, and Xiaojun Wan. 2023. Human-like summarization evaluation with chatgpt. [find](https://scholar.google.com/scholar?q=Mingqi+Gao%2C+Jie+Ruan%2C+Renliang+Sun%2C+Xunjian+Yin%2C+Shiping+Yang%2C+and+Xiaojun+Wan%2E+2023%2E+Human%2Dlike+summarization+evaluation+with+chatgpt%2E)
47. Siddhant Garg, Thuy Vu, and Alessandro Moschitti. 2020. Tanda: Transfer and adapt pre-trained transformer models for answer sentence selection. In Proceedings of the AAAI conference on artificial intelligence, volume 34, pages 7780–7788. [find](https://scholar.google.com/scholar?q=Siddhant+Garg%2C+Thuy+Vu%2C+and+Alessandro+Moschitti%2E+2020%2E+Tanda%3A+Transfer+and+adapt+pre%2Dtrained+transformer+models+for+answer+sentence+selection%2E+In+Proceedings+of+the+AAAI+conference+on+artificial+inte)
48. Danilo Giampiccolo, Bernardo Magnini, Ido Dagan, and Bill Dolan. 2007. The third PASCAL recognizing textual entailment challenge. In Proceedings of the ACL-PASCAL workshop on textual entailment and paraphrasing, pages 1–9. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Danilo+Giampiccolo%2C+Bernardo+Magnini%2C+Ido+Dagan%2C+and+Bill+Dolan%2E+2007%2E+The+third+PASCAL+recognizing+textual+entailment+challenge%2E+In+Proceedings+of+the+ACL%2DPASCAL+workshop+on+textual+entailment+and+pa)
49. Tatsunori B Hashimoto, Hugh Zhang, and Percy Liang. 2019. Unifying human and statistical evaluation for natural language generation. arXiv preprint arXiv:1904.02792. [find](https://scholar.google.com/scholar?q=Tatsunori+B+Hashimoto%2C+Hugh+Zhang%2C+and+Percy+Liang%2E+2019%2E+Unifying+human+and+statistical+evaluation+for+natural+language+generation%2E+arXiv+preprint+arXiv%3A1904%2E02792%2E)
50. Michael A Hedderich, Lukas Lange, Heike Adel, Jannik Strötgen, and Dietrich Klakow. 2021. A survey on recent approaches for natural language processing in low-resource scenarios. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Hum… [find](https://scholar.google.com/scholar?q=Michael+A+Hedderich%2C+Lukas+Lange%2C+Heike+Adel%2C+Jannik+Str%C3%B6tgen%2C+and+Dietrich+Klakow%2E+2021%2E+A+survey+on+recent+approaches+for+natural+language+processing+in+low%2Dresource+scenarios%2E+In+Proceedings+of+the) [our card](https://md.research.elamur.ai/free/en/papers/hedderich-2021-survey-recent-approaches-natural-language/)
51. Dan Hendrycks, Collin Burns, Steven Basart, Andrew Critch, Jerry Li, Dawn Song, and Jacob Steinhardt. 2021a. Aligning ai with shared human values. Proceedings of the International Conference on Learning Representations (ICLR). [find](https://scholar.google.com/scholar?q=Dan+Hendrycks%2C+Collin+Burns%2C+Steven+Basart%2C+Andrew+Critch%2C+Jerry+Li%2C+Dawn+Song%2C+and+Jacob+Steinhardt%2E+2021a%2E+Aligning+ai+with+shared+human+values%2E+Proceedings+of+the+International+Conference+on+Learni) [our card](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-aligning-ai-shared-human-values/)
52. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Dan+Hendrycks%2C+Collin+Burns%2C+Steven+Basart%2C+Andy+Zou%2C+Mantas+Mazeika%2C+Dawn+Song%2C+and+Jacob+Steinhardt%2E+Measuring+massive+multitask+language+understanding%2E+In+International+Conference+on+Learning+Repre) [our card](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/)
53. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. 2021b. Measuring mathematical problem solving with the math dataset. NeurIPS. [find](https://scholar.google.com/scholar?q=Dan+Hendrycks%2C+Collin+Burns%2C+Saurav+Kadavath%2C+Akul+Arora%2C+Steven+Basart%2C+Eric+Tang%2C+Dawn+Song%2C+and+Jacob+Steinhardt%2E+2021b%2E+Measuring+mathematical+problem+solving+with+the+math+dataset%2E+NeurIPS%2E) [our card](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/)
54. Amr Hendy, Mohamed Abdelrehim, Amr Sharaf, Vikas Raunak, Mohamed Gabr, Hitokazu Matsushita, Young Jin Kim, Mohamed Afify, and Hany Hassan Awadalla. 2023. How good are gpt models at machine translation? a comprehensive evaluation. [find](https://scholar.google.com/scholar?q=Amr+Hendy%2C+Mohamed+Abdelrehim%2C+Amr+Sharaf%2C+Vikas+Raunak%2C+Mohamed+Gabr%2C+Hitokazu+Matsushita%2C+Young+Jin+Kim%2C+Mohamed+Afify%2C+and+Hany+Hassan+Awadalla%2E+2023%2E+How+good+are+gpt+models+at+machine+translation)
55. Bogdan Gliwa, Iwona Mochol, Maciej Biesek, and Aleksander Wawer. 2019. Samsum corpus: A human-annotated dialogue dataset for abstractive summarization. In Proceedings of the 2nd Workshop on New Frontiers in Summarization, pages 70–79. [find](https://scholar.google.com/scholar?q=Bogdan+Gliwa%2C+Iwona+Mochol%2C+Maciej+Biesek%2C+and+Aleksander+Wawer%2E+2019%2E+Samsum+corpus%3A+A+human%2Dannotated+dialogue+dataset+for+abstractive+summarization%2E+In+Proceedings+of+the+2nd+Workshop+on+New+Fronti)
56. Karl Moritz Hermann, Tomas Kocisky, Edward Grefenstette, Lasse Espeholt, Will Kay, Mustafa Suleyman, and Phil Blunsom. 2015. Teaching machines to read and comprehend. Advances in neural information processing systems, 28. [find](https://scholar.google.com/scholar?q=Karl+Moritz+Hermann%2C+Tomas+Kocisky%2C+Edward+Grefenstette%2C+Lasse+Espeholt%2C+Will+Kay%2C+Mustafa+Suleyman%2C+and+Phil+Blunsom%2E+2015%2E+Teaching+machines+to+read+and+comprehend%2E+Advances+in+neural+information+pr) [our card](https://md.research.elamur.ai/free/en/papers/hermann-2015-teaching-machines-read-comprehend/)
57. Dongyu Gong. 2023. Assessing working memory capacity of chatgpt. [find](https://scholar.google.com/scholar?q=Dongyu+Gong%2E+2023%2E+Assessing+working+memory+capacity+of+chatgpt%2E)
58. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. 2022. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556. [find](https://scholar.google.com/scholar?q=Jordan+Hoffmann%2C+Sebastian+Borgeaud%2C+Arthur+Mensch%2C+Elena+Buchatskaya%2C+Trevor+Cai%2C+Eliza+Rutherford%2C+Diego+de+Las+Casas%2C+Lisa+Anne+Hendricks%2C+Johannes+Welbl%2C+Aidan+Clark%2C+et+al%2E+2022%2E+Training+compute)
59. Google. 2023. Palm 2 technical report. Goole AI. [find](https://scholar.google.com/scholar?q=Google%2E+2023%2E+Palm+2+technical+report%2E+Goole+AI%2E)
60. Tanya Goyal, Junyi Jessy Li, and Greg Durrett. 2022. News summarization and evaluation in the era of gpt-3. [find](https://scholar.google.com/scholar?q=Tanya+Goyal%2C+Junyi+Jessy+Li%2C+and+Greg+Durrett%2E+2022%2E+News+summarization+and+evaluation+in+the+era+of+gpt%2D3%2E)
61. Yu Gu, Robert Tinn, Hao Cheng, Michael Lucas, Naoto Usuyama, Xiaodong Liu, Tristan Naumann, Jianfeng Gao, and Hoifung Poon. 2020. Domain-specific language model pretraining for biomedical natural language processing. arXiv preprint arXiv:2007.15779. [find](https://scholar.google.com/scholar?q=Yu+Gu%2C+Robert+Tinn%2C+Hao+Cheng%2C+Michael+Lucas%2C+Naoto+Usuyama%2C+Xiaodong+Liu%2C+Tristan+Naumann%2C+Jianfeng+Gao%2C+and+Hoifung+Poon%2E+2020%2E+Domain%2Dspecific+language+model+pretraining+for+biomedical+natural+lang) [our card](https://md.research.elamur.ai/free/en/papers/gu-2020-domain-specific-language-model-pretraining/)
62. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. Lora: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685. [find](https://scholar.google.com/scholar?q=Edward+J+Hu%2C+Yelong+Shen%2C+Phillip+Wallis%2C+Zeyuan+Allen%2DZhu%2C+Yuanzhi+Li%2C+Shean+Wang%2C+Lu+Wang%2C+and+Weizhu+Chen%2E+2021%2E+Lora%3A+Low%2Drank+adaptation+of+large+language+models%2E+arXiv+preprint+arXiv%3A2106%2E09685%2E)
63. Shuaishuai Guo, Yanhu Wang, Shujing Li, and Nasir Saeed. 2023. Semantic communications with ordered importance using chatgpt. [find](https://scholar.google.com/scholar?q=Shuaishuai+Guo%2C+Yanhu+Wang%2C+Shujing+Li%2C+and+Nasir+Saeed%2E+2023%2E+Semantic+communications+with+ordered+importance+using+chatgpt%2E)
64. Junjie Hu, Sebastian Ruder, Aditya Siddhant, Graham Neubig, Orhan Firat, and Melvin Johnson. 2020. Xtreme: A massively multilingual multi-task benchmark for evaluating cross-lingual generalisation. In International Conference on Machine Learning, pages 4411–4421. PMLR. [find](https://scholar.google.com/scholar?q=Junjie+Hu%2C+Sebastian+Ruder%2C+Aditya+Siddhant%2C+Graham+Neubig%2C+Orhan+Firat%2C+and+Melvin+Johnson%2E+2020%2E+Xtreme%3A+A+massively+multilingual+multi%2Dtask+benchmark+for+evaluating+cross%2Dlingual+generalisation%2E+In)
65. Jochen Hartmann, Jasper Schwenzow, and Maximilian Witte. 2023. The political ideology of conversational ai: Converging evidence on chatgpt’s pro-environmental, left-libertarian orientation. [find](https://scholar.google.com/scholar?q=Jochen+Hartmann%2C+Jasper+Schwenzow%2C+and+Maximilian+Witte%2E+2023%2E+The+political+ideology+of+conversational+ai%3A+Converging+evidence+on+chatgpt%E2%80%99s+pro%2Denvironmental%2C+left%2Dlibertarian+orientation%2E)
66. Yan Hu, Iqra Ameer, Xu Zuo, Xueqing Peng, Yujia Zhou, Zehan Li, Yiming Li, Jianfu Li, Xiaoqian Jiang, and Hua Xu. 2023. Zero-shot clinical entity recognition using chatgpt. [find](https://scholar.google.com/scholar?q=Yan+Hu%2C+Iqra+Ameer%2C+Xu+Zuo%2C+Xueqing+Peng%2C+Yujia+Zhou%2C+Zehan+Li%2C+Yiming+Li%2C+Jianfu+Li%2C+Xiaoqian+Jiang%2C+and+Hua+Xu%2E+2023%2E+Zero%2Dshot+clinical+entity+recognition+using+chatgpt%2E)
67. Fan Huang, Haewoon Kwak, and Jisun An. 2023. Is ChatGPT better than human annotators? potential and limitations of ChatGPT in explaining implicit hate speech. In Companion Proceedings of the ACM Web Conference 2023. ACM. [find](https://scholar.google.com/scholar?q=Fan+Huang%2C+Haewoon+Kwak%2C+and+Jisun+An%2E+2023%2E+Is+ChatGPT+better+than+human+annotators%3F+potential+and+limitations+of+ChatGPT+in+explaining+implicit+hate+speech%2E+In+Companion+Proceedings+of+the+ACM+Web+C)
68. Sebastian Krügel, Andreas Ostermaier, and Matthias Uhl. 2023. The moral authority of chatgpt. [find](https://scholar.google.com/scholar?q=Sebastian+Kr%C3%BCgel%2C+Andreas+Ostermaier%2C+and+Matthias+Uhl%2E+2023%2E+The+moral+authority+of+chatgpt%2E)
69. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Red-field, Michael Collins, Ankur Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, et al. 2019. Natural questions: a benchmark for question answering research. Transactions of the Association for Computational Linguisti… [find](https://scholar.google.com/scholar?q=Tom+Kwiatkowski%2C+Jennimaria+Palomaki%2C+Olivia+Red%2Dfield%2C+Michael+Collins%2C+Ankur+Parikh%2C+Chris+Alberti%2C+Danielle+Epstein%2C+Illia+Polosukhin%2C+Jacob+Devlin%2C+Kenton+Lee%2C+et+al%2E+2019%2E+Natural+questions%3A+a+be) [our card](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/)
70. Israt Jahan, Md Tahmid Rahman Laskar, Chun Peng, and Jimmy Huang. 2023. Evaluation of chatgpt on biomedical tasks: A zero-shot comparison with fine-tuned generative transformers. arXiv preprint arXiv:2306.04504. [find](https://scholar.google.com/scholar?q=Israt+Jahan%2C+Md+Tahmid+Rahman+Laskar%2C+Chun+Peng%2C+and+Jimmy+Huang%2E+2023%2E+Evaluation+of+chatgpt+on+biomedical+tasks%3A+A+zero%2Dshot+comparison+with+fine%2Dtuned+generative+transformers%2E+arXiv+preprint+arXiv%3A)
71. Myeongjun Jang and Thomas Lukasiewicz. 2023. Consistency analysis of chatgpt. [find](https://scholar.google.com/scholar?q=Myeongjun+Jang+and+Thomas+Lukasiewicz%2E+2023%2E+Consistency+analysis+of+chatgpt%2E)
72. Wenxiang Jiao, Wenxuan Wang, Jen tse Huang, Xing Wang, and Zhaopeng Tu. 2023. Is chatgpt a good translator? yes with gpt-4 as the engine. [find](https://scholar.google.com/scholar?q=Wenxiang+Jiao%2C+Wenxuan+Wang%2C+Jen+tse+Huang%2C+Xing+Wang%2C+and+Zhaopeng+Tu%2E+2023%2E+Is+chatgpt+a+good+translator%3F+yes+with+gpt%2D4+as+the+engine%2E) [our card](https://md.research.elamur.ai/free/en/papers/jiao-2023-chatgpt-good-translator-yes-gpt/)
73. Mandar Joshi, Eunsol Choi, Daniel S Weld, and Luke Zettlemoyer. 2017. Triviaqa: A large scale distantly supervised challenge dataset for reading comprehension. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1601–1611. [find](https://scholar.google.com/scholar?q=Mandar+Joshi%2C+Eunsol+Choi%2C+Daniel+S+Weld%2C+and+Luke+Zettlemoyer%2E+2017%2E+Triviaqa%3A+A+large+scale+distantly+supervised+challenge+dataset+for+reading+comprehension%2E+In+Proceedings+of+the+55th+Annual+Meetin) [our card](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/)
74. Shankar Kantharaj, Rixie Tiffany Leong, Xiang Lin, Ahmed Masry, Megh Thakkar, Enamul Hoque, and Shafiq Joty. 2022. Chart-to-text: A large-scale benchmark for chart summarization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages… [find](https://scholar.google.com/scholar?q=Shankar+Kantharaj%2C+Rixie+Tiffany+Leong%2C+Xiang+Lin%2C+Ahmed+Masry%2C+Megh+Thakkar%2C+Enamul+Hoque%2C+and+Shafiq+Joty%2E+2022%2E+Chart%2Dto%2Dtext%3A+A+large%2Dscale+benchmark+for+chart+summarization%2E+In+Proceedings+of+the)
75. Jungo Kasai, Yuhei Kasai, Keisuke Sakaguchi, Yutaro Yamada, and Dragomir Radev. 2023. Evaluating gpt-4 and chatgpt on japanese medical licensing examinations. [find](https://scholar.google.com/scholar?q=Jungo+Kasai%2C+Yuhei+Kasai%2C+Keisuke+Sakaguchi%2C+Yutaro+Yamada%2C+and+Dragomir+Radev%2E+2023%2E+Evaluating+gpt%2D4+and+chatgpt+on+japanese+medical+licensing+examinations%2E)
76. Ali Kashefi and Tapan Mukerji. 2023. Chatgpt for programming numerical methods. [find](https://scholar.google.com/scholar?q=Ali+Kashefi+and+Tapan+Mukerji%2E+2023%2E+Chatgpt+for+programming+numerical+methods%2E)
77. Daniel Khashabi, Snigdha Chaturvedi, Michael Roth, Shyam Upadhyay, and Dan Roth. 2018. Looking beyond the surface: A challenge set for reading comprehension over multiple sentences. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics:… [find](https://scholar.google.com/scholar?q=Daniel+Khashabi%2C+Snigdha+Chaturvedi%2C+Michael+Roth%2C+Shyam+Upadhyay%2C+and+Dan+Roth%2E+2018%2E+Looking+beyond+the+surface%3A+A+challenge+set+for+reading+comprehension+over+multiple+sentences%2E+In+Proceedings+of+) [our card](https://md.research.elamur.ai/free/en/papers/khashabi-2018-looking-beyond-surface-challenge-set/)
78. Seungone Kim, Se June Joo, Hyungjoo Chae, Chae-hyeong Kim, Seung-won Hwang, and Jinyoung Yeo. 2022. Mind the gap! injecting commonsense knowledge for abstractive dialogue summarization. In Proceedings of the 29th International Conference on Computational Linguistics, pages 6285–6300. [find](https://scholar.google.com/scholar?q=Seungone+Kim%2C+Se+June+Joo%2C+Hyungjoo+Chae%2C+Chae%2Dhyeong+Kim%2C+Seung%2Dwon+Hwang%2C+and+Jinyoung+Yeo%2E+2022%2E+Mind+the+gap%21+injecting+commonsense+knowledge+for+abstractive+dialogue+summarization%2E+In+Proceedings)
79. Jan Kocoń, Igor Cichecki, Oliwier Kaszyca, Mateusz Kochanek, Dominika Szydło, Joanna Baran, Julita Bielaniewicz, Marcin Gruza, Arkadiusz Janz, Kamil Kanclerz, Anna Kocoń, Bartłomiej Koptyra, Wiktoria Mieleszczenko-Kowszewicz, Piotr Miłkowski, Marcin Oleksy, Maciej Piasecki, Łukasz Radliński, Konr… [find](https://scholar.google.com/scholar?q=Jan+Kocon%CC%81%2C+Igor+Cichecki%2C+Oliwier+Kaszyca%2C+Mateusz+Kochanek%2C+Dominika+Szyd%C5%82o%2C+Joanna+Baran%2C+Julita+Bielaniewicz%2C+Marcin+Gruza%2C+Arkadiusz+Janz%2C+Kamil+Kanclerz%2C+Anna+Kocon%CC%81%2C+Bart%C5%82omiej+Koptyra%2C+Wiktori) [our card](https://md.research.elamur.ai/free/en/papers/kocon-2023-chatgpt-jack-all-trades-master/)
80. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. Race: Large-scale reading comprehension dataset from examinations. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 785– 794. [find](https://scholar.google.com/scholar?q=Guokun+Lai%2C+Qizhe+Xie%2C+Hanxiao+Liu%2C+Yiming+Yang%2C+and+Eduard+Hovy%2E+2017%2E+Race%3A+Large%2Dscale+reading+comprehension+dataset+from+examinations%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in) [our card](https://md.research.elamur.ai/free/en/papers/lai-2017-race-large-scale-reading-comprehension/)
81. Viet Dac Lai, Nghia Trung Ngo, Amir Pouran Ben Veyseh, Hieu Man, Franck Dernoncourt, Trung Bui, and Thien Huu Nguyen. 2023. Chatgpt beyond english: Towards a comprehensive evaluation of large language models in multilingual learning. [find](https://scholar.google.com/scholar?q=Viet+Dac+Lai%2C+Nghia+Trung+Ngo%2C+Amir+Pouran+Ben+Veyseh%2C+Hieu+Man%2C+Franck+Dernoncourt%2C+Trung+Bui%2C+and+Thien+Huu+Nguyen%2E+2023%2E+Chatgpt+beyond+english%3A+Towards+a+comprehensive+evaluation+of+large+language) [our card](https://md.research.elamur.ai/free/en/papers/lai-2023-chatgpt-beyond-english-comprehensive-evaluation/)
82. Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. Albert: A lite bert for self-supervised learning of language representations. arXiv preprint arXiv:1909.11942. [find](https://scholar.google.com/scholar?q=Zhenzhong+Lan%2C+Mingda+Chen%2C+Sebastian+Goodman%2C+Kevin+Gimpel%2C+Piyush+Sharma%2C+and+Radu+Soricut%2E+2019%2E+Albert%3A+A+lite+bert+for+self%2Dsupervised+learning+of+language+representations%2E+arXiv+preprint+arXiv%3A1)
83. Md Tahmid Rahman Laskar, Cheng Chen, Xue-Yong Fu, and Shashi Bhushan TN. 2022a. Improving named entity recognition in telephone conversations via effective active learning with human in the loop. arXiv preprint arXiv:2211.01354. [find](https://scholar.google.com/scholar?q=Md+Tahmid+Rahman+Laskar%2C+Cheng+Chen%2C+Xue%2DYong+Fu%2C+and+Shashi+Bhushan+TN%2E+2022a%2E+Improving+named+entity+recognition+in+telephone+conversations+via+effective+active+learning+with+human+in+the+loop%2E+arXi)
84. Md Tahmid Rahman Laskar, Cheng Chen, Jonathan Johnston, Xue-Yong Fu, Shashi Bhushan TN, and Simon Corston-Oliver. 2022b. An auto encoder-based dimensionality reduction technique for efficient entity linking in business phone conversations. In Proceedings of the 45th International ACM SIGIR Conferenc… [find](https://scholar.google.com/scholar?q=Md+Tahmid+Rahman+Laskar%2C+Cheng+Chen%2C+Jonathan+Johnston%2C+Xue%2DYong+Fu%2C+Shashi+Bhushan+TN%2C+and+Simon+Corston%2DOliver%2E+2022b%2E+An+auto+encoder%2Dbased+dimensionality+reduction+technique+for+efficient+entity+l)
85. Md Tahmid Rahman Laskar, Cheng Chen, Aliaksandr Martsinovich, Jonathan Johnston, Xue-Yong Fu, Shashi Bhushan Tn, and Simon Corston-Oliver. 2022c. BLINK with Elasticsearch for efficient entity linking in business conversations. In Proceedings of the 2022 Conference of the North American Chapter of th… [find](https://scholar.google.com/scholar?q=Md+Tahmid+Rahman+Laskar%2C+Cheng+Chen%2C+Aliaksandr+Martsinovich%2C+Jonathan+Johnston%2C+Xue%2DYong+Fu%2C+Shashi+Bhushan+Tn%2C+and+Simon+Corston%2DOliver%2E+2022c%2E+BLINK+with+Elasticsearch+for+efficient+entity+linking+)
86. Md Tahmid Rahman Laskar, Enamul Hoque, and Jimmy Xiangji Huang. 2022d. Domain adaptation with pre-trained transformers for query-focused abstractive text summarization. Computational Linguistics, 48(2):279–320. [find](https://scholar.google.com/scholar?q=Md+Tahmid+Rahman+Laskar%2C+Enamul+Hoque%2C+and+Jimmy+Xiangji+Huang%2E+2022d%2E+Domain+adaptation+with+pre%2Dtrained+transformers+for+query%2Dfocused+abstractive+text+summarization%2E+Computational+Linguistics%2C+48%282)
87. Md Tahmid Rahman Laskar, Enamul Hoque, and Xiangji Huang. 2020a. WSL-DS: Weakly supervised learning with distant supervision for query focused multi-document abstractive summarization. In Proceedings of the 28th International Conference on Computational Linguistics, pages 5647–5654. [find](https://scholar.google.com/scholar?q=Md+Tahmid+Rahman+Laskar%2C+Enamul+Hoque%2C+and+Xiangji+Huang%2E+2020a%2E+WSL%2DDS%3A+Weakly+supervised+learning+with+distant+supervision+for+query+focused+multi%2Ddocument+abstractive+summarization%2E+In+Proceedings+)
88. Md Tahmid Rahman Laskar, Xiangji Huang, and Enamul Hoque. 2020b. Contextualized embeddings based transformer encoder for sentence similarity modeling in answer selection task. In Proceedings of the Twelfth Language Resources and Evaluation Conference, pages 5505–5514. [find](https://scholar.google.com/scholar?q=Md+Tahmid+Rahman+Laskar%2C+Xiangji+Huang%2C+and+Enamul+Hoque%2E+2020b%2E+Contextualized+embeddings+based+transformer+encoder+for+sentence+similarity+modeling+in+answer+selection+task%2E+In+Proceedings+of+the+Tw)
89. Md Tahmid Rahman Laskar, Mizanur Rahman, Israt Jahan, Enamul Hoque, and Jimmy Huang. 2023. Cqsumdp: A chatgpt-annotated resource for query-focused abstractive summarization based on debatepedia. arXiv preprint arXiv:2305.06147. — [arXiv:2305.06147](https://arxiv.org/abs/2305.06147) [find](https://scholar.google.com/scholar?q=Md+Tahmid+Rahman+Laskar%2C+Mizanur+Rahman%2C+Israt+Jahan%2C+Enamul+Hoque%2C+and+Jimmy+Huang%2E+2023%2E+Cqsumdp%3A+A+chatgpt%2Dannotated+resource+for+query%2Dfocused+abstractive+summarization+based+on+debatepedia%2E+arXiv)
90. Jinhyuk Lee, Wonjin Yoon, Sungdong Kim, Donghyeon Kim, Sunkyu Kim, Chan Ho So, and Jaewoo Kang. 2020. Biobert: a pre-trained biomedical language representation model for biomedical text mining. Bioinformatics, 36(4):1234–1240. [find](https://scholar.google.com/scholar?q=Jinhyuk+Lee%2C+Wonjin+Yoon%2C+Sungdong+Kim%2C+Donghyeon+Kim%2C+Sunkyu+Kim%2C+Chan+Ho+So%2C+and+Jaewoo+Kang%2E+2020%2E+Biobert%3A+a+pre%2Dtrained+biomedical+language+representation+model+for+biomedical+text+mining%2E+Bioinf) [our card](https://md.research.elamur.ai/free/en/papers/lee-2019-biobert-pre-trained-biomedical-language/)
91. Christoph Leiter, Ran Zhang, Yanran Chen, Jonas Belouadi, Daniil Larionov, Vivian Fresen, and Steffen Eger. 2023. Chatgpt: A meta-analysis after 2.5 months. [find](https://scholar.google.com/scholar?q=Christoph+Leiter%2C+Ran+Zhang%2C+Yanran+Chen%2C+Jonas+Belouadi%2C+Daniil+Larionov%2C+Vivian+Fresen%2C+and+Steffen+Eger%2E+2023%2E+Chatgpt%3A+A+meta%2Danalysis+after+2%2E5+months%2E)
92. Brian Lester, Rami Al-Rfou, and Noah Constant. 2021. The power of scale for parameter-efficient prompt tuning. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pages 3045–3059, Online and Punta Cana, Dominican Republic. Association for Computational Linguist… [find](https://scholar.google.com/scholar?q=Brian+Lester%2C+Rami+Al%2DRfou%2C+and+Noah+Constant%2E+2021%2E+The+power+of+scale+for+parameter%2Defficient+prompt+tuning%2E+In+Proceedings+of+the+2021+Conference+on+Empirical+Methods+in+Natural+Language+Processing)
93. Hector J Levesque, Ernest Davis, and Leora Morgenstern. 2011. The Winograd schema challenge. In AAAI Spring Symposium: Logical Formalizations of Commonsense Reasoning, volume 46, page 47. [find](https://scholar.google.com/scholar?q=Hector+J+Levesque%2C+Ernest+Davis%2C+and+Leora+Morgenstern%2E+2011%2E+The+Winograd+schema+challenge%2E+In+AAAI+Spring+Symposium%3A+Logical+Formalizations+of+Commonsense+Reasoning%2C+volume+46%2C+page+47%2E)
94. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the… [find](https://scholar.google.com/scholar?q=Mike+Lewis%2C+Yinhan+Liu%2C+Naman+Goyal%2C+Marjan+Ghazvininejad%2C+Abdelrahman+Mohamed%2C+Omer+Levy%2C+Veselin+Stoyanov%2C+and+Luke+Zettlemoyer%2E+2020%2E+Bart%3A+Denoising+sequence%2Dto%2Dsequence+pre%2Dtraining+for+natural+l) [our card](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/)
95. Aitor Lewkowycz, Anders Johan Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Venkatesh Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, et al. Solving quantitative reasoning problems with language models. In Advances in Neural Information Processing Systems. [find](https://scholar.google.com/scholar?q=Aitor+Lewkowycz%2C+Anders+Johan+Andreassen%2C+David+Dohan%2C+Ethan+Dyer%2C+Henryk+Michalewski%2C+Vinay+Venkatesh+Ramasesh%2C+Ambrose+Slone%2C+Cem+Anil%2C+Imanol+Schlag%2C+Theo+Gutman%2DSolo%2C+et+al%2E+Solving+quantitative+r) [our card](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/)
96. Bo Li, Gexiang Fang, Yang Yang, Quansen Wang, Wei Ye, Wen Zhao, and Shikun Zhang. 2023a. Evaluating chatgpt’s information extraction capabilities: An assessment of performance, explainability, calibration, and faithfulness. [find](https://scholar.google.com/scholar?q=Bo+Li%2C+Gexiang+Fang%2C+Yang+Yang%2C+Quansen+Wang%2C+Wei+Ye%2C+Wen+Zhao%2C+and+Shikun+Zhang%2E+2023a%2E+Evaluating+chatgpt%E2%80%99s+information+extraction+capabilities%3A+An+assessment+of+performance%2C+explainability%2C+calibra)
97. Haoran Li, Dadi Guo, Wei Fan, Mingshi Xu, Jie Huang, Fanpu Meng, and Yangqiu Song. 2023b. Multi-step jailbreaking privacy attacks on chatgpt. [find](https://scholar.google.com/scholar?q=Haoran+Li%2C+Dadi+Guo%2C+Wei+Fan%2C+Mingshi+Xu%2C+Jie+Huang%2C+Fanpu+Meng%2C+and+Yangqiu+Song%2E+2023b%2E+Multi%2Dstep+jailbreaking+privacy+attacks+on+chatgpt%2E) [our card](https://md.research.elamur.ai/free/en/papers/li-2023-multi-step-jailbreaking-privacy-attacks/)
98. Lingyao Li, Lizhou Fan, Shubham Atreja, and Libby Hemphill. 2023c. "hot" chatgpt: The promise of chatgpt in detecting and discriminating hateful, offensive, and toxic comments on social media. [find](https://scholar.google.com/scholar?q=Lingyao+Li%2C+Lizhou+Fan%2C+Shubham+Atreja%2C+and+Libby+Hemphill%2E+2023c%2E+%22hot%22+chatgpt%3A+The+promise+of+chatgpt+in+detecting+and+discriminating+hateful%2C+offensive%2C+and+toxic+comments+on+social+media%2E)
99. Xiang Lisa Li and Percy Liang. 2021. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4582–… [find](https://scholar.google.com/scholar?q=Xiang+Lisa+Li+and+Percy+Liang%2E+2021%2E+Prefix%2Dtuning%3A+Optimizing+continuous+prompts+for+generation%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+11t)
100. Yaobo Liang, Nan Duan, Yeyun Gong, Ning Wu, Fenfei Guo, Weizhen Qi, Ming Gong, Linjun Shou, Daxin Jiang, Guihong Cao, Xiaodong Fan, Ruofei Zhang, Rahul Agrawal, Edward Cui, Sining Wei, Taroon Bharti, Ying Qiao, Jiun-Hung Chen, Winnie Wu, Shuguang Liu, Fan Yang, Daniel Campos, Rangan Majumder, and Mi… [find](https://scholar.google.com/scholar?q=Yaobo+Liang%2C+Nan+Duan%2C+Yeyun+Gong%2C+Ning+Wu%2C+Fenfei+Guo%2C+Weizhen+Qi%2C+Ming+Gong%2C+Linjun+Shou%2C+Daxin+Jiang%2C+Guihong+Cao%2C+Xiaodong+Fan%2C+Ruofei+Zhang%2C+Rahul+Agrawal%2C+Edward+Cui%2C+Sining+Wei%2C+Taroon+Bharti%2C+) [our card](https://md.research.elamur.ai/free/en/papers/liang-2020-xglue-new-benchmark-dataset-cross/)
101. Wenxiong Liao, Zhengliang Liu, Haixing Dai, Shaochen Xu, Zihao Wu, Yiyang Zhang, Xiaoke Huang, Dajiang Zhu, Hongmin Cai, Tianming Liu, and Xiang Li. 2023. Differentiate chatgpt-generated and human-written medical texts. [find](https://scholar.google.com/scholar?q=Wenxiong+Liao%2C+Zhengliang+Liu%2C+Haixing+Dai%2C+Shaochen+Xu%2C+Zihao+Wu%2C+Yiyang+Zhang%2C+Xiaoke+Huang%2C+Dajiang+Zhu%2C+Hongmin+Cai%2C+Tianming+Liu%2C+and+Xiang+Li%2E+2023%2E+Differentiate+chatgpt%2Dgenerated+and+human%2Dwri)
102. Chin-Yew Lin. 2004. Rouge: A package for automatic evaluation of summaries. In Text summarization branches out, pages 74–81. [find](https://scholar.google.com/scholar?q=Chin%2DYew+Lin%2E+2004%2E+Rouge%3A+A+package+for+automatic+evaluation+of+summaries%2E+In+Text+summarization+branches+out%2C+pages+74%E2%80%9381%2E)
103. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. Truthfulqa: Measuring how models mimic human falsehoods. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252. [find](https://scholar.google.com/scholar?q=Stephanie+Lin%2C+Jacob+Hilton%2C+and+Owain+Evans%2E+2022%2E+Truthfulqa%3A+Measuring+how+models+mimic+human+falsehoods%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+) [our card](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/)
104. Aiwei Liu, Xuming Hu, Lijie Wen, and Philip S. Yu. 2023a. A comprehensive evaluation of chatgpt’s zero-shot text-to-sql capability. [find](https://scholar.google.com/scholar?q=Aiwei+Liu%2C+Xuming+Hu%2C+Lijie+Wen%2C+and+Philip+S%2E+Yu%2E+2023a%2E+A+comprehensive+evaluation+of+chatgpt%E2%80%99s+zero%2Dshot+text%2Dto%2Dsql+capability%2E)
105. Chia-Wei Liu, Ryan Lowe, Iulian Serban, Mike Noseworthy, Laurent Charlin, and Joelle Pineau. 2016. How NOT to evaluate your dialogue system: An empirical study of unsupervised evaluation metrics for dialogue response generation. In Proceedings of the 2016 Conference on Empirical Methods in Natural L… [find](https://scholar.google.com/scholar?q=Chia%2DWei+Liu%2C+Ryan+Lowe%2C+Iulian+Serban%2C+Mike+Noseworthy%2C+Laurent+Charlin%2C+and+Joelle+Pineau%2E+2016%2E+How+NOT+to+evaluate+your+dialogue+system%3A+An+empirical+study+of+unsupervised+evaluation+metrics+for+d)
106. Jiawei Liu, Chunqiu Steven Xia, Yuyao Wang, and Lingming Zhang. 2023b. Is your code generated by chatgpt really correct? rigorous evaluation of large language models for code generation. [find](https://scholar.google.com/scholar?q=Jiawei+Liu%2C+Chunqiu+Steven+Xia%2C+Yuyao+Wang%2C+and+Lingming+Zhang%2E+2023b%2E+Is+your+code+generated+by+chatgpt+really+correct%3F+rigorous+evaluation+of+large+language+models+for+code+generation%2E) [our card](https://md.research.elamur.ai/free/en/papers/liu-2023-your-code-generated-chatgpt-really/)
107. Pengfei Liu, Weizhe Yuan, Jinlan Fu, Zhengbao Jiang, Hiroaki Hayashi, and Graham Neubig. 2023c. Pre-train, prompt, and predict: A systematic survey of prompting methods in natural language processing. ACM Computing Surveys, 55(9):1–35. [find](https://scholar.google.com/scholar?q=Pengfei+Liu%2C+Weizhe+Yuan%2C+Jinlan+Fu%2C+Zhengbao+Jiang%2C+Hiroaki+Hayashi%2C+and+Graham+Neubig%2E+2023c%2E+Pre%2Dtrain%2C+prompt%2C+and+predict%3A+A+systematic+survey+of+prompting+methods+in+natural+language+processing%2E) [our card](https://md.research.elamur.ai/free/en/papers/liu-2021-pre-train-prompt-predict-systematic/)
108. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. Roberta: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019%2E+Roberta%3A+A+robustly+optimized+bert+pretraining+approach%2E+a)
109. Hannaneh Hajishirzi, Kenton Lee, Jennimaria Palomaki, et al. 2021. [find](https://scholar.google.com/scholar?q=Hannaneh+Hajishirzi%2C+Kenton+Lee%2C+Jennimaria+Palomaki%2C+et+al%2E+2021%2E)
110. Neurips 2020 efficientqa competition: Systems, analyses and lessons learned. In NeurIPS 2020 Competition and Demonstration Track, pages 86–111. PMLR. [find](https://scholar.google.com/scholar?q=Neurips+2020+efficientqa+competition%3A+Systems%2C+analyses+and+lessons+learned%2E+In+NeurIPS+2020+Competition+and+Demonstration+Track%2C+pages+86%E2%80%93111%2E+PMLR%2E)
111. Yixin Liu, Alex Fabbri, Pengfei Liu, Yilun Zhao, Linyong Nan, Ruilin Han, Simeng Han, Shafiq Joty, Chien-Sheng Jason Wu, Caiming Xiong, and Dragomir Radev. 2023d. Revisiting the gold standard: Grounding summarization evaluation with robust human evaluation. In Proceedings of the 61st Annual Meeting… [find](https://scholar.google.com/scholar?q=Yixin+Liu%2C+Alex+Fabbri%2C+Pengfei+Liu%2C+Yilun+Zhao%2C+Linyong+Nan%2C+Ruilin+Han%2C+Simeng+Han%2C+Shafiq+Joty%2C+Chien%2DSheng+Jason+Wu%2C+Caiming+Xiong%2C+and+Dragomir+Radev%2E+2023d%2E+Revisiting+the+gold+standard%3A+Groundi) [our card](https://md.research.elamur.ai/free/en/papers/liu-2022-revisiting-gold-standard-grounding-summarization/)
112. Swaroop Mishra, Daniel Khashabi, Chitta Baral, and Hannaneh Hajishirzi. 2021. Cross-task generalization via natural language crowdsourcing instructions. arXiv preprint arXiv:2104.08773. [find](https://scholar.google.com/scholar?q=Swaroop+Mishra%2C+Daniel+Khashabi%2C+Chitta+Baral%2C+and+Hannaneh+Hajishirzi%2E+2021%2E+Cross%2Dtask+generalization+via+natural+language+crowdsourcing+instructions%2E+arXiv+preprint+arXiv%3A2104%2E08773%2E)
113. Ryan Lowe, Michael Noseworthy, Iulian V Serban, Nicolas Angelard-Gontier, Yoshua Bengio, and Joelle Pineau. 2017. Towards an automatic turing test: Learning to evaluate dialogue responses. arXiv preprint arXiv:1708.07149. [find](https://scholar.google.com/scholar?q=Ryan+Lowe%2C+Michael+Noseworthy%2C+Iulian+V+Serban%2C+Nicolas+Angelard%2DGontier%2C+Yoshua+Bengio%2C+and+Joelle+Pineau%2E+2017%2E+Towards+an+automatic+turing+test%3A+Learning+to+evaluate+dialogue+responses%2E+arXiv+prepr) [our card](https://md.research.elamur.ai/free/en/papers/lowe-2017-towards-automatic-turing-test-learning/)
114. Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Hailey Schoelkopf, et al. 2022. Crosslingual generalization through multitask finetuning. arXiv preprint arXiv:2211.01786. [find](https://scholar.google.com/scholar?q=Niklas+Muennighoff%2C+Thomas+Wang%2C+Lintang+Sutawika%2C+Adam+Roberts%2C+Stella+Biderman%2C+Teven+Le+Scao%2C+M+Saiful+Bari%2C+Sheng+Shen%2C+Zheng%2DXin+Yong%2C+Hailey+Schoelkopf%2C+et+al%2E+2022%2E+Crosslingual+generalization+) [our card](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-crosslingual-generalization-through-multitask-finetuning/)
115. Shuai Lu, Daya Guo, Shuo Ren, Junjie Huang, Alexey Svyatkovskiy, Ambrosio Blanco, Colin B. Clement, Dawn Drain, Daxin Jiang, Duyu Tang, Ge Li, Lidong Zhou, Linjun Shou, Long Zhou, Michele Tufano, Ming Gong, Ming Zhou, Nan Duan, Neel Sundaresan, Shao Kun Deng, Shengyu Fu, and Shujie Liu. 2021. Codexg… [find](https://scholar.google.com/scholar?q=Shuai+Lu%2C+Daya+Guo%2C+Shuo+Ren%2C+Junjie+Huang%2C+Alexey+Svyatkovskiy%2C+Ambrosio+Blanco%2C+Colin+B%2E+Clement%2C+Dawn+Drain%2C+Daxin+Jiang%2C+Duyu+Tang%2C+Ge+Li%2C+Lidong+Zhou%2C+Linjun+Shou%2C+Long+Zhou%2C+Michele+Tufano%2C+Ming)
116. Renqian Luo, Liai Sun, Yingce Xia, Tao Qin, Sheng Zhang, Hoifung Poon, and Tie-Yan Liu. 2022. Biogpt: generative pre-trained transformer for biomedical text generation and mining. Briefings in bioinformatics, 23(6):bbac409. [find](https://scholar.google.com/scholar?q=Renqian+Luo%2C+Liai+Sun%2C+Yingce+Xia%2C+Tao+Qin%2C+Sheng+Zhang%2C+Hoifung+Poon%2C+and+Tie%2DYan+Liu%2E+2022%2E+Biogpt%3A+generative+pre%2Dtrained+transformer+for+biomedical+text+generation+and+mining%2E+Briefings+in+bioinfo)
117. Andrew L. Maas, Raymond E. Daly, Peter T. Pham, Dan Huang, Andrew Y. Ng, and Christopher Potts. 2011. Learning word vectors for sentiment analysis. In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 142–150, Portland, Oregon… [find](https://scholar.google.com/scholar?q=Andrew+L%2E+Maas%2C+Raymond+E%2E+Daly%2C+Peter+T%2E+Pham%2C+Dan+Huang%2C+Andrew+Y%2E+Ng%2C+and+Christopher+Potts%2E+2011%2E+Learning+word+vectors+for+sentiment+analysis%2E+In+Proceedings+of+the+49th+Annual+Meeting+of+the+Ass) [our card](https://md.research.elamur.ai/free/en/papers/maas-2011-learning-word-vectors-sentiment-analysis/)
118. Shervin Malmasi, Anjie Fang, Besnik Fetahu, Sudipta Kar, and Oleg Rokhlenko. 2022. Semeval-2022 task 11: Multilingual complex named entity recognition (multiconer). In Proceedings of the 16th International Workshop on Semantic Evaluation (SemEval-2022). Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Shervin+Malmasi%2C+Anjie+Fang%2C+Besnik+Fetahu%2C+Sudipta+Kar%2C+and+Oleg+Rokhlenko%2E+2022%2E+Semeval%2D2022+task+11%3A+Multilingual+complex+named+entity+recognition+%28multiconer%29%2E+In+Proceedings+of+the+16th+Internat) [our card](https://md.research.elamur.ai/free/en/papers/malmasi-2022-semeval-2022-task-11-multilingual/)
119. Gary Marcus. 2022. Is chatgpt really a “code red” for google search? [find](https://scholar.google.com/scholar?q=Gary+Marcus%2E+2022%2E+Is+chatgpt+really+a+%E2%80%9Ccode+red%E2%80%9D+for+google+search%3F)
120. Todor Mihaylov, Peter Clark, Tushar Khot, and Ashish Sabharwal. 2018. Can a suit of armor conduct electricity? a new dataset for open book question answering. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2381–2391. [find](https://scholar.google.com/scholar?q=Todor+Mihaylov%2C+Peter+Clark%2C+Tushar+Khot%2C+and+Ashish+Sabharwal%2E+2018%2E+Can+a+suit+of+armor+conduct+electricity%3F+a+new+dataset+for+open+book+question+answering%2E+In+Proceedings+of+the+2018+Conference+on+)
121. Sewon Min, Jordan Boyd-Graber, Chris Alberti, Danqi Chen, Eunsol Choi, Michael Collins, Kelvin Guu, Sandra Mitrović, Davide Andreoletti, and Omran Ayoub. 2023. Chatgpt or human? detect and explain. explaining decisions of machine learning model for detecting short chatgpt-generated text. [find](https://scholar.google.com/scholar?q=Sewon+Min%2C+Jordan+Boyd%2DGraber%2C+Chris+Alberti%2C+Danqi+Chen%2C+Eunsol+Choi%2C+Michael+Collins%2C+Kelvin+Guu%2C+Sandra+Mitrovic%CC%81%2C+Davide+Andreoletti%2C+and+Omran+Ayoub%2E+2023%2E+Chatgpt+or+human%3F+detect+and+explain%2E+e)
122. Shashi Narayan, Shay B Cohen, and Mirella Lapata. 2018. Don’t give me the details, just the summary! topic-aware convolutional neural networks for extreme summarization. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 1797–1807. [find](https://scholar.google.com/scholar?q=Shashi+Narayan%2C+Shay+B+Cohen%2C+and+Mirella+Lapata%2E+2018%2E+Don%E2%80%99t+give+me+the+details%2C+just+the+summary%21+topic%2Daware+convolutional+neural+networks+for+extreme+summarization%2E+In+Proceedings+of+the+2018+Con)
123. Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020. Adversarial NLI: A new benchmark for natural language understanding. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4885–4901, Online. Association for Computa… [find](https://scholar.google.com/scholar?q=Yixin+Nie%2C+Adina+Williams%2C+Emily+Dinan%2C+Mohit+Bansal%2C+Jason+Weston%2C+and+Douwe+Kiela%2E+2020%2E+Adversarial+NLI%3A+A+new+benchmark+for+natural+language+understanding%2E+In+Proceedings+of+the+58th+Annual+Meetin)
124. Oluwatosin Ogundare, Srinath Madasu, and Nathanial Wiggins. 2023. Industrial engineering with large language models: A case study of chatgpt’s performance on oil & gas problems. [find](https://scholar.google.com/scholar?q=Oluwatosin+Ogundare%2C+Srinath+Madasu%2C+and+Nathanial+Wiggins%2E+2023%2E+Industrial+engineering+with+large+language+models%3A+A+case+study+of+chatgpt%E2%80%99s+performance+on+oil+%26+gas+problems%2E)
125. OpenAI. 2023. Gpt-4 technical report. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2023%2E+Gpt%2D4+technical+report%2E) [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
126. OpenAI-Blog. 2022. Chatgpt: Optimizing language models for dialogue. [find](https://scholar.google.com/scholar?q=OpenAI%2DBlog%2E+2022%2E+Chatgpt%3A+Optimizing+language+models+for+dialogue%2E)
127. Miguel Ortega-Martín, Óscar García-Sierra, Alfonso Ardoiz, Jorge Álvarez, Juan Carlos Armenteros, and Adrián Alonso. 2023. Linguistic ambiguity analysis in chatgpt. [find](https://scholar.google.com/scholar?q=Miguel+Ortega%2DMart%C3%ADn%2C+%C3%93scar+Garc%C3%ADa%2DSierra%2C+Alfonso+Ardoiz%2C+Jorge+%C3%81lvarez%2C+Juan+Carlos+Armenteros%2C+and+Adri%C3%A1n+Alonso%2E+2023%2E+Linguistic+ambiguity+analysis+in+chatgpt%2E)
128. Pedro Javier Ortiz Suárez, Benoît Sagot, and Laurent Romary. 2019. Asynchronous pipelines for processing huge corpora on medium to low resource infrastructures. Proceedings of the Workshop on Challenges in the Management of Large Corpora (CMLC-7) 2019. Cardiff, 22nd July 2019, pages 9 – 16, Mannheim… [find](https://scholar.google.com/scholar?q=Pedro+Javier+Ortiz+Su%C3%A1rez%2C+Beno%C3%AEt+Sagot%2C+and+Laurent+Romary%2E+2019%2E+Asynchronous+pipelines+for+processing+huge+corpora+on+medium+to+low+resource+infrastructures%2E+Proceedings+of+the+Workshop+on+Challeng)
129. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155. [find](https://scholar.google.com/scholar?q=Long+Ouyang%2C+Jeff+Wu%2C+Xu+Jiang%2C+Diogo+Almeida%2C+Carroll+L+Wainwright%2C+Pamela+Mishkin%2C+Chong+Zhang%2C+Sandhini+Agarwal%2C+Katarina+Slama%2C+Alex+Ray%2C+et+al%2E+2022%2E+Training+language+models+to+follow+instructio)
130. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+annual+meeting+of+the+Association+for+Comp) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
131. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, et al. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(140):1–67. [find](https://scholar.google.com/scholar?q=Colin+Raffel%2C+Noam+Shazeer%2C+Adam+Roberts%2C+Katherine+Lee%2C+Sharan+Narang%2C+Michael+Matena%2C+Yanqi+Zhou%2C+Wei+Li%2C+Peter+J+Liu%2C+et+al%2E+2020%2E+Exploring+the+limits+of+transfer+learning+with+a+unified+text%2Dto%2Dt)
132. Ralph Peeters and Christian Bizer. 2023. Using chatgpt for entity matching. [find](https://scholar.google.com/scholar?q=Ralph+Peeters+and+Christian+Bizer%2E+2023%2E+Using+chatgpt+for+entity+matching%2E)
133. Alessandro Pegoraro, Kavita Kumari, Hossein Ferei-dooni, and Ahmad-Reza Sadeghi. 2023. To chatgpt, or not to chatgpt: That is the question! [find](https://scholar.google.com/scholar?q=Alessandro+Pegoraro%2C+Kavita+Kumari%2C+Hossein+Ferei%2Ddooni%2C+and+Ahmad%2DReza+Sadeghi%2E+2023%2E+To+chatgpt%2C+or+not+to+chatgpt%3A+That+is+the+question%21)
134. Raian Rahman, Rizvi Hasan, Abdullah Al Farhad, Md Tahmid Rahman Laskar, Md Ashmafee, Abu Raihan Mostofa Kamal, et al. 2023. Chartsumm: A comprehensive benchmark for automatic chart summarization of long and short summaries. arXiv preprint arXiv:2304.13620. [find](https://scholar.google.com/scholar?q=Raian+Rahman%2C+Rizvi+Hasan%2C+Abdullah+Al+Farhad%2C+Md+Tahmid+Rahman+Laskar%2C+Md+Ashmafee%2C+Abu+Raihan+Mostofa+Kamal%2C+et+al%2E+2023%2E+Chartsumm%3A+A+comprehensive+benchmark+for+automatic+chart+summarization+of+lo)
135. Keqin Peng, Liang Ding, Qihuang Zhong, Li Shen, Xuebo Liu, Min Zhang, Yuanxin Ouyang, and Dacheng Tao. 2023. Towards making the most of chatgpt for machine translation. [find](https://scholar.google.com/scholar?q=Keqin+Peng%2C+Liang+Ding%2C+Qihuang+Zhong%2C+Li+Shen%2C+Xuebo+Liu%2C+Min+Zhang%2C+Yuanxin+Ouyang%2C+and+Dacheng+Tao%2E+2023%2E+Towards+making+the+most+of+chatgpt+for+machine+translation%2E) [our card](https://md.research.elamur.ai/free/en/papers/peng-2023-making-most-chatgpt-machine-translation/)
136. Pranav Rajpurkar, Robin Jia, and Percy Liang. 2018. Know what you don’t know: Unanswerable questions for squad. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 784–789. [find](https://scholar.google.com/scholar?q=Pranav+Rajpurkar%2C+Robin+Jia%2C+and+Percy+Liang%2E+2018%2E+Know+what+you+don%E2%80%99t+know%3A+Unanswerable+questions+for+squad%2E+In+Proceedings+of+the+56th+Annual+Meeting+of+the+Association+for+Computational+Linguisti) [our card](https://md.research.elamur.ai/free/en/papers/rajpurkar-2018-know-what-you-don-t/)
137. Yifan Peng, Shankai Yan, and Zhiyong Lu. 2019. Transfer learning in biomedical natural language processing: An evaluation of bert and elmo on ten benchmarking datasets. In Proceedings of the 18th BioNLP Workshop and Shared Task, pages 58–65. [find](https://scholar.google.com/scholar?q=Yifan+Peng%2C+Shankai+Yan%2C+and+Zhiyong+Lu%2E+2019%2E+Transfer+learning+in+biomedical+natural+language+processing%3A+An+evaluation+of+bert+and+elmo+on+ten+benchmarking+datasets%2E+In+Proceedings+of+the+18th+BioN) [our card](https://md.research.elamur.ai/free/en/papers/peng-2019-transfer-learning-biomedical-natural-language/)
138. Ethan Perez and Ian McKenzie. Inverse scaling prize: Round 1 winners. [find](https://scholar.google.com/scholar?q=Ethan+Perez+and+Ian+McKenzie%2E+Inverse+scaling+prize%3A+Round+1+winners%2E)
139. Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. 2021. AdapterFusion: Non-destructive task composition for transfer learning. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 487–503… [find](https://scholar.google.com/scholar?q=Jonas+Pfeiffer%2C+Aishwarya+Kamath%2C+Andreas+R%C3%BCckl%C3%A9%2C+Kyunghyun+Cho%2C+and+Iryna+Gurevych%2E+2021%2E+AdapterFusion%3A+Non%2Ddestructive+task+composition+for+transfer+learning%2E+In+Proceedings+of+the+16th+Conference+) [our card](https://md.research.elamur.ai/free/en/papers/pfeiffer-2020-adapterfusion-non-destructive-task-composition/)
140. Mohammad Taher Pilehvar and Jose Camacho-Collados. 2019. WiC: The word-in-context dataset for evaluating context-sensitive meaning representations. In Proceedings of NAACL-HLT. [find](https://scholar.google.com/scholar?q=Mohammad+Taher+Pilehvar+and+Jose+Camacho%2DCollados%2E+2019%2E+WiC%3A+The+word%2Din%2Dcontext+dataset+for+evaluating+context%2Dsensitive+meaning+representations%2E+In+Proceedings+of+NAACL%2DHLT%2E) [our card](https://md.research.elamur.ai/free/en/papers/pilehvar-2019-wic-word-context-dataset-evaluating/)
141. Adam Poliak, Aparajita Haldar, Rachel Rudinger, J. Edward Hu, Ellie Pavlick, Aaron Steven White, and Benjamin Van Durme. 2018. Collecting diverse natural language inference problems for sentence representation evaluation. In Proceedings of EMNLP. [find](https://scholar.google.com/scholar?q=Adam+Poliak%2C+Aparajita+Haldar%2C+Rachel+Rudinger%2C+J%2E+Edward+Hu%2C+Ellie+Pavlick%2C+Aaron+Steven+White%2C+and+Benjamin+Van+Durme%2E+2018%2E+Collecting+diverse+natural+language+inference+problems+for+sentence+repre) [our card](https://md.research.elamur.ai/free/en/papers/poliak-2018-collecting-diverse-natural-language-inference/)
142. Chengwei Qin, Aston Zhang, Zhuosheng Zhang, Jiaao Chen, Michihiro Yasunaga, and Diyi Yang. 2023. Is chatgpt a general-purpose natural language processing task solver? [find](https://scholar.google.com/scholar?q=Chengwei+Qin%2C+Aston+Zhang%2C+Zhuosheng+Zhang%2C+Jiaao+Chen%2C+Michihiro+Yasunaga%2C+and+Diyi+Yang%2E+2023%2E+Is+chatgpt+a+general%2Dpurpose+natural+language+processing+task+solver%3F) [our card](https://md.research.elamur.ai/free/en/papers/qin-2023-chatgpt-general-purpose-natural-language/)
143. Aman Rangapur and Haoran Wang. 2023. Chatgpt-crawler: Find out if chatgpt really knows what it’s talking about. [find](https://scholar.google.com/scholar?q=Aman+Rangapur+and+Haoran+Wang%2E+2023%2E+Chatgpt%2Dcrawler%3A+Find+out+if+chatgpt+really+knows+what+it%E2%80%99s+talking+about%2E)
144. Haocong Rao, Cyril Leung, and Chunyan Miao. 2023. Can chatgpt assess human personalities? a general evaluation framework. [find](https://scholar.google.com/scholar?q=Haocong+Rao%2C+Cyril+Leung%2C+and+Chunyan+Miao%2E+2023%2E+Can+chatgpt+assess+human+personalities%3F+a+general+evaluation+framework%2E)
145. Mathieu Ravaut, Shafiq Joty, and Nancy Chen. 2022. Summareranker: A multi-task mixture-of-experts reranking framework for abstractive summarization. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 4504–4524. [find](https://scholar.google.com/scholar?q=Mathieu+Ravaut%2C+Shafiq+Joty%2C+and+Nancy+Chen%2E+2022%2E+Summareranker%3A+A+multi%2Dtask+mixture%2Dof%2Dexperts+reranking+framework+for+abstractive+summarization%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+As)
146. Melissa Roemmele, Cosmin Adrian Bejan, and Andrew S. Gordon. 2011. Choice of plausible alternatives: An evaluation of commonsense causal reasoning. In 2011 AAAI Spring Symposium Series. [find](https://scholar.google.com/scholar?q=Melissa+Roemmele%2C+Cosmin+Adrian+Bejan%2C+and+Andrew+S%2E+Gordon%2E+2011%2E+Choice+of+plausible+alternatives%3A+An+evaluation+of+commonsense+causal+reasoning%2E+In+2011+AAAI+Spring+Symposium+Series%2E)
147. Sayak Saha Roy, Krishna Vamsi Naragam, and Shirin Nilizadeh. 2023. Generating phishing attacks using chatgpt. [find](https://scholar.google.com/scholar?q=Sayak+Saha+Roy%2C+Krishna+Vamsi+Naragam%2C+and+Shirin+Nilizadeh%2E+2023%2E+Generating+phishing+attacks+using+chatgpt%2E)
148. Rachel Rudinger, Jason Naradowsky, Brian Leonard, and Benjamin Van Durme. 2018. Gender bias in coreference resolution. In Proceedings of NAACL-HLT. [find](https://scholar.google.com/scholar?q=Rachel+Rudinger%2C+Jason+Naradowsky%2C+Brian+Leonard%2C+and+Benjamin+Van+Durme%2E+2018%2E+Gender+bias+in+coreference+resolution%2E+In+Proceedings+of+NAACL%2DHLT%2E) [our card](https://md.research.elamur.ai/free/en/papers/rudinger-2018-gender-bias-coreference-resolution/)
149. Keisuke Sakaguchi, Ronan Le Bras, Chandra Bhagavatula, and Yejin Choi. 2020. Winogrande: An adversarial winograd schema challenge at scale. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 34, pages 8732–8740. [find](https://scholar.google.com/scholar?q=Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Chandra+Bhagavatula%2C+and+Yejin+Choi%2E+2020%2E+Winogrande%3A+An+adversarial+winograd+schema+challenge+at+scale%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intellig)
150. Michael Sandel. 2019. The moral side of murder. [find](https://scholar.google.com/scholar?q=Michael+Sandel%2E+2019%2E+The+moral+side+of+murder%2E)
151. Alec Radford, Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever, et al. 2019. Language models are unsupervised multitask learners. OpenAI blog, 1(8):9. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Jeffrey+Wu%2C+Rewon+Child%2C+David+Luan%2C+Dario+Amodei%2C+Ilya+Sutskever%2C+et+al%2E+2019%2E+Language+models+are+unsupervised+multitask+learners%2E+OpenAI+blog%2C+1%288%29%3A9%2E)
152. Victor Sanh, Lysandre Debut, Julien Chaumond, and Thomas Wolf. 2019. Distilbert, a distilled version of bert: smaller, faster, cheaper and lighter. arXiv preprint arXiv:1910.01108. [find](https://scholar.google.com/scholar?q=Victor+Sanh%2C+Lysandre+Debut%2C+Julien+Chaumond%2C+and+Thomas+Wolf%2E+2019%2E+Distilbert%2C+a+distilled+version+of+bert%3A+smaller%2C+faster%2C+cheaper+and+lighter%2E+arXiv+preprint+arXiv%3A1910%2E01108%2E)
153. Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. 2021. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446. [find](https://scholar.google.com/scholar?q=Jack+W+Rae%2C+Sebastian+Borgeaud%2C+Trevor+Cai%2C+Katie+Millican%2C+Jordan+Hoffmann%2C+Francis+Song%2C+John+Aslanides%2C+Sarah+Henderson%2C+Roman+Ring%2C+Susannah+Young%2C+et+al%2E+2021%2E+Scaling+language+models%3A+Methods%2C+a)
154. Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. 2021. Multitask prompted training enables zero-shot task generalization. arXiv preprint arXiv:2110.08207. [find](https://scholar.google.com/scholar?q=Victor+Sanh%2C+Albert+Webson%2C+Colin+Raffel%2C+Stephen+H+Bach%2C+Lintang+Sutawika%2C+Zaid+Alyafeai%2C+Antoine+Chaffin%2C+Arnaud+Stiegler%2C+Teven+Le+Scao%2C+Arun+Raja%2C+et+al%2E+2021%2E+Multitask+prompted+training+enables+) [our card](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)
155. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. 2019. Social iqa: Commonsense reasoning about social interactions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processi… [find](https://scholar.google.com/scholar?q=Maarten+Sap%2C+Hannah+Rashkin%2C+Derek+Chen%2C+Ronan+Le+Bras%2C+and+Yejin+Choi%2E+2019%2E+Social+iqa%3A+Commonsense+reasoning+about+social+interactions%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in)
156. Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. 2022. Bloom: A 176b-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100. [find](https://scholar.google.com/scholar?q=Teven+Le+Scao%2C+Angela+Fan%2C+Christopher+Akiki%2C+Ellie+Pavlick%2C+Suzana+Ilic%CC%81%2C+Daniel+Hesslow%2C+Roman+Castagn%C3%A9%2C+Alexandra+Sasha+Luccioni%2C+Fran%C3%A7ois+Yvon%2C+Matthias+Gall%C3%A9%2C+et+al%2E+2022%2E+Bloom%3A+A+176b%2Dparameter)
157. Abigail See, Peter J. Liu, and Christopher D. Manning. 2017. Get to the point: Summarization with pointer-generator networks. In Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1073– 1083, Vancouver, Canada. Association for Compu… [find](https://scholar.google.com/scholar?q=Abigail+See%2C+Peter+J%2E+Liu%2C+and+Christopher+D%2E+Manning%2E+2017%2E+Get+to+the+point%3A+Summarization+with+pointer%2Dgenerator+networks%2E+In+Proceedings+of+the+55th+Annual+Meeting+of+the+Association+for+Computati)
158. Xinyue Shen, Zeyuan Chen, Michael Backes, and Yang Zhang. 2023. In chatgpt we trust? measuring and characterizing the reliability of chatgpt. [find](https://scholar.google.com/scholar?q=Xinyue+Shen%2C+Zeyuan+Chen%2C+Michael+Backes%2C+and+Yang+Zhang%2E+2023%2E+In+chatgpt+we+trust%3F+measuring+and+characterizing+the+reliability+of+chatgpt%2E)
159. Freda Shi, Mirac Suzgun, Markus Freitag, Xuezhi Wang, Suraj Srivats, Soroush Vosoughi, Hyung Won Chung, Yi Tay, Sebastian Ruder, Denny Zhou, et al. 2022. Language models are multilingual chain-of-thought reasoners. arXiv preprint arXiv:2210.03057. [find](https://scholar.google.com/scholar?q=Freda+Shi%2C+Mirac+Suzgun%2C+Markus+Freitag%2C+Xuezhi+Wang%2C+Suraj+Srivats%2C+Soroush+Vosoughi%2C+Hyung+Won+Chung%2C+Yi+Tay%2C+Sebastian+Ruder%2C+Denny+Zhou%2C+et+al%2E+2022%2E+Language+models+are+multilingual+chain%2Dof%2Dthou) [our card](https://md.research.elamur.ai/free/en/papers/shi-2022-language-models-multilingual-chain-thought/)
160. Zhou, et al. 2022. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261. [find](https://scholar.google.com/scholar?q=Zhou%2C+et+al%2E+2022%2E+Challenging+big%2Dbench+tasks+and+whether+chain%2Dof%2Dthought+can+solve+them%2E+arXiv+preprint+arXiv%3A2210%2E09261%2E) [our card](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/)
161. Pittawat Taveekitworachai, Febri Abdullah, Mury F. De-wantoro, Ruck Thawonmas, Julian Togelius, and Jochen Renz. 2023. Chatgpt4pcg competition: Character-like level generation for science birds. [find](https://scholar.google.com/scholar?q=Pittawat+Taveekitworachai%2C+Febri+Abdullah%2C+Mury+F%2E+De%2Dwantoro%2C+Ruck+Thawonmas%2C+Julian+Togelius%2C+and+Jochen+Renz%2E+2023%2E+Chatgpt4pcg+competition%3A+Character%2Dlike+level+generation+for+science+birds%2E)
162. Yi Tay, Mostafa Dehghani, Vinh Q Tran, Xavier Garcia, Dara Bahri, Tal Schuster, Huaixiu Steven Zheng, Neil Houlsby, and Donald Metzler. 2022. Unifying language learning paradigms. arXiv preprint arXiv:2205.05131. [find](https://scholar.google.com/scholar?q=Yi+Tay%2C+Mostafa+Dehghani%2C+Vinh+Q+Tran%2C+Xavier+Garcia%2C+Dara+Bahri%2C+Tal+Schuster%2C+Huaixiu+Steven+Zheng%2C+Neil+Houlsby%2C+and+Donald+Metzler%2E+2022%2E+Unifying+language+learning+paradigms%2E+arXiv+preprint+arXiv) [our card](https://md.research.elamur.ai/free/en/papers/tay-2022-ul2-unifying-language-learning-paradigms/)
163. Judith Jarvis Thomson. 2020. The Trolley Problem/Das Trolley-Problem (Englisch/Deutsch): Reclam Great Papers Philosophie. Reclam Verlag. [find](https://scholar.google.com/scholar?q=Judith+Jarvis+Thomson%2E+2020%2E+The+Trolley+Problem%2FDas+Trolley%2DProblem+%28Englisch%2FDeutsch%29%3A+Reclam+Great+Papers+Philosophie%2E+Reclam+Verlag%2E)
164. Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, et al. 2022. Lamda: Language models for dialog applications. arXiv preprint arXiv:2201.08239. [find](https://scholar.google.com/scholar?q=Romal+Thoppilan%2C+Daniel+De+Freitas%2C+Jamie+Hall%2C+Noam+Shazeer%2C+Apoorv+Kulshreshtha%2C+Heng%2DTze+Cheng%2C+Alicia+Jin%2C+Taylor+Bos%2C+Leslie+Baker%2C+Yu+Du%2C+et+al%2E+2022%2E+Lamda%3A+Language+models+for+dialog+applicati) [our card](https://md.research.elamur.ai/free/en/papers/thoppilan-2022-lamda-language-models-dialog-applications/)
165. Erik F Tjong Kim Sang and Fien De Meulder. 2003. Introduction to the conll-2003 shared task: language-independent named entity recognition. In Proceedings of the seventh conference on Natural language learning at HLT-NAACL 2003-Volume 4, pages 142– 147. [find](https://scholar.google.com/scholar?q=Erik+F+Tjong+Kim+Sang+and+Fien+De+Meulder%2E+2003%2E+Introduction+to+the+conll%2D2003+shared+task%3A+language%2Dindependent+named+entity+recognition%2E+In+Proceedings+of+the+seventh+conference+on+Natural+language)
166. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. 2023. Llama: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971. [find](https://scholar.google.com/scholar?q=Hugo+Touvron%2C+Thibaut+Lavril%2C+Gautier+Izacard%2C+Xavier+Martinet%2C+Marie%2DAnne+Lachaux%2C+Timoth%C3%A9e+Lacroix%2C+Baptiste+Rozi%C3%A8re%2C+Naman+Goyal%2C+Eric+Hambro%2C+Faisal+Azhar%2C+et+al%2E+2023%2E+Llama%3A+Open+and+efficient+f)
167. Aarohi Srivastava, Abhinav Rastogi, Abhishek Rao, Abu Awal Md Shoeb, Abubakar Abid, Adam Fisch, Adam R Brown, Adam Santoro, Aditya Gupta, Adrià Garriga-Alonso, et al. 2022. Beyond the imitation game: Quantifying and extrapolating the capabilities of language models. arXiv preprint arXiv:2206.04615. [find](https://scholar.google.com/scholar?q=Aarohi+Srivastava%2C+Abhinav+Rastogi%2C+Abhishek+Rao%2C+Abu+Awal+Md+Shoeb%2C+Abubakar+Abid%2C+Adam+Fisch%2C+Adam+R+Brown%2C+Adam+Santoro%2C+Aditya+Gupta%2C+Adri%C3%A0+Garriga%2DAlonso%2C+et+al%2E+2022%2E+Beyond+the+imitation+game%3A+) [our card](https://md.research.elamur.ai/free/en/papers/srivastava-2022-imitation-game-quantifying-extrapolating-capabilities/)
168. Yixuan Su, Fangyu Liu, Zaiqiao Meng, Tian Lan, Lei Shu, Ehsan Shareghi, and Nigel Collier. 2022. TaCL: Improving BERT pre-training with token-aware contrastive learning. In Findings of the Association for Computational Linguistics: NAACL 2022, pages 2497–2507, Seattle, United States. Association for… [find](https://scholar.google.com/scholar?q=Yixuan+Su%2C+Fangyu+Liu%2C+Zaiqiao+Meng%2C+Tian+Lan%2C+Lei+Shu%2C+Ehsan+Shareghi%2C+and+Nigel+Collier%2E+2022%2E+TaCL%3A+Improving+BERT+pre%2Dtraining+with+token%2Daware+contrastive+learning%2E+In+Findings+of+the+Association)
169. Tianxiang Sun, Yunfan Shao, Hong Qian, Xuanjing Huang, and Xipeng Qiu. 2022. Black-box tuning for language-model-as-a-service. In ICML, volume 162 of Proceedings of Machine Learning Research, pages 20841–20855. PMLR. [find](https://scholar.google.com/scholar?q=Tianxiang+Sun%2C+Yunfan+Shao%2C+Hong+Qian%2C+Xuanjing+Huang%2C+and+Xipeng+Qiu%2E+2022%2E+Black%2Dbox+tuning+for+language%2Dmodel%2Das%2Da%2Dservice%2E+In+ICML%2C+volume+162+of+Proceedings+of+Machine+Learning+Research%2C+pages+20)
170. Shangqing Tu, Chunyang Li, Jifan Yu, Xiaozhi Wang, Lei Hou, and Juanzi Li. 2023. Chatlog: Recording and analyzing chatgpt across time. [find](https://scholar.google.com/scholar?q=Shangqing+Tu%2C+Chunyang+Li%2C+Jifan+Yu%2C+Xiaozhi+Wang%2C+Lei+Hou%2C+and+Juanzi+Li%2E+2023%2E+Chatlog%3A+Recording+and+analyzing+chatgpt+across+time%2E)
171. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. Attention is all you need. Advances in neural information processing systems, 30. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+2017%2E+Attention+is+all+you+need%2E+Advances+in+neural+information+processing+)
172. Alex Wang, Yada Pruksachatkun, Nikita Nangia, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2019. Superglue: A stickier benchmark for general-purpose language understanding systems. Advances in neural information processing systems, 32. [find](https://scholar.google.com/scholar?q=Alex+Wang%2C+Yada+Pruksachatkun%2C+Nikita+Nangia%2C+Amanpreet+Singh%2C+Julian+Michael%2C+Felix+Hill%2C+Omer+Levy%2C+and+Samuel+Bowman%2E+2019%2E+Superglue%3A+A+stickier+benchmark+for+general%2Dpurpose+language+understandin) [our card](https://md.research.elamur.ai/free/en/papers/wang-2019-superglue-stickier-benchmark-general-purpose/)
173. Weiwei Sun, Lingyong Yan, Xinyu Ma, Pengjie Ren, Dawei Yin, and Zhaochun Ren. 2023. Is chatgpt good at search? investigating large language models as re-ranking agent. [find](https://scholar.google.com/scholar?q=Weiwei+Sun%2C+Lingyong+Yan%2C+Xinyu+Ma%2C+Pengjie+Ren%2C+Dawei+Yin%2C+and+Zhaochun+Ren%2E+2023%2E+Is+chatgpt+good+at+search%3F+investigating+large+language+models+as+re%2Dranking+agent%2E)
174. Teo Susnjak. 2022. Chatgpt: The end of online exam integrity? [find](https://scholar.google.com/scholar?q=Teo+Susnjak%2E+2022%2E+Chatgpt%3A+The+end+of+online+exam+integrity%3F)
175. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel Bowman. 2018. GLUE: A multi-task benchmark and analysis platform for natural lang… [find](https://scholar.google.com/scholar?q=Mirac+Suzgun%2C+Nathan+Scales%2C+Nathanael+Sch%C3%A4rli%2C+Sebastian+Gehrmann%2C+Yi+Tay%2C+Hyung+Won+Chung%2C+Aakanksha+Chowdhery%2C+Quoc+V+Le%2C+Ed+H+Chi%2C+Denny+Alex+Wang%2C+Amanpreet+Singh%2C+Julian+Michael%2C+Felix+Hill%2C+Ome) [our card](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/)
176. Ben Wang. 2021. Mesh-Transformer-JAX: Model-Parallel Implementation of Transformer Language Model with JAX. [https://github.com/](https://github.com/) kingoflolz/mesh-transformer-jax. [↗](https://github.com/)
177. Bowen Zhang, Daijun Ding, and Liwen Jing. 2023. How would stance detection techniques evolve after the launch of chatgpt? [find](https://scholar.google.com/scholar?q=Bowen+Zhang%2C+Daijun+Ding%2C+and+Liwen+Jing%2E+2023%2E+How+would+stance+detection+techniques+evolve+after+the+launch+of+chatgpt%3F)
178. Jiaan Wang, Yunlong Liang, Fandong Meng, Zengkui Sun, Haoxiang Shi, Zhixu Li, Jinan Xu, Jianfeng Qu, and Jie Zhou. 2023a. Is chatgpt a good nlg evaluator? a preliminary study. [find](https://scholar.google.com/scholar?q=Jiaan+Wang%2C+Yunlong+Liang%2C+Fandong+Meng%2C+Zengkui+Sun%2C+Haoxiang+Shi%2C+Zhixu+Li%2C+Jinan+Xu%2C+Jianfeng+Qu%2C+and+Jie+Zhou%2E+2023a%2E+Is+chatgpt+a+good+nlg+evaluator%3F+a+preliminary+study%2E) [our card](https://md.research.elamur.ai/free/en/papers/wang-2023-chatgpt-good-nlg-evaluator-preliminary/)
179. Sheng Zhang, Xiaodong Liu, Jingjing Liu, Jianfeng Gao, Kevin Duh, and Benjamin Van Durme. 2018. ReCoRD: Bridging the gap between human and machine commonsense reading comprehension. arXiv preprint 1810.12885. [find](https://scholar.google.com/scholar?q=Sheng+Zhang%2C+Xiaodong+Liu%2C+Jingjing+Liu%2C+Jianfeng+Gao%2C+Kevin+Duh%2C+and+Benjamin+Van+Durme%2E+2018%2E+ReCoRD%3A+Bridging+the+gap+between+human+and+machine+commonsense+reading+comprehension%2E+arXiv+preprint+181) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2018-record-bridging-gap-between-human/)
180. Yizhong Wang, Swaroop Mishra, Pegah Alipoormolabashi, Yeganeh Kordi, Amirreza Mirzaei, Anjana Arunkumar, Arjun Ashok, Arut Selvan Dhanasekaran, Atharva Naik, David Stap, et al. 2022. Super-naturalinstructions:generalization via declarative instructions on 1600+ tasks. In EMNLP. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Swaroop+Mishra%2C+Pegah+Alipoormolabashi%2C+Yeganeh+Kordi%2C+Amirreza+Mirzaei%2C+Anjana+Arunkumar%2C+Arjun+Ashok%2C+Arut+Selvan+Dhanasekaran%2C+Atharva+Naik%2C+David+Stap%2C+et+al%2E+2022%2E+Super%2Dnaturalinst)
181. Yizhe Zhang, Siqi Sun, Michel Galley, Yen-Chun Chen, Chris Brockett, Xiang Gao, Jianfeng Gao, Jingjing Liu, and Bill Dolan. 2020. Dialogpt: Large-scale generative pre-training for conversational response generation. In ACL, system demonstration. [find](https://scholar.google.com/scholar?q=Yizhe+Zhang%2C+Siqi+Sun%2C+Michel+Galley%2C+Yen%2DChun+Chen%2C+Chris+Brockett%2C+Xiang+Gao%2C+Jianfeng+Gao%2C+Jingjing+Liu%2C+and+Bill+Dolan%2E+2020%2E+Dialogpt%3A+Large%2Dscale+generative+pre%2Dtraining+for+conversational+respo) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2019-dialogpt-large-scale-generative-pre/)
182. Zengzhi Wang, Qiming Xie, Zixiang Ding, Yi Feng, and Rui Xia. 2023b. Is chatgpt a good sentiment analyzer? a preliminary study. [find](https://scholar.google.com/scholar?q=Zengzhi+Wang%2C+Qiming+Xie%2C+Zixiang+Ding%2C+Yi+Feng%2C+and+Rui+Xia%2E+2023b%2E+Is+chatgpt+a+good+sentiment+analyzer%3F+a+preliminary+study%2E)
183. Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, and Kai-Wei Chang. 2018. Gender bias in coreference resolution: Evaluation and debiasing methods. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies,… [find](https://scholar.google.com/scholar?q=Jieyu+Zhao%2C+Tianlu+Wang%2C+Mark+Yatskar%2C+Vicente+Ordonez%2C+and+Kai%2DWei+Chang%2E+2018%2E+Gender+bias+in+coreference+resolution%3A+Evaluation+and+debiasing+methods%2E+In+Proceedings+of+the+2018+Conference+of+the+N) [our card](https://md.research.elamur.ai/free/en/papers/zhao-2018-gender-bias-coreference-resolution-evaluation/)
184. Jason Wei, Maarten Bosma, Vincent Y Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M Dai, and Quoc V Le. 2021. Finetuned language models are zero-shot learners. arXiv preprint arXiv:2109.01652. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Maarten+Bosma%2C+Vincent+Y+Zhao%2C+Kelvin+Guu%2C+Adams+Wei+Yu%2C+Brian+Lester%2C+Nan+Du%2C+Andrew+M+Dai%2C+and+Quoc+V+Le%2E+2021%2E+Finetuned+language+models+are+zero%2Dshot+learners%2E+arXiv+preprint+arXiv%3A2109)
185. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. 2022a. Emergent abilities of large language models. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Yi+Tay%2C+Rishi+Bommasani%2C+Colin+Raffel%2C+Barret+Zoph%2C+Sebastian+Borgeaud%2C+Dani+Yogatama%2C+Maarten+Bosma%2C+Denny+Zhou%2C+Donald+Metzler%2C+Ed+H%2E+Chi%2C+Tatsunori+Hashimoto%2C+Oriol+Vinyals%2C+Percy+Liang%2C) [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-emergent-abilities-large-language-models/)
186. Jason Wei, Yi Tay, and Quoc V Le. 2022b. Inverse scaling can become u-shaped. arXiv preprint arXiv:2211.02011. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Yi+Tay%2C+and+Quoc+V+Le%2E+2022b%2E+Inverse+scaling+can+become+u%2Dshaped%2E+arXiv+preprint+arXiv%3A2211%2E02011%2E)
187. Haoran Wu, Wenxuan Wang, Yuxuan Wan, Wenxiang Jiao, and Michael Lyu. 2023. Chatgpt or grammarly? evaluating chatgpt on grammatical error correction benchmark. [find](https://scholar.google.com/scholar?q=Haoran+Wu%2C+Wenxuan+Wang%2C+Yuxuan+Wan%2C+Wenxiang+Jiao%2C+and+Michael+Lyu%2E+2023%2E+Chatgpt+or+grammarly%3F+evaluating+chatgpt+on+grammatical+error+correction+benchmark%2E)
188. Ledell Wu, Fabio Petroni, Martin Josifoski, Sebastian Riedel, and Luke Zettlemoyer. 2020. Scalable zero-shot entity linking with dense entity retrieval. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 6397–6407. [find](https://scholar.google.com/scholar?q=Ledell+Wu%2C+Fabio+Petroni%2C+Martin+Josifoski%2C+Sebastian+Riedel%2C+and+Luke+Zettlemoyer%2E+2020%2E+Scalable+zero%2Dshot+entity+linking+with+dense+entity+retrieval%2E+In+Proceedings+of+the+2020+Conference+on+Empiri) [our card](https://md.research.elamur.ai/free/en/papers/wu-2020-scalable-zero-shot-entity-linking/)
189. Chenhan Yuan, Qianqian Xie, and Sophia Ananiadou. 2023. Zero-shot temporal relation extraction with chatgpt. [find](https://scholar.google.com/scholar?q=Chenhan+Yuan%2C+Qianqian+Xie%2C+and+Sophia+Ananiadou%2E+2023%2E+Zero%2Dshot+temporal+relation+extraction+with+chatgpt%2E)
190. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4791–4800. [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+2019%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Com)
191. Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, et al. 2022. Glm-130b: An open bilingual pre-trained model. arXiv preprint arXiv:2210.02414. [find](https://scholar.google.com/scholar?q=Aohan+Zeng%2C+Xiao+Liu%2C+Zhengxiao+Du%2C+Zihan+Wang%2C+Hanyu+Lai%2C+Ming+Ding%2C+Zhuoyi+Yang%2C+Yifan+Xu%2C+Wendi+Zheng%2C+Xiao+Xia%2C+et+al%2E+2022%2E+Glm%2D130b%3A+An+open+bilingual+pre%2Dtrained+model%2E+arXiv+preprint+arXiv%3A221) [our card](https://md.research.elamur.ai/free/en/papers/zeng-2022-glm-130b-open-bilingual-pre/)
192. Weixiang Zhao, Yanyan Zhao, Xin Lu, Shilong Wang, Yanpeng Tong, and Bing Qin. 2023. Is chatgpt equipped with emotional dialogue capabilities? [find](https://scholar.google.com/scholar?q=Weixiang+Zhao%2C+Yanyan+Zhao%2C+Xin+Lu%2C+Shilong+Wang%2C+Yanpeng+Tong%2C+and+Bing+Qin%2E+2023%2E+Is+chatgpt+equipped+with+emotional+dialogue+capabilities%3F)
193. Ming Zhong, Da Yin, Tao Yu, Ahmad Zaidi, Mutethia Mutuma, Rahul Jha, Ahmed Hassan Awadallah, Asli Celikyilmaz, Yang Liu, Xipeng Qiu, and Dragomir Radev. 2021. QMSum: A new benchmark for query-based multi-domain meeting summarization. In Proceedings of the 2021 Conference of the North American Chapte… [find](https://scholar.google.com/scholar?q=Ming+Zhong%2C+Da+Yin%2C+Tao+Yu%2C+Ahmad+Zaidi%2C+Mutethia+Mutuma%2C+Rahul+Jha%2C+Ahmed+Hassan+Awadallah%2C+Asli+Celikyilmaz%2C+Yang+Liu%2C+Xipeng+Qiu%2C+and+Dragomir+Radev%2E+2021%2E+QMSum%3A+A+new+benchmark+for+query%2Dbased+mu) [our card](https://md.research.elamur.ai/free/en/papers/zhong-2021-qmsum-new-benchmark-query-based/)
194. Qihuang Zhong, Liang Ding, Juhua Liu, Bo Du, and Dacheng Tao. 2023. Can chatgpt understand too? a comparative study on chatgpt and fine-tuned bert. arXiv preprint arXiv:2302.10198. — [arXiv:2302.10198](https://arxiv.org/abs/2302.10198) [find](https://scholar.google.com/scholar?q=Qihuang+Zhong%2C+Liang+Ding%2C+Juhua+Liu%2C+Bo+Du%2C+and+Dacheng+Tao%2E+2023%2E+Can+chatgpt+understand+too%3F+a+comparative+study+on+chatgpt+and+fine%2Dtuned+bert%2E+arXiv+preprint+arXiv%3A2302%2E10198%2E+%E2%80%94+%5BarXiv%3A2302%2E10198)
195. Ce Zhou, Qian Li, Chen Li, Jun Yu, Yixin Liu, Guangjing Wang, Kai Zhang, Cheng Ji, Qiben Yan, Lifang He, et al. 2023. A comprehensive survey on pretrained foundation models: A history from bert to chatgpt. arXiv preprint arXiv:2302.09419. — [arXiv:2302.09419](https://arxiv.org/abs/2302.09419) [find](https://scholar.google.com/scholar?q=Ce+Zhou%2C+Qian+Li%2C+Chen+Li%2C+Jun+Yu%2C+Yixin+Liu%2C+Guangjing+Wang%2C+Kai+Zhang%2C+Cheng+Ji%2C+Qiben+Yan%2C+Lifang+He%2C+et+al%2E+2023%2E+A+comprehensive+survey+on+pretrained+foundation+models%3A+A+history+from+bert+to+cha)