---
title: "Qwen Technical Report — Bai 2023"
description: "This report from the Alibaba team presents Qwen, the company's first series of large language models."
authors: ["Jinze Bai", "Shuai Bai", "Yunfei Chu", "Zeyu Cui", "Kai Dang", "Xiaodong Deng", "Yang Fan", "Wenbin Ge", "Yu Han", "Fei Huang", "Binyuan Hui", "Luo Ji", "Mei Li", "Junyang Lin", "Runji Lin", "Dayiheng Liu", "Gao Liu", "Chengqiang Lu", "Keming Lu", "Jianxin Ma", "Rui Men", "Xingzhang Ren", "Xuancheng Ren", "Chuanqi Tan", "Sinan Tan", "Jianhong Tu", "Peng Wang", "Shijie Wang", "Wei Wang", "Shengguang Wu", "Benfeng Xu", "Jin Xu", "An Yang", "Hao Yang", "Jian Yang", "Shusheng Yang", "Yang Yao", "Bowen Yu", "Hongyi Yuan", "Zheng Yuan", "Jianwei Zhang", "Xingxuan Zhang", "Yichang Zhang", "Zhenru Zhang", "Chang Zhou", "Jingren Zhou", "Xiaohuan Zhou", "Tianhang Zhu"]
year: 2023
venue: "arXiv:2309.16609"
source: "https://arxiv.org/abs/2309.16609"
doi: "10.48550/arxiv.2309.16609"
arxiv: "2309.16609"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/bai-2023-qwen-technical-report/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/6/) / Qwen Technical Report

# Qwen Technical Report

**Authors:** [Jinze Bai](https://md.research.elamur.ai/free/en/authors/jinze-bai/), [Shuai Bai](https://md.research.elamur.ai/free/en/authors/shuai-bai/), [Yunfei Chu](https://md.research.elamur.ai/free/en/authors/yunfei-chu/), [Zeyu Cui](https://md.research.elamur.ai/free/en/authors/zeyu-cui/), [Kai Dang](https://md.research.elamur.ai/free/en/authors/kai-dang/), Xiaodong Deng, [Yang Fan](https://md.research.elamur.ai/free/en/authors/yang-fan/), [Wenbin Ge](https://md.research.elamur.ai/free/en/authors/wenbin-ge/), Yu Han, [Fei Huang](https://md.research.elamur.ai/free/en/authors/fei-huang/), [Binyuan Hui](https://md.research.elamur.ai/free/en/authors/binyuan-hui/), Luo Ji et al. (48 authors) · **Year:** 2023 · **Venue:** arXiv:2309.16609 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2309.16609) · [PDF](https://arxiv.org/pdf/2309.16609) · [DOI 10.48550/arxiv.2309.16609](https://doi.org/10.48550/arxiv.2309.16609) · [arXiv 2309.16609](https://arxiv.org/abs/2309.16609) · [OpenAlex](https://openalex.org/W4387225871)

**Citations:** 108 citations (openalex-snapshot · 2026-10-14) · [1535](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
This report from the Alibaba team presents Qwen, the company's first series of large language models. The authors aim to build open LLMs that developers can easily reproduce, fine-tune and deploy, and that can at the same time compete with larger systems. The base Qwen models were pretrained on a multilingual corpus of up to 3 trillion tokens. The corpus consists mainly of English and Chinese texts: web pages, encyclopedias, books and program code, each cleaned separately. The models were then aligned with human preferences, first through supervised finetuning and then through RLHF with a separate reward model. This produced Qwen-Chat, which can use tools and a code interpreter and can act as an agent. The specialized Code-Qwen and Math-Qwen-Chat were built on the same base. Qwen-14B clearly outperforms previous open models of similar size but still lags behind GPT-3.5 and GPT-4. The work's practical value is that the weights of the 7B and 14B models have been released openly, so they can be further trained for one's own tasks.

## Key points
- The base models Qwen-7B and Qwen-14B and the corresponding chat versions Qwen-Chat have been released openly (repository: github.com/QwenLM/Qwen).
- On 12 datasets (language understanding, knowledge, reasoning), Qwen-14B substantially outperforms the previous SOTA among models of about 13B. In human evaluation, the RLHF version proved competitive but still inferior to GPT-4.
- Code-Qwen (7B, 14B) and its chat variants were tested on HumanEval, MBPP and HumanEvalPack. Among open models, they showed a high level of code generation and understanding.
- Math-Qwen-Chat (7B, 14B) outperforms open models of the same size on GSM8K and MATH by a wide margin and approaches GPT-3.5.
- When working with a CSV file via ReAct, Qwen-Chat first checks which columns the file contains and only then builds a plot. Code Llama, by contrast, refers to columns that do not exist in the file. The multimodal Qwen-VL and Qwen-VL-Chat were built on the same base models.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [AI & machine learning](https://md.research.elamur.ai/free/en/shelves/ai/)

## Cited by

Cited by

1535 papers of our library cite it (of the 140884 whose bibliographies we read); 76 of them have a page here, 1459 not yet.

- [Yi: Open Foundation Models by 01.AI](https://md.research.elamur.ai/free/en/papers/ai-2024-yi-open-foundation-models-01/)
- [SmolLM2: When Smol Goes Big -- Data-Centric Training of a Small Language Model](https://md.research.elamur.ai/free/en/papers/allal-2025-smollm2-smol-goes-big-data/)
- [Tower: An Open Multilingual Large Language Model for Translation-Related Tasks](https://md.research.elamur.ai/free/en/papers/alves-2024-tower-open-multilingual-large-language/)
- [Arabic Stable LM: Adapting Stable LM 2 1.6B to Arabic](https://md.research.elamur.ai/free/en/papers/alyafeai-2024-arabic-stable-lm-adapting-stable/)
- [Cutting Through the Noise: Boosting LLM Performance on Math Word Problems](https://md.research.elamur.ai/free/en/papers/anantheswaran-2024-cutting-through-noise-boosting-llm/)
- [Refusal in Language Models Is Mediated by a Single Direction](https://md.research.elamur.ai/free/en/papers/arditi-2024-refusal-language-models-mediated-single/)
- [ResearchAgent: Iterative Research Idea Generation over Scientific Literature with Large Language Models](https://md.research.elamur.ai/free/en/papers/baek-2024-researchagent-iterative-research-idea/)
- [InternLM2 Technical Report](https://md.research.elamur.ai/free/en/papers/cai-2024-internlm2-technical-report/)
- [InternVL: Scaling up Vision Foundation Models and Aligning for Generic Visual-Linguistic Tasks](https://md.research.elamur.ai/free/en/papers/chen-2023-internvl-scaling-up-vision/)
- [ShareGPT4V: Improving Large Multi-Modal Models with Better Captions](https://md.research.elamur.ai/free/en/papers/chen-2023-sharegpt4v-improving-large-multi-modal/)
- [How Far Are We to GPT-4V? Closing the Gap to Commercial Multimodal Models with Open-Source Suites](https://md.research.elamur.ai/free/en/papers/chen-2024-far-gpt-4v-closing/)
- [Are We on the Right Way for Evaluating Large Vision-Language Models?](https://md.research.elamur.ai/free/en/papers/chen-2024-right-way-evaluating-large-vision/)
- [Benchmarking LLMs for Translating Classical Chinese Poetry: Evaluating Adequacy, Fluency, and Elegance](https://md.research.elamur.ai/free/en/papers/chen-2025-benchmarking-llms-translating-classical-chinese/)
- [Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models](https://md.research.elamur.ai/free/en/papers/chu-2023-qwen-audio-advancing-universal/)
- [Qwen2-Audio Technical Report](https://md.research.elamur.ai/free/en/papers/chu-2024-qwen2-audio-technical-report/)
- [Chatlaw: A Multi-Agent Legal Assistant based on a Role-Aligned Mixture-of-Experts Architecture](https://md.research.elamur.ai/free/en/papers/cui-2023-chatlaw-multi-agent-legal-assistant/)
- [A Survey on In-context Learning](https://md.research.elamur.ai/free/en/papers/dong-2023-survey-context-learning/)
- [ArabiGEE: A Hierarchical Taxonomy for Arabic Grammatical Error Explanation](https://md.research.elamur.ai/free/en/papers/elhady-2026-arabigee-hierarchical-taxonomy-arabic-grammatical/)
- [Data Engineering for Scaling Language Models to 128K Context](https://md.research.elamur.ai/free/en/papers/fu-2024-data-engineering-scaling-language-models/)
- [Bridging Cultures in the Kitchen: A Framework and Benchmark for Cross-Cultural Recipe Retrieval](https://md.research.elamur.ai/free/en/papers/hu-2024-bridging-cultures-kitchen-framework-benchmark/)
- [MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies](https://md.research.elamur.ai/free/en/papers/hu-2024-minicpm-unveiling-potential-small-language/)
- [Qwen2.5-Coder Technical Report](https://md.research.elamur.ai/free/en/papers/hui-2024-qwen2-5-coder-technical-report/)
- [MInference 1.0: Accelerating Pre-filling for Long-Context LLMs via Dynamic Sparse Attention](https://md.research.elamur.ai/free/en/papers/jiang-2024-minference-1-0-accelerating-pre/)
- [A Survey on Large Language Models for Code Generation](https://md.research.elamur.ai/free/en/papers/jiang-2024-survey-large-language-models-code/)
- [Developing a Pragmatic Benchmark for Assessing Korean Legal Language Understanding in Large Language Models](https://md.research.elamur.ai/free/en/papers/kim-2024-developing-pragmatic-benchmark-assessing-korean/)
- [MT-Eval: A Multi-Turn Capabilities Evaluation Benchmark for Large Language Models](https://md.research.elamur.ai/free/en/papers/kwan-2024-mt-eval-multi-turn-capabilities/)
- [Step-DPO: Step-wise Preference Optimization for Long-chain Reasoning of LLMs](https://md.research.elamur.ai/free/en/papers/lai-2024-step-dpo-step-wise-preference/)
- [Mitigating Object Hallucinations in Large Vision-Language Models through Visual Contrastive Decoding](https://md.research.elamur.ai/free/en/papers/leng-2023-mitigating-object-hallucinations-large-vision/)
- [Do Large Language Models Understand Word Senses?](https://md.research.elamur.ai/free/en/papers/lex-2025-meconi-llms-word-senses/)
- [LLMs-as-Judges: A Comprehensive Survey on LLM-based Evaluation Methods](https://md.research.elamur.ai/free/en/papers/li-2024-llms-judges-comprehensive-survey-llm/)
- [Multimodal ArXiv: A Dataset for Improving Scientific Comprehension of Large Vision-Language Models](https://md.research.elamur.ai/free/en/papers/li-2024-multimodal-arxiv-dataset-improving-scientific/)
- [SALAD-Bench: A Hierarchical and Comprehensive Safety Benchmark for Large Language Models](https://md.research.elamur.ai/free/en/papers/li-2024-salad-bench-hierarchical-comprehensive-safety/)
- [LegalAgentBench: Evaluating LLM Agents in Legal Domain](https://md.research.elamur.ai/free/en/papers/li-2025-legalagentbench-evaluating-llm-agents-legal/)
- [Alignment at Pre-training! Towards Native Alignment for Arabic LLMs](https://md.research.elamur.ai/free/en/papers/liang-2024-alignment-pre-training-towards-native/)
- [MMBench: Is Your Multi-modal Model an All-around Player?](https://md.research.elamur.ai/free/en/papers/liu-2023-mmbench-multi-modal-model-all/)
- [OCRBench: On the Hidden Mystery of OCR in Large Multimodal Models](https://md.research.elamur.ai/free/en/papers/liu-2023-ocrbench-hidden-mystery-ocr-large/)
- [SiniticMTError: A Machine Translation Dataset with Error Annotations for Sinitic Languages](https://md.research.elamur.ai/free/en/papers/liu-2026-siniticmterror-machine-translation-dataset-error/)
- [WizardMath: Empowering Mathematical Reasoning for Large Language Models via Reinforced Evol-Instruct](https://md.research.elamur.ai/free/en/papers/luo-2023-wizardmath-empowering-mathematical-reasoning-large/)
- [Structured Prompting for Arabic Essay Proficiency: A Trait-Centric Evaluation Approach](https://md.research.elamur.ai/free/en/papers/mandhari-2026-structured-prompting-arabic-essay-proficiency/)
- [Russian Generative Spelling, Punctuation and Capitalization Correction](https://md.research.elamur.ai/free/en/papers/martynov-2026-russian-generative-spelling-punctuation-capitalization/)
- [HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal](https://md.research.elamur.ai/free/en/papers/mazeika-2024-harmbench-standardized-evaluation-framework-automated/)
- [从基础智能到通用智能: 基于大模型的GenAI和AGI之现状与展望](https://md.research.elamur.ai/free/en/papers/miao-2024-foundation-intelligence-general/)
- [Are the LLMs Capable of Maintaining at Least the Language Genus?](https://md.research.elamur.ai/free/en/papers/mitrovic-2026-are-llms-capable-maintaining-least/)
- [Evaluating Discriminability of Vision-Language Models](https://md.research.elamur.ai/free/en/papers/muraoka-2026-evaluating-discriminability-vision-language-models/)
- [BLEnD: A Benchmark for LLMs on Everyday Knowledge in Diverse Cultures and Languages](https://md.research.elamur.ai/free/en/papers/myung-2024-blend-benchmark-llms-everyday-knowledge/)
- [InstructSum: A Benchmark to Evaluate Instruction-Following Capability of Large Language Models in Summarization](https://md.research.elamur.ai/free/en/papers/nishida-2026-instructsum-benchmark-evaluate-instruction-following/)
- [Team QUESPA System Submission for the IWSLT 2026 Dialectal and Low-resource Speech Translation Task](https://md.research.elamur.ai/free/en/papers/ortega-2026-team-quespa-system-submission-iwslt/)
- [Controllable Sentence Simplification in Italian: Fine-Tuning Large Language Models on Automatically Generated Resources](https://md.research.elamur.ai/free/en/papers/papucci-2026-controllable-sentence-simplification-italian-fine/)
- [MultiPragEval: Multilingual Pragmatic Evaluation of Large Language Models](https://md.research.elamur.ai/free/en/papers/park-2024-multiprageval-multilingual-pragmatic-evaluation-large/)
- [SemEval-2025 Task 1: AdMIRe - Advancing Multimodal Idiomaticity Representation](https://md.research.elamur.ai/free/en/papers/pickard-2025-semeval-2025-task-1-admire/)
- [On the Robustness of Morphosyntactic Transformation with Large Language Models: The Case of Quechua Collao](https://md.research.elamur.ai/free/en/papers/pocco-2026-robustness-morphosyntactic-transformation-large-language/)
- [Interactive Agents: Simulating Counselor-Client Psychological Counseling via Role-Playing LLM-to-LLM Interactions](https://md.research.elamur.ai/free/en/papers/qiu-2026-interactive-agents-simulating-counselor-client/)
- [Large Language Models Put to the Test on Chinese Noun Compounds: Experiments on Natural Language Inference and Compound Semantics](https://md.research.elamur.ai/free/en/papers/qiu-2026-large-language-models-put-test/)
- [DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models](https://md.research.elamur.ai/free/en/papers/shao-2024-deepseekmath-pushing-limits-mathematical-reasoning/)
- [大模型参数高效微调方法综述: 技术、趋势与挑战](https://md.research.elamur.ai/free/en/papers/tang-2026-parameter-efficient-fine/)
- [Cambrian-1: A Fully Open, Vision-Centric Exploration of Multimodal LLMs](https://md.research.elamur.ai/free/en/papers/tong-2024-cambrian-1-fully-open-vision/)
- [Merge and Conquer: Instructing Multilingual Models by Adding Target Language Weights](https://md.research.elamur.ai/free/en/papers/valero-2026-merge-conquer-instructing-multilingual-models/)
- [MMLU-Pro: A More Robust and Challenging Multi-Task Language Understanding Benchmark](https://md.research.elamur.ai/free/en/papers/wang-2024-mmlu-pro-more-robust-challenging/)
- [Qwen2-VL: Enhancing Vision-Language Model's Perception of the World at Any Resolution](https://md.research.elamur.ai/free/en/papers/wang-2024-qwen2-vl-enhancing-vision/)
- [Smarter, Better, Faster, Longer: A Modern Bidirectional Encoder for Fast, Memory Efficient, and Long Context Finetuning and Inference](https://md.research.elamur.ai/free/en/papers/warner-2024-smarter-better-faster-longer-modern/)
- [RedPajama: an Open Dataset for Training Large Language Models](https://md.research.elamur.ai/free/en/papers/weber-2024-redpajama-open-dataset-training-large/)
- [OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments](https://md.research.elamur.ai/free/en/papers/xie-2024-osworld-benchmarking-multimodal-agents-open/)
- [Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing](https://md.research.elamur.ai/free/en/papers/xu-2024-magpie-alignment-data-synthesis-scratch/)
- [基于大语言模型的中文实体链接实证研究](https://md.research.elamur.ai/free/en/papers/xu-2025-empirical-study-chinese/)
- [Qwen2.5-Omni Technical Report](https://md.research.elamur.ai/free/en/papers/xu-2025-qwen2-5-omni-technical/)
- [Qwen3-Omni Technical Report](https://md.research.elamur.ai/free/en/papers/xu-2025-qwen3-omni-technical-report/)
- [Qwen2.5-1M Technical Report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen2-5-1m-technical/)
- [Qwen3 Technical Report](https://md.research.elamur.ai/free/en/papers/yang-2025-qwen3-technical-report/)
- [Is Sarcasm Detection A Step-by-Step Reasoning Process in Large Language Models?](https://md.research.elamur.ai/free/en/papers/yao-2024-sarcasm-detection-step-step-reasoning/)
- [Justice or Prejudice? Quantifying Biases in LLM-as-a-Judge](https://md.research.elamur.ai/free/en/papers/ye-2024-justice-prejudice-quantifying-biases-llm/)
- [MAmmoTH2: Scaling Instructions from the Web](https://md.research.elamur.ai/free/en/papers/yue-2024-mammoth2-scaling-instructions-web/)
- [TinyLlama: An Open-Source Small Language Model](https://md.research.elamur.ai/free/en/papers/zhang-2024-tinyllama-open-source-small-language/)
- [A Survey of Large Language Models](https://md.research.elamur.ai/free/en/papers/zhao-2023-survey-large-language-models/)
- [LlamaFactory: Unified Efficient Fine-Tuning of 100+ Language Models](https://md.research.elamur.ai/free/en/papers/zheng-2024-llamafactory-unified-efficient-fine-tuning/)
- [JudgeLM: Fine-tuned Large Language Models are Scalable Judges](https://md.research.elamur.ai/free/en/papers/zhu-2023-judgelm-fine-tuned-large-language/)
- [BigCodeBench: Benchmarking Code Generation with Diverse Function Calls and Complex Instructions](https://md.research.elamur.ai/free/en/papers/zhuo-2024-bigcodebench-benchmarking-code-generation-diverse/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{bai2023qwen,
  title = {{Qwen Technical Report}},
  author = {Bai, Jinze and Bai, Shuai and Chu, Yunfei and Cui, Zeyu and Dang, Kai and Deng, Xiaodong and Fan, Yang and Ge, Wenbin and Han, Yu and Huang, Fei and Hui, Binyuan and Ji, Luo and Li, Mei and Lin, Junyang and Lin, Runji and Liu, Dayiheng and Liu, Gao and Lu, Chengqiang and Lu, Keming and Ma, Jianxin and Men, Rui and Ren, Xingzhang and Ren, Xuancheng and Tan, Chuanqi and Tan, Sinan and Tu, Jianhong and Wang, Peng and Wang, Shijie and Wang, Wei and Wu, Shengguang and Xu, Benfeng and Xu, Jin and Yang, An and Yang, Hao and Yang, Jian and Yang, Shusheng and Yao, Yang and Yu, Bowen and Yuan, Hongyi and Yuan, Zheng and Zhang, Jianwei and Zhang, Xingxuan and Zhang, Yichang and Zhang, Zhenru and Zhou, Chang and Zhou, Jingren and Zhou, Xiaohuan and Zhu, Tianhang},
  year = {2023},
  doi = {10.48550/arxiv.2309.16609},
  eprint = {2309.16609},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2309.16609}
}
```

- [← previous QVoice: Arabic Speech Pronunciation Learning Application](https://md.research.elamur.ai/free/en/papers/kheir-2023-qvoice-arabic-speech-pronunciation-learning/)
- [next → Qwen-Audio: Advancing Universal Audio Understanding via Unified Large-Scale Audio-Language Models](https://md.research.elamur.ai/free/en/papers/chu-2023-qwen-audio-advancing-universal/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/6/)

## References

173 references, 67 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Loubna Ben Allal, Raymond Li, Denis Kocetkov, Chenghao Mou, Christopher Akiki, Carlos Munoz Ferrandis, Niklas Muennighoff, Mayank Mishra, Alex Gu, Manan Dey, et al. SantaCoder: Don’t reach for the stars! arXiv preprint arXiv:2301.03988, 2023. [find](https://scholar.google.com/scholar?q=Loubna+Ben+Allal%2C+Raymond+Li%2C+Denis+Kocetkov%2C+Chenghao+Mou%2C+Christopher+Akiki%2C+Carlos+Munoz+Ferrandis%2C+Niklas+Muennighoff%2C+Mayank+Mishra%2C+Alex+Gu%2C+Manan+Dey%2C+et+al%2E+SantaCoder%3A+Don%E2%80%99t+reach+for+the+sta) [our card](https://md.research.elamur.ai/free/en/papers/allal-2023-santacoder-don-t-reach-stars/)
2. Ebtesam Almazrouei, Hamza Alobeidli, Abdulaziz Alshamsi, Alessandro Cappelli, Ruxandra Cojocaru, Merouane Debbah, Etienne Goffinet, Daniel Heslow, Julien Launay, Quentin Malartic, Badreddine Noune, Baptiste Pannier, and Guilherme Penedo. Falcon-40B: An open large language model with state-of-the-art… [find](https://scholar.google.com/scholar?q=Ebtesam+Almazrouei%2C+Hamza+Alobeidli%2C+Abdulaziz+Alshamsi%2C+Alessandro+Cappelli%2C+Ruxandra+Cojocaru%2C+Merouane+Debbah%2C+Etienne+Goffinet%2C+Daniel+Heslow%2C+Julien+Launay%2C+Quentin+Malartic%2C+Badreddine+Noune%2C+Ba)
3. Rohan Anil, Andrew M Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, et al. PaLM 2 technical report. arXiv preprint arXiv:2305.10403, 2023. [find](https://scholar.google.com/scholar?q=Rohan+Anil%2C+Andrew+M+Dai%2C+Orhan+Firat%2C+Melvin+Johnson%2C+Dmitry+Lepikhin%2C+Alexandre+Passos%2C+Siamak+Shakeri%2C+Emanuel+Taropa%2C+Paige+Bailey%2C+Zhifeng+Chen%2C+et+al%2E+PaLM+2+technical+report%2E+arXiv+preprint+arX) [our card](https://md.research.elamur.ai/free/en/papers/anil-2023-palm-2-technical-report/)
4. Anthropic. Introducing Claude, 2023a. introducing-claude. [find](https://scholar.google.com/scholar?q=Anthropic%2E+Introducing+Claude%2C+2023a%2E+introducing%2Dclaude%2E)
5. URL [https://www.anthropic.com/index/Anthropic](https://www.anthropic.com/index/Anthropic). Claude 2. Technical report, Anthropic, 2023b. URL https://www-files. anthropic.com/production/images/Model-Card-Claude-2.pdf. [↗](https://www.anthropic.com/index/Anthropic)
6. Vamsi Aribandi, Yi Tay, Tal Schuster, Jinfeng Rao, Huaixiu Steven Zheng, Sanket Vaibhav Mehta, Honglei Zhuang, Vinh Q Tran, Dara Bahri, Jianmo Ni, et al. ExT5: Towards extreme multi-task scaling for transfer learning. arXiv preprint arXiv:2111.10952, 2021. [find](https://scholar.google.com/scholar?q=Vamsi+Aribandi%2C+Yi+Tay%2C+Tal+Schuster%2C+Jinfeng+Rao%2C+Huaixiu+Steven+Zheng%2C+Sanket+Vaibhav+Mehta%2C+Honglei+Zhuang%2C+Vinh+Q+Tran%2C+Dara+Bahri%2C+Jianmo+Ni%2C+et+al%2E+ExT5%3A+Towards+extreme+multi%2Dtask+scaling+for+t)
7. Amanda Askell, Yuntao Bai, Anna Chen, Dawn Drain, Deep Ganguli, Tom Henighan, Andy Jones, Nicholas Joseph, Ben Mann, Nova DasSarma, et al. A general language assistant as a laboratory for alignment. arXiv preprint arXiv:2112.00861, 2021. [find](https://scholar.google.com/scholar?q=Amanda+Askell%2C+Yuntao+Bai%2C+Anna+Chen%2C+Dawn+Drain%2C+Deep+Ganguli%2C+Tom+Henighan%2C+Andy+Jones%2C+Nicholas+Joseph%2C+Ben+Mann%2C+Nova+DasSarma%2C+et+al%2E+A+general+language+assistant+as+a+laboratory+for+alignment%2E+a)
8. Jacob Austin, Augustus Odena, Maxwell Nye, Maarten Bosma, Henryk Michalewski, David Dohan, Ellen Jiang, Carrie Cai, Michael Terry, Quoc Le, et al. Program synthesis with large language models. arXiv preprint arXiv:2108.07732, 2021. [find](https://scholar.google.com/scholar?q=Jacob+Austin%2C+Augustus+Odena%2C+Maxwell+Nye%2C+Maarten+Bosma%2C+Henryk+Michalewski%2C+David+Dohan%2C+Ellen+Jiang%2C+Carrie+Cai%2C+Michael+Terry%2C+Quoc+Le%2C+et+al%2E+Program+synthesis+with+large+language+models%2E+arXiv+p) [our card](https://md.research.elamur.ai/free/en/papers/austin-2021-program-synthesis-large-language-models/)
9. AutoGPT. AutoGPT: The heart of the open-source agent ecosystem, 2023. URL [https://github.com/Significant-Gravitas/Auto-GPT](https://github.com/Significant-Gravitas/Auto-GPT). [↗](https://github.com/Significant-Gravitas/Auto-GPT)
10. Lei Jimmy Ba, Jamie Ryan Kiros, and Geoffrey E. Hinton. Layer normalization. abs/1607.06450, 2016. URL http://arxiv.org/abs/1607.06450. [↗](http://arxiv.org/abs/1607.06450) [our card](https://md.research.elamur.ai/free/en/papers/ba-2016-layer-normalization/)
11. CoRR, Jinze Bai, Rui Men, Hao Yang, Xuancheng Ren, Kai Dang, Yichang Zhang, Xiaohuan Zhou, Peng Wang, Sinan Tan, An Yang andf Zeyu Cui, Yu Han, Shuai Bai, Wenbin Ge, Jianxin Ma, Junyang Lin, Jingren Zhou, and Chang Zhou. OFASys: A multi-modal multi-task learning system for building generalist models… — [doi:10.48550/arxiv.2212.04408](https://doi.org/10.48550/arxiv.2212.04408) [find](https://scholar.google.com/scholar?q=CoRR%2C+Jinze+Bai%2C+Rui+Men%2C+Hao+Yang%2C+Xuancheng+Ren%2C+Kai+Dang%2C+Yichang+Zhang%2C+Xiaohuan+Zhou%2C+Peng+Wang%2C+Sinan+Tan%2C+An+Yang+andf+Zeyu+Cui%2C+Yu+Han%2C+Shuai+Bai%2C+Wenbin+Ge%2C+Jianxin+Ma%2C+Junyang+Lin%2C+Jingren+Z)
12. Jinze Bai, Shuai Bai, Shusheng Yang, Shijie Wang, Sinan Tan, Peng Wang, Junyang Lin, Chang Zhou, and Jingren Zhou. Qwen-VL: A versatile vision-language model for understanding, localization, text reading, and beyond. CoRR, abs/2308.12966, 2023. doi: 10.48550/arXiv.2308.12966. URL https://doi.org/10.… [find](https://scholar.google.com/scholar?q=Jinze+Bai%2C+Shuai+Bai%2C+Shusheng+Yang%2C+Shijie+Wang%2C+Sinan+Tan%2C+Peng+Wang%2C+Junyang+Lin%2C+Chang+Zhou%2C+and+Jingren+Zhou%2E+Qwen%2DVL%3A+A+versatile+vision%2Dlanguage+model+for+understanding%2C+localization%2C+text+read) [our card](https://md.research.elamur.ai/free/en/papers/bai-2023-qwen-vl-versatile-vision-language/)
13. Yuntao Bai, Andy Jones, Kamal Ndousse, Amanda Askell, Anna Chen, Nova DasSarma, Dawn Drain, Stanislav Fort, Deep Ganguli, Tom Henighan, et al. Training a helpful and harmless assistant with reinforcement learning from human feedback. arXiv preprint arXiv:2204.05862, 2022b. [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Andy+Jones%2C+Kamal+Ndousse%2C+Amanda+Askell%2C+Anna+Chen%2C+Nova+DasSarma%2C+Dawn+Drain%2C+Stanislav+Fort%2C+Deep+Ganguli%2C+Tom+Henighan%2C+et+al%2E+Training+a+helpful+and+harmless+assistant+with+reinforcem)
14. Yuntao Bai, Saurav Kadavath, Sandipan Kundu, Amanda Askell, Jackson Kernion, Andy Jones, Anna Chen, Anna Goldie, Azalia Mirhoseini, Cameron McKinnon, et al. Constitutional AI: Harmlessness from AI feedback. arXiv preprint arXiv:2212.08073, 2022c. [find](https://scholar.google.com/scholar?q=Yuntao+Bai%2C+Saurav+Kadavath%2C+Sandipan+Kundu%2C+Amanda+Askell%2C+Jackson+Kernion%2C+Andy+Jones%2C+Anna+Chen%2C+Anna+Goldie%2C+Azalia+Mirhoseini%2C+Cameron+McKinnon%2C+et+al%2E+Constitutional+AI%3A+Harmlessness+from+AI+fee) [our card](https://md.research.elamur.ai/free/en/papers/bai-2022-constitutional-ai-harmlessness-ai-feedback/)
15. Iz Beltagy, Matthew E Peters, and Arman Cohan. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150, 2020. [find](https://scholar.google.com/scholar?q=Iz+Beltagy%2C+Matthew+E+Peters%2C+and+Arman+Cohan%2E+Longformer%3A+The+long%2Ddocument+transformer%2E+arXiv+preprint+arXiv%3A2004%2E05150%2C+2020%2E)
16. Yonatan Bisk, Rowan Zellers, Ronan Le Bras, Jianfeng Gao, and Yejin Choi. PIQA: reasoning about physical commonsense in natural language. In The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, The Thirty-Second Innovative Applications of Artificial Intelligence Conference, IAAI… [find](https://scholar.google.com/scholar?q=Yonatan+Bisk%2C+Rowan+Zellers%2C+Ronan+Le+Bras%2C+Jianfeng+Gao%2C+and+Yejin+Choi%2E+PIQA%3A+reasoning+about+physical+commonsense+in+natural+language%2E+In+The+Thirty%2DFourth+AAAI+Conference+on+Artificial+Intelligenc) [our card](https://md.research.elamur.ai/free/en/papers/bisk-2019-piqa-reasoning-about-physical-commonsense/)
17. AAAI Press, 2020. doi: 10.1609/aaai.v34i05.6239. URL https://doi.org/10.1609/aaai.v34i05.6239. — [doi:10.1609/aaai.v34i05.6239](https://doi.org/10.1609/aaai.v34i05.6239) [↗](https://doi.org/10.1609/aaai.v34i05.6239)
18. Sid Black, Stella Biderman, Eric Hallahan, Quentin Anthony, Leo Gao, Laurence Golding, Horace He, Connor Leahy, Kyle McDonell, Jason Phang, et al. GPT-NeoX-20B: An open-source autoregressive language model. arXiv preprint arXiv:2204.06745, 2022. bloc97. NTK-aware scaled RoPE allows LLaMA models to h… [find](https://scholar.google.com/scholar?q=Sid+Black%2C+Stella+Biderman%2C+Eric+Hallahan%2C+Quentin+Anthony%2C+Leo+Gao%2C+Laurence+Golding%2C+Horace+He%2C+Connor+Leahy%2C+Kyle+McDonell%2C+Jason+Phang%2C+et+al%2E+GPT%2DNeoX%2D20B%3A+An+open%2Dsource+autoregressive+language+) [our card](https://md.research.elamur.ai/free/en/papers/black-2022-gpt-neox-20b-open-source/)
19. Rishi Bommasani, Drew A Hudson, Ehsan Adeli, Russ Altman, Simran Arora, Sydney von Arx, Michael S Bernstein, Jeannette Bohg, Antoine Bosselut, Emma Brunskill, et al. On the opportunities and risks of foundation models. arXiv preprint arXiv:2108.07258, 2021. [find](https://scholar.google.com/scholar?q=Rishi+Bommasani%2C+Drew+A+Hudson%2C+Ehsan+Adeli%2C+Russ+Altman%2C+Simran+Arora%2C+Sydney+von+Arx%2C+Michael+S+Bernstein%2C+Jeannette+Bohg%2C+Antoine+Bosselut%2C+Emma+Brunskill%2C+et+al%2E+On+the+opportunities+and+risks+of+) [our card](https://md.research.elamur.ai/free/en/papers/bommasani-2021-opportunities-risks-foundation-models/)
20. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901, 2020. [find](https://scholar.google.com/scholar?q=Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+et+al%2E+Language+models+are+few%2Dshot+learners%2E+)
21. ChatGLM2 Team. ChatGLM2-6B: An open bilingual chat LLM, 2023. URL [https://github.com/THUDM/ChatGLM2-6B](https://github.com/THUDM/ChatGLM2-6B). [↗](https://github.com/THUDM/ChatGLM2-6B)
22. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Pondé de Oliveira Pinto, Jared Kaplan, Harrison Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mi… [find](https://scholar.google.com/scholar?q=Mark+Chen%2C+Jerry+Tworek%2C+Heewoo+Jun%2C+Qiming+Yuan%2C+Henrique+Ponde%CC%81+de+Oliveira+Pinto%2C+Jared+Kaplan%2C+Harrison+Edwards%2C+Yuri+Burda%2C+Nicholas+Joseph%2C+Greg+Brockman%2C+Alex+Ray%2C+Raul+Puri%2C+Gretchen+Krueger%2C+) [our card](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
23. Shouyuan Chen, Sherman Wong, Liangjian Chen, and Yuandong Tian. Extending context window of large language models via positional interpolation. arXiv preprint arXiv:2306.15595, 2023a. [find](https://scholar.google.com/scholar?q=Shouyuan+Chen%2C+Sherman+Wong%2C+Liangjian+Chen%2C+and+Yuandong+Tian%2E+Extending+context+window+of+large+language+models+via+positional+interpolation%2E+arXiv+preprint+arXiv%3A2306%2E15595%2C+2023a%2E)
24. Weize Chen, Yusheng Su, Jingwei Zuo, Cheng Yang, Chenfei Yuan, Chen Qian, Chi-Min Chan, Yujia Qin, Yaxi Lu, Ruobing Xie, et al. Agentverse: Facilitating multi-agent collaboration and exploring emergent behaviors in agents. arXiv preprint arXiv:2308.10848, 2023b. [find](https://scholar.google.com/scholar?q=Weize+Chen%2C+Yusheng+Su%2C+Jingwei+Zuo%2C+Cheng+Yang%2C+Chenfei+Yuan%2C+Chen+Qian%2C+Chi%2DMin+Chan%2C+Yujia+Qin%2C+Yaxi+Lu%2C+Ruobing+Xie%2C+et+al%2E+Agentverse%3A+Facilitating+multi%2Dagent+collaboration+and+exploring+emergen) [our card](https://md.research.elamur.ai/free/en/papers/chen-2023-agentverse-facilitating-multi-agent/)
25. Zhihong Chen, Feng Jiang, Junying Chen, Tiannan Wang, Fei Yu, Guiming Chen, Hongbo Zhang, Juhao Liang, Chen Zhang, Zhiyi Zhang, et al. Phoenix: Democratizing ChatGPT across languages. arXiv preprint arXiv:2304.10453, 2023c. [find](https://scholar.google.com/scholar?q=Zhihong+Chen%2C+Feng+Jiang%2C+Junying+Chen%2C+Tiannan+Wang%2C+Fei+Yu%2C+Guiming+Chen%2C+Hongbo+Zhang%2C+Juhao+Liang%2C+Chen+Zhang%2C+Zhiyi+Zhang%2C+et+al%2E+Phoenix%3A+Democratizing+ChatGPT+across+languages%2E+arXiv+preprint+a)
26. Ethan Chern, Haoyang Zou, Xuefeng Li, Jiewen Hu, Kehua Feng, Junlong Li, and Pengfei Liu. Generative ai for math: Abel. [https://github.com/GAIR-NLP/abel](https://github.com/GAIR-NLP/abel), 2023a. [↗](https://github.com/GAIR-NLP/abel)
27. I Chern, Steffi Chern, Shiqi Chen, Weizhe Yuan, Kehua Feng, Chunting Zhou, Junxian He, Graham Neubig, Pengfei Liu, et al. Factool: Factuality detection in generative ai–a tool augmented framework for multi-task and multi-domain scenarios. arXiv preprint arXiv:2307.13528, 2023b. [find](https://scholar.google.com/scholar?q=I+Chern%2C+Steffi+Chern%2C+Shiqi+Chen%2C+Weizhe+Yuan%2C+Kehua+Feng%2C+Chunting+Zhou%2C+Junxian+He%2C+Graham+Neubig%2C+Pengfei+Liu%2C+et+al%2E+Factool%3A+Factuality+detection+in+generative+ai%E2%80%93a+tool+augmented+framework+for+) [our card](https://md.research.elamur.ai/free/en/papers/chern-2023-factool-factuality-detection-generative-ai/)
28. David Chiang and Peter Cholak. Overcoming a theoretical limitation of self-attention. In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 7654–7664, 2022. [find](https://scholar.google.com/scholar?q=David+Chiang+and+Peter+Cholak%2E+Overcoming+a+theoretical+limitation+of+self%2Dattention%2E+In+Proceedings+of+the+60th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Volume+1%3A+Long+Papers%29)
29. Wei-Lin Chiang, Zhuohan Li, Zi Lin, Ying Sheng, Zhanghao Wu, Hao Zhang, Lianmin Zheng, Siyuan Zhuang, Yonghao Zhuang, Joseph E. Gonzalez, Ion Stoica, and Eric P. Xing. Vicuna: An open-source chatbot impressing GPT-4 with 90%* ChatGPT quality, March 2023. URL [https://lmsys.org/blog/2023-03-30-vicuna/…](https://lmsys.org/blog/2023-03-30-vicuna/…) [↗](https://lmsys.org/blog/2023-03-30-vicuna/…)
30. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. PaLM: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311, 2022. [find](https://scholar.google.com/scholar?q=Aakanksha+Chowdhery%2C+Sharan+Narang%2C+Jacob+Devlin%2C+Maarten+Bosma%2C+Gaurav+Mishra%2C+Adam+Roberts%2C+Paul+Barham%2C+Hyung+Won+Chung%2C+Charles+Sutton%2C+Sebastian+Gehrmann%2C+et+al%2E+PaLM%3A+Scaling+language+modeling+w)
31. Paul F. Christiano, Jan Leike, Tom B. Brown, Miljan Martic, Shane Legg, and Dario Amodei. Deep reinforcement learning from human preferences. In Isabelle Guyon, Ulrike von Luxburg, Samy Bengio, Hanna M. Wallach, Rob Fergus, S. V. N. Vishwanathan, and Roman Garnett (eds.), Advances in Neural Informat… [find](https://scholar.google.com/scholar?q=Paul+F%2E+Christiano%2C+Jan+Leike%2C+Tom+B%2E+Brown%2C+Miljan+Martic%2C+Shane+Legg%2C+and+Dario+Amodei%2E+Deep+reinforcement+learning+from+human+preferences%2E+In+Isabelle+Guyon%2C+Ulrike+von+Luxburg%2C+Samy+Bengio%2C+Hanna+)
32. Hyung Won Chung, Le Hou, Shayne Longpre, Barret Zoph, Yi Tay, William Fedus, Eric Li, Xuezhi Wang, Mostafa Dehghani, Siddhartha Brahma, et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416, 2022. [find](https://scholar.google.com/scholar?q=Hyung+Won+Chung%2C+Le+Hou%2C+Shayne+Longpre%2C+Barret+Zoph%2C+Yi+Tay%2C+William+Fedus%2C+Eric+Li%2C+Xuezhi+Wang%2C+Mostafa+Dehghani%2C+Siddhartha+Brahma%2C+et+al%2E+Scaling+instruction%2Dfinetuned+language+models%2E+arXiv+prep)
33. Christopher Clark, Kenton Lee, Ming-Wei Chang, Tom Kwiatkowski, Michael Collins, and Kristina Toutanova. Boolq: Exploring the surprising difficulty of natural yes/no questions. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapte… [find](https://scholar.google.com/scholar?q=Christopher+Clark%2C+Kenton+Lee%2C+Ming%2DWei+Chang%2C+Tom+Kwiatkowski%2C+Michael+Collins%2C+and+Kristina+Toutanova%2E+Boolq%3A+Exploring+the+surprising+difficulty+of+natural+yes%2Fno+questions%2E+In+Jill+Burstein%2C+Chris) [our card](https://md.research.elamur.ai/free/en/papers/clark-2019-boolq-exploring-surprising-difficulty-natural/)
34. Peter Clark, Isaac Cowhey, Oren Etzioni, Tushar Khot, Ashish Sabharwal, Carissa Schoenick, and Oyvind Tafjord. Think you have solved question answering? try arc, the AI2 reasoning challenge. CoRR, abs/1803.05457, 2018. URL http://arxiv.org/abs/1803.05457. [↗](http://arxiv.org/abs/1803.05457)
35. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. Training verifiers to solve math word problems. arXiv preprint arXiv:2110.14168, 2021. [find](https://scholar.google.com/scholar?q=Karl+Cobbe%2C+Vineet+Kosaraju%2C+Mohammad+Bavarian%2C+Mark+Chen%2C+Heewoo+Jun%2C+Lukasz+Kaiser%2C+Matthias+Plappert%2C+Jerry+Tworek%2C+Jacob+Hilton%2C+Reiichiro+Nakano%2C+et+al%2E+Training+verifiers+to+solve+math+word+prob) [our card](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/)
36. Alexis Conneau, Kartikay Khandelwal, Naman Goyal, Vishrav Chaudhary, Guillaume Wenzek, Francisco Guzmán, Edouard Grave, Myle Ott, Luke Zettlemoyer, and Veselin Stoyanov. Unsupervised cross-lingual representation learning at scale. arXiv preprint arXiv:1911.02116, 2019. [find](https://scholar.google.com/scholar?q=Alexis+Conneau%2C+Kartikay+Khandelwal%2C+Naman+Goyal%2C+Vishrav+Chaudhary%2C+Guillaume+Wenzek%2C+Francisco+Guzma%CC%81n%2C+Edouard+Grave%2C+Myle+Ott%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+Unsupervised+cross%2Dlingual+re)
37. Mike Conover, Matt Hayes, Ankit Mathur, Jianwei Xie, Jun Wan, Sam Shah, Ali Ghodsi, Patrick Wendell, Matei Zaharia, and Reynold Xin. Free Dolly: Introducing the world’s first truly open instruction-tuned LLM, 2023. URL [https://www.databricks.com/blog/2023/04/](https://www.databricks.com/blog/2023/04/) 12/dolly-first-open-commercially-viable-… [↗](https://www.databricks.com/blog/2023/04/)
38. Wenliang Dai, Junnan Li, Dongxu Li, Anthony Meng, Huat Tiong, Junqi Zhao, Weisheng Wang, Boyang Li, Pascale Fung, and Steven Hoi. InstructBLIP: Towards general-purpose vision-language models with instruction tuning. arXiv preprint arXiv:2305.06500, 2023. [find](https://scholar.google.com/scholar?q=Wenliang+Dai%2C+Junnan+Li%2C+Dongxu+Li%2C+Anthony+Meng%2C+Huat+Tiong%2C+Junqi+Zhao%2C+Weisheng+Wang%2C+Boyang+Li%2C+Pascale+Fung%2C+and+Steven+Hoi%2E+InstructBLIP%3A+Towards+general%2Dpurpose+vision%2Dlanguage+models+with+inst) [our card](https://md.research.elamur.ai/free/en/papers/dai-2023-instructblip-towards-general-purpose-vision/)
39. Tri Dao, Daniel Y. Fu, Stefano Ermon, Atri Rudra, and Christopher Ré. FlashAttention: Fast and memory-efficient exact attention with io-awareness. In NeurIPS, 2022. URL [http://papers.nips.cc/paper_files/paper/2022/hash/](http://papers.nips.cc/paper_files/paper/2022/hash/) 67d57c32e20fd0a7a302cb81d36e40d5-Abstract-Conference.html. [↗](http://papers.nips.cc/paper_files/paper/2022/hash/)
40. Yann N Dauphin, Angela Fan, Michael Auli, and David Grangier. Language modeling with gated convolutional networks. In International conference on machine learning, pp. 933–941. PMLR, 2017. [find](https://scholar.google.com/scholar?q=Yann+N+Dauphin%2C+Angela+Fan%2C+Michael+Auli%2C+and+David+Grangier%2E+Language+modeling+with+gated+convolutional+networks%2E+In+International+conference+on+machine+learning%2C+pp%2E+933%E2%80%93941%2E+PMLR%2C+2017%2E) [our card](https://md.research.elamur.ai/free/en/papers/dauphin-2016-language-modeling-gated-convolutional-networks/)
41. Tim Dettmers, Mike Lewis, Younes Belkada, and Luke Zettlemoyer. LLM.int8(): 8-bit matrix multiplication for transformers at scale. arXiv preprint arXiv:2208.07339, 2022. [find](https://scholar.google.com/scholar?q=Tim+Dettmers%2C+Mike+Lewis%2C+Younes+Belkada%2C+and+Luke+Zettlemoyer%2E+LLM%2Eint8%28%29%3A+8%2Dbit+matrix+multiplication+for+transformers+at+scale%2E+arXiv+preprint+arXiv%3A2208%2E07339%2C+2022%2E)
42. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. QLoRA: Efficient finetuning of quantized LLMs. arXiv preprint arXiv:2305.14314, 2023. [find](https://scholar.google.com/scholar?q=Tim+Dettmers%2C+Artidoro+Pagnoni%2C+Ari+Holtzman%2C+and+Luke+Zettlemoyer%2E+QLoRA%3A+Efficient+finetuning+of+quantized+LLMs%2E+arXiv+preprint+arXiv%3A2305%2E14314%2C+2023%2E)
43. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018. [find](https://scholar.google.com/scholar?q=Jacob+Devlin%2C+Ming%2DWei+Chang%2C+Kenton+Lee%2C+and+Kristina+Toutanova%2E+BERT%3A+Pre%2Dtraining+of+deep+bidirectional+transformers+for+language+understanding%2E+arXiv+preprint+arXiv%3A1810%2E04805%2C+2018%2E)
44. Ning Ding, Yulin Chen, Bokai Xu, Yujia Qin, Zhi Zheng, Shengding Hu, Zhiyuan Liu, Maosong Sun, and Bowen Zhou. Enhancing chat language models by scaling high-quality instructional conversations. arXiv preprint arXiv:2305.14233, 2023. [find](https://scholar.google.com/scholar?q=Ning+Ding%2C+Yulin+Chen%2C+Bokai+Xu%2C+Yujia+Qin%2C+Zhi+Zheng%2C+Shengding+Hu%2C+Zhiyuan+Liu%2C+Maosong+Sun%2C+and+Bowen+Zhou%2E+Enhancing+chat+language+models+by+scaling+high%2Dquality+instructional+conversations%2E+arXiv) [our card](https://md.research.elamur.ai/free/en/papers/ding-2023-enhancing-chat-language-models-scaling/)
45. Danny Driess, Fei Xia, Mehdi SM Sajjadi, Corey Lynch, Aakanksha Chowdhery, Brian Ichter, Ayzaan Wahid, Jonathan Tompson, Quan Vuong, Tianhe Yu, et al. Palm-e: An embodied multimodal language model. arXiv preprint arXiv:2303.03378, 2023. [find](https://scholar.google.com/scholar?q=Danny+Driess%2C+Fei+Xia%2C+Mehdi+SM+Sajjadi%2C+Corey+Lynch%2C+Aakanksha+Chowdhery%2C+Brian+Ichter%2C+Ayzaan+Wahid%2C+Jonathan+Tompson%2C+Quan+Vuong%2C+Tianhe+Yu%2C+et+al%2E+Palm%2De%3A+An+embodied+multimodal+language+model%2E+ar)
46. Nan Du, Yanping Huang, Andrew M Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, et al. GLaM: Efficient scaling of language models with mixture-of-experts. In International Conference on Machine Learning, pp. 5547–5569. PMLR, 2022. [find](https://scholar.google.com/scholar?q=Nan+Du%2C+Yanping+Huang%2C+Andrew+M+Dai%2C+Simon+Tong%2C+Dmitry+Lepikhin%2C+Yuanzhong+Xu%2C+Maxim+Krikun%2C+Yanqi+Zhou%2C+Adams+Wei+Yu%2C+Orhan+Firat%2C+et+al%2E+GLaM%3A+Efficient+scaling+of+language+models+with+mixture%2Dof%2De) [our card](https://md.research.elamur.ai/free/en/papers/du-2021-glam-efficient-scaling-language-models/)
47. Zhengxiao Du, Yujie Qian, Xiao Liu, Ming Ding, Jiezhong Qiu, Zhilin Yang, and Jie Tang. GLM: General language model pretraining with autoregressive blank infilling. arXiv preprint arXiv:2103.10360, 2021. [find](https://scholar.google.com/scholar?q=Zhengxiao+Du%2C+Yujie+Qian%2C+Xiao+Liu%2C+Ming+Ding%2C+Jiezhong+Qiu%2C+Zhilin+Yang%2C+and+Jie+Tang%2E+GLM%3A+General+language+model+pretraining+with+autoregressive+blank+infilling%2E+arXiv+preprint+arXiv%3A2103%2E10360%2C+20) [our card](https://md.research.elamur.ai/free/en/papers/du-2022-glm-general-language-model-pretraining/)
48. Kawin Ethayarajh, Yejin Choi, and Swabha Swayamdipta. Understanding dataset difficulty with V-usable information. In Kamalika Chaudhuri, Stefanie Jegelka, Le Song, Csaba Szepesvari, Gang Niu, and Sivan Sabato (eds.), Proceedings of the 39th International Conference on Machine Learning, volume 162 of… [find](https://scholar.google.com/scholar?q=Kawin+Ethayarajh%2C+Yejin+Choi%2C+and+Swabha+Swayamdipta%2E+Understanding+dataset+difficulty+with+V%2Dusable+information%2E+In+Kamalika+Chaudhuri%2C+Stefanie+Jegelka%2C+Le+Song%2C+Csaba+Szepesvari%2C+Gang+Niu%2C+and+Siva)
49. William Fedus, Barret Zoph, and Noam Shazeer. Switch transformers: Scaling to trillion parameter models with simple and efficient sparsity. The Journal of Machine Learning Research, 23(1): 5232–5270, 2022. [find](https://scholar.google.com/scholar?q=William+Fedus%2C+Barret+Zoph%2C+and+Noam+Shazeer%2E+Switch+transformers%3A+Scaling+to+trillion+parameter+models+with+simple+and+efficient+sparsity%2E+The+Journal+of+Machine+Learning+Research%2C+23%281%29%3A+5232%E2%80%935270%2C+)
50. Elias Frantar, Saleh Ashkboos, Torsten Hoefler, and Dan Alistarh. GPTQ: Accurate post-training quantization for generative pre-trained transformers. arXiv preprint arXiv:2210.17323, 2022. [find](https://scholar.google.com/scholar?q=Elias+Frantar%2C+Saleh+Ashkboos%2C+Torsten+Hoefler%2C+and+Dan+Alistarh%2E+GPTQ%3A+Accurate+post%2Dtraining+quantization+for+generative+pre%2Dtrained+transformers%2E+arXiv+preprint+arXiv%3A2210%2E17323%2C+2022%2E)
51. Daniel Fried, Armen Aghajanyan, Jessy Lin, Sida I. Wang, Eric Wallace, Freda Shi, Ruiqi Zhong, Wen tau Yih, Luke Zettlemoyer, and Mike Lewis. Incoder: A generative model for code infilling and synthesis. ArXiv, abs/2204.05999, 2022. [find](https://scholar.google.com/scholar?q=Daniel+Fried%2C+Armen+Aghajanyan%2C+Jessy+Lin%2C+Sida+I%2E+Wang%2C+Eric+Wallace%2C+Freda+Shi%2C+Ruiqi+Zhong%2C+Wen+tau+Yih%2C+Luke+Zettlemoyer%2C+and+Mike+Lewis%2E+Incoder%3A+A+generative+model+for+code+infilling+and+synthes)
52. Google. An important next step on our AI journey, 2023. URL [https://blog.google/](https://blog.google/) technology/ai/bard-google-ai-search-updates/. [↗](https://blog.google/)
53. Dan Hendrycks and Kevin Gimpel. Bridging nonlinearities and stochastic regularizers with Gaussian error linear units. CoRR, abs/1606.08415, 2016. URL http://arxiv.org/abs/1606. 08415. [↗](http://arxiv.org/abs/1606)
54. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. Measuring massive multitask language understanding. arXiv preprint arXiv:2009.03300, 2020. [find](https://scholar.google.com/scholar?q=Dan+Hendrycks%2C+Collin+Burns%2C+Steven+Basart%2C+Andy+Zou%2C+Mantas+Mazeika%2C+Dawn+Song%2C+and+Jacob+Steinhardt%2E+Measuring+massive+multitask+language+understanding%2E+arXiv+preprint+arXiv%3A2009%2E03300%2C+2020%2E) [our card](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/)
55. Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt. Measuring mathematical problem solving with the math dataset. arXiv preprint arXiv:2103.03874, 2021. [find](https://scholar.google.com/scholar?q=Dan+Hendrycks%2C+Collin+Burns%2C+Saurav+Kadavath%2C+Akul+Arora%2C+Steven+Basart%2C+Eric+Tang%2C+Dawn+Song%2C+and+Jacob+Steinhardt%2E+Measuring+mathematical+problem+solving+with+the+math+dataset%2E+arXiv+preprint+arXiv%3A) [our card](https://md.research.elamur.ai/free/en/papers/hendrycks-2021-measuring-mathematical-problem-solving-math/)
56. Jordan Hoffmann, Sebastian Borgeaud, Arthur Mensch, Elena Buchatskaya, Trevor Cai, Eliza Rutherford, Diego de Las Casas, Lisa Anne Hendricks, Johannes Welbl, Aidan Clark, et al. Training compute-optimal large language models. arXiv preprint arXiv:2203.15556, 2022. [find](https://scholar.google.com/scholar?q=Jordan+Hoffmann%2C+Sebastian+Borgeaud%2C+Arthur+Mensch%2C+Elena+Buchatskaya%2C+Trevor+Cai%2C+Eliza+Rutherford%2C+Diego+de+Las+Casas%2C+Lisa+Anne+Hendricks%2C+Johannes+Welbl%2C+Aidan+Clark%2C+et+al%2E+Training+compute%2Doptim)
57. Sirui Hong, Xiawu Zheng, Jonathan Chen, Yuheng Cheng, Ceyao Zhang, Zili Wang, Steven Ka Shing Yau, Zijuan Lin, Liyang Zhou, Chenyu Ran, et al. Metagpt: Meta programming for multi-agent collaborative framework. arXiv preprint arXiv:2308.00352, 2023. [find](https://scholar.google.com/scholar?q=Sirui+Hong%2C+Xiawu+Zheng%2C+Jonathan+Chen%2C+Yuheng+Cheng%2C+Ceyao+Zhang%2C+Zili+Wang%2C+Steven+Ka+Shing+Yau%2C+Zijuan+Lin%2C+Liyang+Zhou%2C+Chenyu+Ran%2C+et+al%2E+Metagpt%3A+Meta+programming+for+multi%2Dagent+collaborative+f) [our card](https://md.research.elamur.ai/free/en/papers/hong-2023-metagpt-meta-programming-multi/)
58. Chenxu Hu, Jie Fu, Chenzhuang Du, Simian Luo, Junbo Zhao, and Hang Zhao. Chatdb: Augmenting llms with databases as their symbolic memory. arXiv preprint arXiv:2306.03901, 2023. — [arXiv:2306.03901](https://arxiv.org/abs/2306.03901) [find](https://scholar.google.com/scholar?q=Chenxu+Hu%2C+Jie+Fu%2C+Chenzhuang+Du%2C+Simian+Luo%2C+Junbo+Zhao%2C+and+Hang+Zhao%2E+Chatdb%3A+Augmenting+llms+with+databases+as+their+symbolic+memory%2E+arXiv+preprint+arXiv%3A2306%2E03901%2C+2023%2E+%E2%80%94+%5BarXiv%3A2306%2E03901%5D%28ht)
59. Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. arXiv preprint arXiv:2106.09685, 2021. [find](https://scholar.google.com/scholar?q=Edward+J+Hu%2C+Yelong+Shen%2C+Phillip+Wallis%2C+Zeyuan+Allen%2DZhu%2C+Yuanzhi+Li%2C+Shean+Wang%2C+Lu+Wang%2C+and+Weizhu+Chen%2E+LoRA%3A+Low%2Drank+adaptation+of+large+language+models%2E+arXiv+preprint+arXiv%3A2106%2E09685%2C+2021%2E)
60. Hai Hu, Kyle Richardson, Liang Xu, Lu Li, Sandra Kübler, and Lawrence S. Moss. OCNLI: original chinese natural language inference. In Trevor Cohn, Yulan He, and Yang Liu (eds.), Findings of the Association for Computational Linguistics: EMNLP 2020, Online Event, 16-20 November 2020, volume EMNLP 20… [find](https://scholar.google.com/scholar?q=Hai+Hu%2C+Kyle+Richardson%2C+Liang+Xu%2C+Lu+Li%2C+Sandra+Ku%CC%88bler%2C+and+Lawrence+S%2E+Moss%2E+OCNLI%3A+original+chinese+natural+language+inference%2E+In+Trevor+Cohn%2C+Yulan+He%2C+and+Yang+Liu+%28eds%2E%29%2C+Findings+of+the+Assoc)
61. Yuzhen Huang, Yuzhuo Bai, Zhihao Zhu, Junlei Zhang, Jinghan Zhang, Tangjun Su, Junteng Liu, Chuancheng Lv, Yikai Zhang, Jiayi Lei, et al. C-Eval: A multi-level multi-discipline chinese evaluation suite for foundation models. arXiv preprint arXiv:2305.08322, 2023. [find](https://scholar.google.com/scholar?q=Yuzhen+Huang%2C+Yuzhuo+Bai%2C+Zhihao+Zhu%2C+Junlei+Zhang%2C+Jinghan+Zhang%2C+Tangjun+Su%2C+Junteng+Liu%2C+Chuancheng+Lv%2C+Yikai+Zhang%2C+Jiayi+Lei%2C+et+al%2E+C%2DEval%3A+A+multi%2Dlevel+multi%2Ddiscipline+chinese+evaluation+suit) [our card](https://md.research.elamur.ai/free/en/papers/huang-2023-c-eval-multi-level-multi/)
62. Hugging Face. Transformers agents, 2023. URL [https://huggingface.co/docs/](https://huggingface.co/docs/) transformers/transformers_agents. [↗](https://huggingface.co/docs/)
63. Baichuan Inc. Baichuan-7B: A large-scale 7B pretraining language model developed by BaiChuan-Inc, 2023a. URL [https://github.com/baichuan-inc/Baichuan-7B](https://github.com/baichuan-inc/Baichuan-7B). [↗](https://github.com/baichuan-inc/Baichuan-7B)
64. XVERSE Technology Inc. XVERSE-13B: A multilingual large language model developed by XVERSE Technology Inc., 2023b. URL [https://github.com/xverse-ai/](https://github.com/xverse-ai/) XVERSE-13B. [↗](https://github.com/xverse-ai/)
65. InternLM Team. InternLM: A multilingual language model with progressively enhanced capabilities, 2023. URL [https://github.com/InternLM/InternLM](https://github.com/InternLM/InternLM). [↗](https://github.com/InternLM/InternLM)
66. Shantanu Jain. tiktoken: A fast BPE tokeniser for use with OpenAI’s models, 2022. URL [https://github.com/openai/tiktoken/](https://github.com/openai/tiktoken/). [↗](https://github.com/openai/tiktoken/)
67. Yunjie Ji, Yong Deng, Yan Gong, Yiping Peng, Qiang Niu, Lei Zhang, Baochang Ma, and Xiangang Li. Exploring the impact of instruction data scaling on large language models: An empirical study on real-world use cases. arXiv preprint arXiv:2303.14742, 2023. [find](https://scholar.google.com/scholar?q=Yunjie+Ji%2C+Yong+Deng%2C+Yan+Gong%2C+Yiping+Peng%2C+Qiang+Niu%2C+Lei+Zhang%2C+Baochang+Ma%2C+and+Xiangang+Li%2E+Exploring+the+impact+of+instruction+data+scaling+on+large+language+models%3A+An+empirical+study+on+real%2Dw)
68. Zixuan Jiang, Jiaqi Gu, Hanqing Zhu, and David Z. Pan. Pre-RMSNorm and Pre-CRMSNorm transformers: Equivalent and efficient pre-LN transformers. CoRR, abs/2305.14858, 2023. doi: 10.48550/arXiv.2305.14858. URL https://doi.org/10.48550/arXiv.2305.14858. — [doi:10.48550/arxiv.2305.14858](https://doi.org/10.48550/arxiv.2305.14858) [↗](https://doi.org/10.48550/arXiv.2305.14858)
69. Diederik P Kingma and Jimmy Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014. [find](https://scholar.google.com/scholar?q=Diederik+P+Kingma+and+Jimmy+Ba%2E+Adam%3A+A+method+for+stochastic+optimization%2E+arXiv+preprint+arXiv%3A1412%2E6980%2C+2014%2E)
70. Tom Kwiatkowski, Jennimaria Palomaki, Olivia Redfield, Michael Collins, Ankur P. Parikh, Chris Alberti, Danielle Epstein, Illia Polosukhin, Jacob Devlin, Kenton Lee, Kristina Toutanova, Llion Jones, Matthew Kelcey, Ming-Wei Chang, Andrew M. Dai, Jakob Uszkoreit, Quoc Le, and Slav Petrov. Natural que… [find](https://scholar.google.com/scholar?q=Tom+Kwiatkowski%2C+Jennimaria+Palomaki%2C+Olivia+Redfield%2C+Michael+Collins%2C+Ankur+P%2E+Parikh%2C+Chris+Alberti%2C+Danielle+Epstein%2C+Illia+Polosukhin%2C+Jacob+Devlin%2C+Kenton+Lee%2C+Kristina+Toutanova%2C+Llion+Jones%2C+M) [our card](https://md.research.elamur.ai/free/en/papers/kwiatkowski-2019-natural-questions-benchmark-question-answering/)
71. Woosuk Kwon, Zhuohan Li, Siyuan Zhuang, Ying Sheng, Lianmin Zheng, Cody Hao Yu, Joseph E. Gonzalez, Hao Zhang, and Ion Stoica. Efficient memory management for large language model serving with PagedAttention. In Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles, 2023. [find](https://scholar.google.com/scholar?q=Woosuk+Kwon%2C+Zhuohan+Li%2C+Siyuan+Zhuang%2C+Ying+Sheng%2C+Lianmin+Zheng%2C+Cody+Hao+Yu%2C+Joseph+E%2E+Gonzalez%2C+Hao+Zhang%2C+and+Ion+Stoica%2E+Efficient+memory+management+for+large+language+model+serving+with+PagedAt)
72. LangChain, Inc. LangChain: Building applications with LLMs through composability, 2023. URL [https://python.langchain.com/](https://python.langchain.com/). [↗](https://python.langchain.com/)
73. Dmitry Lepikhin, HyoukJoong Lee, Yuanzhong Xu, Dehao Chen, Orhan Firat, Yanping Huang, Maxim Krikun, Noam Shazeer, and Zhifeng Chen. GShard: Scaling giant models with conditional computation and automatic sharding. arXiv preprint arXiv:2006.16668, 2020. [find](https://scholar.google.com/scholar?q=Dmitry+Lepikhin%2C+HyoukJoong+Lee%2C+Yuanzhong+Xu%2C+Dehao+Chen%2C+Orhan+Firat%2C+Yanping+Huang%2C+Maxim+Krikun%2C+Noam+Shazeer%2C+and+Zhifeng+Chen%2E+GShard%3A+Scaling+giant+models+with+conditional+computation+and+autom)
74. Aitor Lewkowycz, Anders Andreassen, David Dohan, Ethan Dyer, Henryk Michalewski, Vinay Ramasesh, Ambrose Slone, Cem Anil, Imanol Schlag, Theo Gutman-Solo, Yuhuai Wu, Behnam Neyshabur, Guy Gur-Ari, and Vedant Misra. Solving quantitative reasoning problems with language models, 2022. [find](https://scholar.google.com/scholar?q=Aitor+Lewkowycz%2C+Anders+Andreassen%2C+David+Dohan%2C+Ethan+Dyer%2C+Henryk+Michalewski%2C+Vinay+Ramasesh%2C+Ambrose+Slone%2C+Cem+Anil%2C+Imanol+Schlag%2C+Theo+Gutman%2DSolo%2C+Yuhuai+Wu%2C+Behnam+Neyshabur%2C+Guy+Gur%2DAri%2C+and) [our card](https://md.research.elamur.ai/free/en/papers/lewkowycz-2022-solving-quantitative-reasoning-problems-language/)
75. Chenliang Li, Hehong Chen, Ming Yan, Weizhou Shen, Haiyang Xu, Zhikai Wu, Zhicheng Zhang, Wenmeng Zhou, Yingda Chen, Chen Cheng, et al. ModelScope-Agent: Building your customizable agent system with open-source large language models. arXiv preprint arXiv:2309.00986, 2023a. [find](https://scholar.google.com/scholar?q=Chenliang+Li%2C+Hehong+Chen%2C+Ming+Yan%2C+Weizhou+Shen%2C+Haiyang+Xu%2C+Zhikai+Wu%2C+Zhicheng+Zhang%2C+Wenmeng+Zhou%2C+Yingda+Chen%2C+Chen+Cheng%2C+et+al%2E+ModelScope%2DAgent%3A+Building+your+customizable+agent+system+with+o)
76. Guohao Li, Hasan Abed Al Kader Hammoud, Hani Itani, Dmitrii Khizbullin, and Bernard Ghanem. Camel: Communicative agents for “mind” exploration of large scale language model society. arXiv preprint arXiv:2303.17760, 2023b. [find](https://scholar.google.com/scholar?q=Guohao+Li%2C+Hasan+Abed+Al+Kader+Hammoud%2C+Hani+Itani%2C+Dmitrii+Khizbullin%2C+and+Bernard+Ghanem%2E+Camel%3A+Communicative+agents+for+%E2%80%9Cmind%E2%80%9D+exploration+of+large+scale+language+model+society%2E+arXiv+preprint+arX) [our card](https://md.research.elamur.ai/free/en/papers/li-2023-camel-communicative-agents-mind/)
77. Haonan Li, Yixuan Zhang, Fajri Koto, Yifei Yang, Hai Zhao, Yeyun Gong, Nan Duan, and Timothy Baldwin. CMMLU: Measuring massive multitask language understanding in Chinese. arXiv preprint arXiv:2306.09212, 2023c. [find](https://scholar.google.com/scholar?q=Haonan+Li%2C+Yixuan+Zhang%2C+Fajri+Koto%2C+Yifei+Yang%2C+Hai+Zhao%2C+Yeyun+Gong%2C+Nan+Duan%2C+and+Timothy+Baldwin%2E+CMMLU%3A+Measuring+massive+multitask+language+understanding+in+Chinese%2E+arXiv+preprint+arXiv%3A2306%2E09) [our card](https://md.research.elamur.ai/free/en/papers/li-2023-cmmlu-measuring-massive-multitask-language/)
78. Raymond Li, Loubna Ben Allal, Yangtian Zi, Niklas Muennighoff, Denis Kocetkov, Chenghao Mou, Marc Marone, Christopher Akiki, Jia Li, Jenny Chim, Qian Liu, Evgenii Zheltonozhskii, Terry Yue Zhuo, Thomas Wang, Olivier Dehaene, Mishig Davaadorj, Joel Lamy-Poirier, João Monteiro, Oleh Shliazhko, Nicola… [find](https://scholar.google.com/scholar?q=Raymond+Li%2C+Loubna+Ben+Allal%2C+Yangtian+Zi%2C+Niklas+Muennighoff%2C+Denis+Kocetkov%2C+Chenghao+Mou%2C+Marc+Marone%2C+Christopher+Akiki%2C+Jia+Li%2C+Jenny+Chim%2C+Qian+Liu%2C+Evgenii+Zheltonozhskii%2C+Terry+Yue+Zhuo%2C+Thoma)
79. Yujia Li, David H. Choi, Junyoung Chung, Nate Kushman, Julian Schrittwieser, Rémi Leblond, Tom Eccles, James Keeling, Felix Gimeno, Agustin Dal Lago, Thomas Hubert, Peter Choy, Cyprien de Masson d’Autume, Igor Babuschkin, Xinyun Chen, Po-Sen Huang, Johannes Welbl, Sven Gowal, Alexey Cherepanov, Jam… [find](https://scholar.google.com/scholar?q=Yujia+Li%2C+David+H%2E+Choi%2C+Junyoung+Chung%2C+Nate+Kushman%2C+Julian+Schrittwieser%2C+Re%CC%81mi+Leblond%2C+Tom+Eccles%2C+James+Keeling%2C+Felix+Gimeno%2C+Agustin+Dal+Lago%2C+Thomas+Hubert%2C+Peter+Choy%2C+Cyprien+de+Masson+d%E2%80%99Au) [our card](https://md.research.elamur.ai/free/en/papers/li-2022-competition-level-code-generation-alphacode/)
80. Hunter Lightman, Vineet Kosaraju, Yura Burda, Harri Edwards, Bowen Baker, Teddy Lee, Jan Leike, John Schulman, Ilya Sutskever, and Karl Cobbe. Let’s verify step by step. arXiv preprint arXiv:2305.20050, 2023. [find](https://scholar.google.com/scholar?q=Hunter+Lightman%2C+Vineet+Kosaraju%2C+Yura+Burda%2C+Harri+Edwards%2C+Bowen+Baker%2C+Teddy+Lee%2C+Jan+Leike%2C+John+Schulman%2C+Ilya+Sutskever%2C+and+Karl+Cobbe%2E+Let%E2%80%99s+verify+step+by+step%2E+arXiv+preprint+arXiv%3A2305%2E2005) [our card](https://md.research.elamur.ai/free/en/papers/lightman-2023-let-s-verify-step-step/)
81. Chenxiao Liu and Xiaojun Wan. CodeQA: A question answering dataset for source code comprehension. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic… [find](https://scholar.google.com/scholar?q=Chenxiao+Liu+and+Xiaojun+Wan%2E+CodeQA%3A+A+question+answering+dataset+for+source+code+comprehension%2E+In+Marie%2DFrancine+Moens%2C+Xuanjing+Huang%2C+Lucia+Specia%2C+and+Scott+Wen%2Dtau+Yih+%28eds%2E%29%2C+Findings+of+the+A)
82. Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. arXiv preprint arXiv:2304.08485, 2023a. [find](https://scholar.google.com/scholar?q=Haotian+Liu%2C+Chunyuan+Li%2C+Qingyang+Wu%2C+and+Yong+Jae+Lee%2E+Visual+instruction+tuning%2E+arXiv+preprint+arXiv%3A2304%2E08485%2C+2023a%2E) [our card](https://md.research.elamur.ai/free/en/papers/liu-2023-visual-instruction-tuning/)
83. Xiao Liu, Hanyu Lai, Hao Yu, Yifan Xu, Aohan Zeng, Zhengxiao Du, Peng Zhang, Yuxiao Dong, and Jie Tang. WebGLM: Towards an efficient web-enhanced question answering system with human preferences. arXiv preprint arXiv:2306.07906, 2023b. [find](https://scholar.google.com/scholar?q=Xiao+Liu%2C+Hanyu+Lai%2C+Hao+Yu%2C+Yifan+Xu%2C+Aohan+Zeng%2C+Zhengxiao+Du%2C+Peng+Zhang%2C+Yuxiao+Dong%2C+and+Jie+Tang%2E+WebGLM%3A+Towards+an+efficient+web%2Denhanced+question+answering+system+with+human+preferences%2E+arXi)
84. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. RoBERTa: A robustly optimized BERT pretraining approach. arXiv preprint arXiv:1907.11692, 2019. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+RoBERTa%3A+A+robustly+optimized+BERT+pretraining+approach%2E+arXiv+p)
85. Yue Liu, Thanh Le-Cong, Ratnadira Widyasari, Chakkrit Tantithamthavorn, Li Li, Xuan-Bach Dinh Le, and David Lo. Refining ChatGPT-generated code: Characterizing and mitigating code quality issues. CoRR, abs/2307.12596, 2023c. doi: 10.48550/arXiv.2307.12596. URL https://doi.org/10.48550/arXiv.2307.12… — [doi:10.48550/arxiv.2307.12596](https://doi.org/10.48550/arxiv.2307.12596) [↗](https://doi.org/10.48550/arXiv.2307.12…)
86. Shayne Longpre, Le Hou, Tu Vu, Albert Webson, Hyung Won Chung, Yi Tay, Denny Zhou, Quoc V Le, Barret Zoph, Jason Wei, et al. The Flan collection: Designing data and methods for effective instruction tuning. arXiv preprint arXiv:2301.13688, 2023. [find](https://scholar.google.com/scholar?q=Shayne+Longpre%2C+Le+Hou%2C+Tu+Vu%2C+Albert+Webson%2C+Hyung+Won+Chung%2C+Yi+Tay%2C+Denny+Zhou%2C+Quoc+V+Le%2C+Barret+Zoph%2C+Jason+Wei%2C+et+al%2E+The+Flan+collection%3A+Designing+data+and+methods+for+effective+instruction+t) [our card](https://md.research.elamur.ai/free/en/papers/longpre-2023-flan-collection-designing-data-methods/)
87. Ilya Loshchilov and Frank Hutter. arXiv:1711.05101, 2017. [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+arXiv%3A1711%2E05101%2C+2017%2E)
88. Decoupled weight decay regularization. arXiv preprint Keming Lu, Hongyi Yuan, Zheng Yuan, Runji Lin, Junyang Lin, Chuanqi Tan, Chang Zhou, and Jingren Zhou. \#InsTag: Instruction tagging for analyzing supervised fine-tuning of large language models. CoRR, abs/2308.07074, 2023. doi: 10.48550/arXiv.230… [find](https://scholar.google.com/scholar?q=Decoupled+weight+decay+regularization%2E+arXiv+preprint+Keming+Lu%2C+Hongyi+Yuan%2C+Zheng+Yuan%2C+Runji+Lin%2C+Junyang+Lin%2C+Chuanqi+Tan%2C+Chang+Zhou%2C+and+Jingren+Zhou%2E+%5C%23InsTag%3A+Instruction+tagging+for+analyzing)
89. Haipeng Luo, Qingfeng Sun, Can Xu, Pu Zhao, Jianguang Lou, Chongyang Tao, Xiubo Geng, Qingwei Lin, Shifeng Chen, and Dongmei Zhang. WizardMath: Empowering mathematical reasoning for large language models via reinforced evol-instruct. arXiv preprint arXiv:2308.09583, 2023a. [find](https://scholar.google.com/scholar?q=Haipeng+Luo%2C+Qingfeng+Sun%2C+Can+Xu%2C+Pu+Zhao%2C+Jianguang+Lou%2C+Chongyang+Tao%2C+Xiubo+Geng%2C+Qingwei+Lin%2C+Shifeng+Chen%2C+and+Dongmei+Zhang%2E+WizardMath%3A+Empowering+mathematical+reasoning+for+large+language+mod) [our card](https://md.research.elamur.ai/free/en/papers/luo-2023-wizardmath-empowering-mathematical-reasoning-large/)
90. Ziyang Luo, Can Xu, Pu Zhao, Qingfeng Sun, Xiubo Geng, Wenxiang Hu, Chongyang Tao, Jing Ma, Qingwei Lin, and Daxin Jiang. WizardCoder: Empowering code large language models with evol-instruct. arXiv preprint arXiv:2306.08568, 2023b. [find](https://scholar.google.com/scholar?q=Ziyang+Luo%2C+Can+Xu%2C+Pu+Zhao%2C+Qingfeng+Sun%2C+Xiubo+Geng%2C+Wenxiang+Hu%2C+Chongyang+Tao%2C+Jing+Ma%2C+Qingwei+Lin%2C+and+Daxin+Jiang%2E+WizardCoder%3A+Empowering+code+large+language+models+with+evol%2Dinstruct%2E+arXiv+p)
91. Mosaic ML. MPT-30B: Raising the bar for open-source foundation models, 2023. URL [https://www.mosaicml.com/blog/mpt-30b](https://www.mosaicml.com/blog/mpt-30b). [↗](https://www.mosaicml.com/blog/mpt-30b)
92. Niklas Muennighoff, Thomas Wang, Lintang Sutawika, Adam Roberts, Stella Biderman, Teven Le Scao, M Saiful Bari, Sheng Shen, Zheng-Xin Yong, Hailey Schoelkopf, et al. Crosslingual generalization through multitask finetuning. arXiv preprint arXiv:2211.01786, 2022. [find](https://scholar.google.com/scholar?q=Niklas+Muennighoff%2C+Thomas+Wang%2C+Lintang+Sutawika%2C+Adam+Roberts%2C+Stella+Biderman%2C+Teven+Le+Scao%2C+M+Saiful+Bari%2C+Sheng+Shen%2C+Zheng%2DXin+Yong%2C+Hailey+Schoelkopf%2C+et+al%2E+Crosslingual+generalization+throug) [our card](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-crosslingual-generalization-through-multitask-finetuning/)
93. Niklas Muennighoff, Qian Liu, Armel Zebaze, Qinkai Zheng, Binyuan Hui, Terry Yue Zhuo, Swayam Singh, Xiangru Tang, Leandro von Werra, and Shayne Longpre. OctoPack: Instruction tuning code large language models. CoRR, abs/2308.07124, 2023. [find](https://scholar.google.com/scholar?q=Niklas+Muennighoff%2C+Qian+Liu%2C+Armel+Zebaze%2C+Qinkai+Zheng%2C+Binyuan+Hui%2C+Terry+Yue+Zhuo%2C+Swayam+Singh%2C+Xiangru+Tang%2C+Leandro+von+Werra%2C+and+Shayne+Longpre%2E+OctoPack%3A+Instruction+tuning+code+large+langua)
94. Reiichiro Nakano, Jacob Hilton, Suchir Balaji, Jeff Wu, Long Ouyang, Christina Kim, Christopher Hesse, Shantanu Jain, Vineet Kosaraju, William Saunders, et al. WebGPT: Browser-assisted question-answering with human feedback. arXiv preprint arXiv:2112.09332, 2021. [find](https://scholar.google.com/scholar?q=Reiichiro+Nakano%2C+Jacob+Hilton%2C+Suchir+Balaji%2C+Jeff+Wu%2C+Long+Ouyang%2C+Christina+Kim%2C+Christopher+Hesse%2C+Shantanu+Jain%2C+Vineet+Kosaraju%2C+William+Saunders%2C+et+al%2E+WebGPT%3A+Browser%2Dassisted+question%2Danswer) [our card](https://md.research.elamur.ai/free/en/papers/nakano-2021-webgpt-browser-assisted-question-answering/)
95. Maxwell Nye, Anders Andreassen, Guy Gur-Ari, Henryk Michalewski, Jacob Austin, David Bieber, David Dohan, Aitor Lewkowycz, Maarten Bosma, David Luan, Charles Sutton, and Augustus Odena. Show your work: Scratchpads for intermediate computation with language models. ArXiv, abs/2112.00114, 2021. [find](https://scholar.google.com/scholar?q=Maxwell+Nye%2C+Anders+Andreassen%2C+Guy+Gur%2DAri%2C+Henryk+Michalewski%2C+Jacob+Austin%2C+David+Bieber%2C+David+Dohan%2C+Aitor+Lewkowycz%2C+Maarten+Bosma%2C+David+Luan%2C+Charles+Sutton%2C+and+Augustus+Odena%2E+Show+your+work) [our card](https://md.research.elamur.ai/free/en/papers/al-2021-show-your-work-scratchpads-intermediate/)
96. OpenAI. Introducing ChatGPT, 2022. URL [https://openai.com/blog/chatgpt](https://openai.com/blog/chatgpt). [↗](https://openai.com/blog/chatgpt)
97. OpenAI. ChatML, 2022. URL [https://github.com/openai/openai-python/blob/](https://github.com/openai/openai-python/blob/) e389823ba013a24b4c32ce38fa0bd87e6bccae94/chatml.md. [↗](https://github.com/openai/openai-python/blob/)
98. OpenAI. GPT4 technical report. arXiv preprint arXiv:2303.08774, 2023. [find](https://scholar.google.com/scholar?q=OpenAI%2E+GPT4+technical+report%2E+arXiv+preprint+arXiv%3A2303%2E08774%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/)
99. OpenCompass Team. OpenCompass: A universal evaluation platform for foundation models, 2023. URL [https://opencompass.org.cn/leaderboard-llm](https://opencompass.org.cn/leaderboard-llm). [↗](https://opencompass.org.cn/leaderboard-llm)
100. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll L. Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul F. Christiano, Jan Leike, and Ryan Lowe. Train… [find](https://scholar.google.com/scholar?q=Long+Ouyang%2C+Jeffrey+Wu%2C+Xu+Jiang%2C+Diogo+Almeida%2C+Carroll+L%2E+Wainwright%2C+Pamela+Mishkin%2C+Chong+Zhang%2C+Sandhini+Agarwal%2C+Katarina+Slama%2C+Alex+Ray%2C+John+Schulman%2C+Jacob+Hilton%2C+Fraser+Kelton%2C+Luke+Mille)
101. Denis Paperno, Germán Kruszewski, Angeliki Lazaridou, Quan Ngoc Pham, Raffaella Bernardi, Sandro Pezzelle, Marco Baroni, Gemma Boleda, and Raquel Fernández. The LAMBADA dataset: Word prediction requiring a broad discourse context. In Proceedings of the 54th Annual Meeting of the Association for Co… [find](https://scholar.google.com/scholar?q=Denis+Paperno%2C+Germa%CC%81n+Kruszewski%2C+Angeliki+Lazaridou%2C+Quan+Ngoc+Pham%2C+Raffaella+Bernardi%2C+Sandro+Pezzelle%2C+Marco+Baroni%2C+Gemma+Boleda%2C+and+Raquel+Ferna%CC%81ndez%2E+The+LAMBADA+dataset%3A+Word+prediction+requ) [our card](https://md.research.elamur.ai/free/en/papers/paperno-2016-lambada-dataset-word-prediction-requiring/)
102. Bowen Peng, Jeffrey Quesnelle, Honglu Fan, and Enrico Shippole. YaRN: Efficient context window extension of large language models. arXiv preprint arXiv:2309.00071, 2023a. [find](https://scholar.google.com/scholar?q=Bowen+Peng%2C+Jeffrey+Quesnelle%2C+Honglu+Fan%2C+and+Enrico+Shippole%2E+YaRN%3A+Efficient+context+window+extension+of+large+language+models%2E+arXiv+preprint+arXiv%3A2309%2E00071%2C+2023a%2E)
103. Zhiliang Peng, Wenhui Wang, Li Dong, Yaru Hao, Shaohan Huang, Shuming Ma, and Furu Wei. Kosmos-2: Grounding multimodal large language models to the world. arXiv preprint arXiv:2306.14824, 2023b. [find](https://scholar.google.com/scholar?q=Zhiliang+Peng%2C+Wenhui+Wang%2C+Li+Dong%2C+Yaru+Hao%2C+Shaohan+Huang%2C+Shuming+Ma%2C+and+Furu+Wei%2E+Kosmos%2D2%3A+Grounding+multimodal+large+language+models+to+the+world%2E+arXiv+preprint+arXiv%3A2306%2E14824%2C+2023b%2E) [our card](https://md.research.elamur.ai/free/en/papers/peng-2023-kosmos-2-grounding-multimodal-large/)
104. Qwen Team, Alibaba Group. Evaluation benchmark for code intepreter, 2023a. URL [https://github.com/QwenLM/Qwen-Agent/tree/main/benchmark](https://github.com/QwenLM/Qwen-Agent/tree/main/benchmark). [↗](https://github.com/QwenLM/Qwen-Agent/tree/main/benchmark)
105. Qwen Team, Alibaba Group. Evaluation benchmark for tool usage through ReAct prompting, 2023b. URL [https://github.com/QwenLM/Qwen-7B/tree/main/eval](https://github.com/QwenLM/Qwen-7B/tree/main/eval). [↗](https://github.com/QwenLM/Qwen-7B/tree/main/eval)
106. Alec Radford, Karthik Narasimhan, Tim Salimans, Ilya Sutskever, et al. Improving language understanding by generative pre-training. Technical report, OpenAI, 2018. [find](https://scholar.google.com/scholar?q=Alec+Radford%2C+Karthik+Narasimhan%2C+Tim+Salimans%2C+Ilya+Sutskever%2C+et+al%2E+Improving+language+understanding+by+generative+pre%2Dtraining%2E+Technical+report%2C+OpenAI%2C+2018%2E)
107. Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446, 2021. [find](https://scholar.google.com/scholar?q=Jack+W+Rae%2C+Sebastian+Borgeaud%2C+Trevor+Cai%2C+Katie+Millican%2C+Jordan+Hoffmann%2C+Francis+Song%2C+John+Aslanides%2C+Sarah+Henderson%2C+Roman+Ring%2C+Susannah+Young%2C+et+al%2E+Scaling+language+models%3A+Methods%2C+analysi)
108. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn. Direct preference optimization: Your language model is secretly a reward model. arXiv preprint arXiv:2305.18290, 2023. [find](https://scholar.google.com/scholar?q=Rafael+Rafailov%2C+Archit+Sharma%2C+Eric+Mitchell%2C+Stefano+Ermon%2C+Christopher+D+Manning%2C+and+Chelsea+Finn%2E+Direct+preference+optimization%3A+Your+language+model+is+secretly+a+reward+model%2E+arXiv+preprint+ar)
109. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J Liu. Exploring the limits of transfer learning with a unified text-to-text transformer. The Journal of Machine Learning Research, 21(1):5485–5551, 2020. [find](https://scholar.google.com/scholar?q=Colin+Raffel%2C+Noam+Shazeer%2C+Adam+Roberts%2C+Katherine+Lee%2C+Sharan+Narang%2C+Michael+Matena%2C+Yanqi+Zhou%2C+Wei+Li%2C+and+Peter+J+Liu%2E+Exploring+the+limits+of+transfer+learning+with+a+unified+text%2Dto%2Dtext+trans)
110. Prajit Ramachandran, Barret Zoph, and Quoc V Le. Searching for activation functions. arXiv preprint arXiv:1710.05941, 2017. [find](https://scholar.google.com/scholar?q=Prajit+Ramachandran%2C+Barret+Zoph%2C+and+Quoc+V+Le%2E+Searching+for+activation+functions%2E+arXiv+preprint+arXiv%3A1710%2E05941%2C+2017%2E)
111. Scott E. Reed, Konrad Zolna, Emilio Parisotto, Sergio Gómez Colmenarejo, Alexander Novikov, Gabriel Barth-Maron, Mai Gimenez, Yury Sulsky, Jackie Kay, Jost Tobias Springenberg, Tom Eccles, Jake Bruce, Ali Razavi, Ashley Edwards, Nicolas Heess, Yutian Chen, Raia Hadsell, Oriol Vinyals, Mahyar Bordba… [find](https://scholar.google.com/scholar?q=Scott+E%2E+Reed%2C+Konrad+Zolna%2C+Emilio+Parisotto%2C+Sergio+Go%CC%81mez+Colmenarejo%2C+Alexander+Novikov%2C+Gabriel+Barth%2DMaron%2C+Mai+Gimenez%2C+Yury+Sulsky%2C+Jackie+Kay%2C+Jost+Tobias+Springenberg%2C+Tom+Eccles%2C+Jake+Bruce)
112. Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, et al. Code Llama: Open foundation models for code. arXiv preprint arXiv:2308.12950, 2023. [find](https://scholar.google.com/scholar?q=Baptiste+Rozie%CC%80re%2C+Jonas+Gehring%2C+Fabian+Gloeckle%2C+Sten+Sootla%2C+Itai+Gat%2C+Xiaoqing+Ellen+Tan%2C+Yossi+Adi%2C+Jingyu+Liu%2C+Tal+Remez%2C+Je%CC%81re%CC%81my+Rapin%2C+et+al%2E+Code+Llama%3A+Open+foundation+models+for+code%2E+arXi) [our card](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/)
113. Victor Sanh, Albert Webson, Colin Raffel, Stephen H Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. Multitask prompted training enables zero-shot task generalization. arXiv preprint arXiv:2110.08207, 2021. [find](https://scholar.google.com/scholar?q=Victor+Sanh%2C+Albert+Webson%2C+Colin+Raffel%2C+Stephen+H+Bach%2C+Lintang+Sutawika%2C+Zaid+Alyafeai%2C+Antoine+Chaffin%2C+Arnaud+Stiegler%2C+Teven+Le+Scao%2C+Arun+Raja%2C+et+al%2E+Multitask+prompted+training+enables+zero%2Ds) [our card](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)
114. Maarten Sap, Hannah Rashkin, Derek Chen, Ronan Le Bras, and Yejin Choi. SocialIQA: Commonsense reasoning about social interactions. CoRR, abs/1904.09728, 2019. URL http://arxiv.org/abs/1904.09728. [↗](http://arxiv.org/abs/1904.09728) [our card](https://md.research.elamur.ai/free/en/papers/sap-2019-socialiqa-commonsense-reasoning-about-social/)
115. Teven Le Scao, Angela Fan, Christopher Akiki, Ellie Pavlick, Suzana Ilić, Daniel Hesslow, Roman Castagné, Alexandra Sasha Luccioni, François Yvon, Matthias Gallé, et al. BLOOM: A 176B-parameter open-access multilingual language model. arXiv preprint arXiv:2211.05100, 2022. [find](https://scholar.google.com/scholar?q=Teven+Le+Scao%2C+Angela+Fan%2C+Christopher+Akiki%2C+Ellie+Pavlick%2C+Suzana+Ilic%CC%81%2C+Daniel+Hesslow%2C+Roman+Castagne%CC%81%2C+Alexandra+Sasha+Luccioni%2C+Franc%CC%A7ois+Yvon%2C+Matthias+Galle%CC%81%2C+et+al%2E+BLOOM%3A+A+176B%2Dparameter+op)
116. Timo Schick, Jane Dwivedi-Yu, Roberto Dessı̀, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. Toolformer: Language models can teach themselves to use tools. arXiv preprint arXiv:2302.04761, 2023. [find](https://scholar.google.com/scholar?q=Timo+Schick%2C+Jane+Dwivedi%2DYu%2C+Roberto+Dess%C4%B1%CC%80%2C+Roberta+Raileanu%2C+Maria+Lomeli%2C+Luke+Zettlemoyer%2C+Nicola+Cancedda%2C+and+Thomas+Scialom%2E+Toolformer%3A+Language+models+can+teach+themselves+to+use+tools%2E+arXi) [our card](https://md.research.elamur.ai/free/en/papers/schick-2023-toolformer-language-models-can-teach/)
117. John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov. Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347, 2017. [find](https://scholar.google.com/scholar?q=John+Schulman%2C+Filip+Wolski%2C+Prafulla+Dhariwal%2C+Alec+Radford%2C+and+Oleg+Klimov%2E+Proximal+policy+optimization+algorithms%2E+arXiv+preprint+arXiv%3A1707%2E06347%2C+2017%2E)
118. Noam Shazeer. GLU variants improve transformer. arXiv preprint arXiv:2002.05202, 2020. [find](https://scholar.google.com/scholar?q=Noam+Shazeer%2E+GLU+variants+improve+transformer%2E+arXiv+preprint+arXiv%3A2002%2E05202%2C+2020%2E)
119. Yongliang Shen, Kaitao Song, Xu Tan, Dongsheng Li, Weiming Lu, and Yueting Zhuang. HuggingGPT: Solving AI tasks with ChatGPT and its friends in HuggingFace. arXiv preprint arXiv:2303.17580, 2023. [find](https://scholar.google.com/scholar?q=Yongliang+Shen%2C+Kaitao+Song%2C+Xu+Tan%2C+Dongsheng+Li%2C+Weiming+Lu%2C+and+Yueting+Zhuang%2E+HuggingGPT%3A+Solving+AI+tasks+with+ChatGPT+and+its+friends+in+HuggingFace%2E+arXiv+preprint+arXiv%3A2303%2E17580%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/shen-2023-hugginggpt-solving-ai-tasks-chatgpt/)
120. Mohammad Shoeybi, Mostofa Patwary, Raul Puri, Patrick LeGresley, Jared Casper, and Bryan Catanzaro. Megatron-LM: Training multi-billion parameter language models using model parallelism. arXiv preprint arXiv:1909.08053, 2019. [find](https://scholar.google.com/scholar?q=Mohammad+Shoeybi%2C+Mostofa+Patwary%2C+Raul+Puri%2C+Patrick+LeGresley%2C+Jared+Casper%2C+and+Bryan+Catanzaro%2E+Megatron%2DLM%3A+Training+multi%2Dbillion+parameter+language+models+using+model+parallelism%2E+arXiv+preprin)
121. Qingyi Si, Tong Wang, Naibin Gu, Rui Liu, and Zheng Lin. Alpaca-CoT: An instruction-tuning platform with unified interface of instruction collection, parameter-efficient methods, and large language models, 2023. URL [https://github.com/PhoebusSi/alpaca-CoT](https://github.com/PhoebusSi/alpaca-CoT). [↗](https://github.com/PhoebusSi/alpaca-CoT)
122. Feifan Song, Bowen Yu, Minghao Li, Haiyang Yu, Fei Huang, Yongbin Li, and Houfeng Wang. Preference ranking optimization for human alignment. arXiv preprint arXiv:2306.17492, 2023. [find](https://scholar.google.com/scholar?q=Feifan+Song%2C+Bowen+Yu%2C+Minghao+Li%2C+Haiyang+Yu%2C+Fei+Huang%2C+Yongbin+Li%2C+and+Houfeng+Wang%2E+Preference+ranking+optimization+for+human+alignment%2E+arXiv+preprint+arXiv%3A2306%2E17492%2C+2023%2E) [our card](https://md.research.elamur.ai/free/en/papers/song-2023-preference-ranking-optimization-human-alignment/)
123. Stability AI. StableBeluga2, 2023. StableBeluga2. [find](https://scholar.google.com/scholar?q=Stability+AI%2E+StableBeluga2%2C+2023%2E+StableBeluga2%2E)
124. URL [https://huggingface.co/stabilityai/Nisan](https://huggingface.co/stabilityai/Nisan) Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. Advances in Neural Information Processing Systems, 33:3008–3021, 2020. [↗](https://huggingface.co/stabilityai/Nisan)
125. Jianlin Su. Improving transformer: Length extrapolation ability and position robustness, 2023a. URL [https://spaces.ac.cn/archives/9444](https://spaces.ac.cn/archives/9444). [↗](https://spaces.ac.cn/archives/9444)
126. Jianlin Su. The magical effect of the Bias term: RoPE + Bias = better length extrapolation, 2023b. URL [https://spaces.ac.cn/archives/9577](https://spaces.ac.cn/archives/9577). [↗](https://spaces.ac.cn/archives/9577)
127. Jianlin Su, Yu Lu, Shengfeng Pan, Ahmed Murtadha, Bo Wen, and Yunfeng Liu. Roformer: Enhanced transformer with rotary position embedding. arXiv preprint arXiv:2104.09864, 2021. [find](https://scholar.google.com/scholar?q=Jianlin+Su%2C+Yu+Lu%2C+Shengfeng+Pan%2C+Ahmed+Murtadha%2C+Bo+Wen%2C+and+Yunfeng+Liu%2E+Roformer%3A+Enhanced+transformer+with+rotary+position+embedding%2E+arXiv+preprint+arXiv%3A2104%2E09864%2C+2021%2E)
128. Tianxiang Sun, Xiaotian Zhang, Zhengfu He, Peng Li, Qinyuan Cheng, Hang Yan, Xiangyang Liu, Yunfan Shao, Qiong Tang, Xingjian Zhao, Ke Chen, Yining Zheng, Zhejian Zhou, Ruixiao Li, Jun Zhan, Yunhua Zhou, Linyang Li, Xiaogui Yang, Lingling Wu, Zhangyue Yin, Xuanjing Huang, and Xipeng Qiu. MOSS: Train… [find](https://scholar.google.com/scholar?q=Tianxiang+Sun%2C+Xiaotian+Zhang%2C+Zhengfu+He%2C+Peng+Li%2C+Qinyuan+Cheng%2C+Hang+Yan%2C+Xiangyang+Liu%2C+Yunfan+Shao%2C+Qiong+Tang%2C+Xingjian+Zhao%2C+Ke+Chen%2C+Yining+Zheng%2C+Zhejian+Zhou%2C+Ruixiao+Li%2C+Jun+Zhan%2C+Yunhua+Zh)
129. Zhiqing Sun, Yikang Shen, Qinhong Zhou, Hongxin Zhang, Zhenfang Chen, David Cox, Yiming Yang, and Chuang Gan. Principle-driven self-alignment of language models from scratch with minimal human supervision. arXiv preprint arXiv:2305.03047, 2023b. [find](https://scholar.google.com/scholar?q=Zhiqing+Sun%2C+Yikang+Shen%2C+Qinhong+Zhou%2C+Hongxin+Zhang%2C+Zhenfang+Chen%2C+David+Cox%2C+Yiming+Yang%2C+and+Chuang+Gan%2E+Principle%2Ddriven+self%2Dalignment+of+language+models+from+scratch+with+minimal+human+supervi) [our card](https://md.research.elamur.ai/free/en/papers/sun-2023-principle-driven-self-alignment-language/)
130. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc V Le, Ed H Chi, Denny Zhou, et al. Challenging big-bench tasks and whether chain-of-thought can solve them. arXiv preprint arXiv:2210.09261, 2022. [find](https://scholar.google.com/scholar?q=Mirac+Suzgun%2C+Nathan+Scales%2C+Nathanael+Scha%CC%88rli%2C+Sebastian+Gehrmann%2C+Yi+Tay%2C+Hyung+Won+Chung%2C+Aakanksha+Chowdhery%2C+Quoc+V+Le%2C+Ed+H+Chi%2C+Denny+Zhou%2C+et+al%2E+Challenging+big%2Dbench+tasks+and+whether+chain) [our card](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/)
131. Marc Szafraniec, Baptiste Rozière, Hugh Leather, Patrick Labatut, François Charton, and Gabriel Synnaeve. Code translation with compiler representations. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. [find](https://scholar.google.com/scholar?q=Marc+Szafraniec%2C+Baptiste+Rozie%CC%80re%2C+Hugh+Leather%2C+Patrick+Labatut%2C+Franc%CC%A7ois+Charton%2C+and+Gabriel+Synnaeve%2E+Code+translation+with+compiler+representations%2E+In+The+Eleventh+International+Conference+on+)
132. OpenReview.net, 2023. URL [https://openreview.net/pdf?id=XomEU3eNeSQ](https://openreview.net/pdf?id=XomEU3eNeSQ). [↗](https://openreview.net/pdf?id=XomEU3eNeSQ)
133. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. CommonsenseQA: A question answering challenge targeting commonsense knowledge. In Jill Burstein, Christy Doran, and Thamar Solorio (eds.), Proceedings of the 2019 Conference of the North American Chapter of the Association for Compu… [find](https://scholar.google.com/scholar?q=Alon+Talmor%2C+Jonathan+Herzig%2C+Nicholas+Lourie%2C+and+Jonathan+Berant%2E+CommonsenseQA%3A+A+question+answering+challenge+targeting+commonsense+knowledge%2E+In+Jill+Burstein%2C+Christy+Doran%2C+and+Thamar+Solorio+%28)
134. Rohan Taori, Ishaan Gulrajani, Tianyi Zhang, Yann Dubois, Xuechen Li, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. Stanford Alpaca: An instruction-following LLaMA model, 2023. URL [https://github.com/tatsu-lab/stanford_alpaca](https://github.com/tatsu-lab/stanford_alpaca). [↗](https://github.com/tatsu-lab/stanford_alpaca)
135. Ross Taylor, Marcin Kardas, Guillem Cucurull, Thomas Scialom, Anthony Hartshorn, Elvis Saravia, Andrew Poulton, Viktor Kerkez, and Robert Stojnic. Galactica: A large language model for science, 2022. [find](https://scholar.google.com/scholar?q=Ross+Taylor%2C+Marcin+Kardas%2C+Guillem+Cucurull%2C+Thomas+Scialom%2C+Anthony+Hartshorn%2C+Elvis+Saravia%2C+Andrew+Poulton%2C+Viktor+Kerkez%2C+and+Robert+Stojnic%2E+Galactica%3A+A+large+language+model+for+science%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/taylor-2022-galactica-large-language-model-science/)
136. Romal Thoppilan, Daniel De Freitas, Jamie Hall, Noam Shazeer, Apoorv Kulshreshtha, Heng-Tze Cheng, Alicia Jin, Taylor Bos, Leslie Baker, Yu Du, YaGuang Li, Hongrae Lee, Huaixiu Steven Zheng, Amin Ghafouri, Marcelo Menegali, Yanping Huang, Maxim Krikun, Dmitry Lepikhin, James Qin, Dehao Chen, Yuanzho… [find](https://scholar.google.com/scholar?q=Romal+Thoppilan%2C+Daniel+De+Freitas%2C+Jamie+Hall%2C+Noam+Shazeer%2C+Apoorv+Kulshreshtha%2C+Heng%2DTze+Cheng%2C+Alicia+Jin%2C+Taylor+Bos%2C+Leslie+Baker%2C+Yu+Du%2C+YaGuang+Li%2C+Hongrae+Lee%2C+Huaixiu+Steven+Zheng%2C+Amin+Ghaf) [our card](https://md.research.elamur.ai/free/en/papers/thoppilan-2022-lamda-language-models-dialog-applications/)
137. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al. LLaMA: Open and efficient foundation language models. arXiv preprint arXiv:2302.13971, 2023a. [find](https://scholar.google.com/scholar?q=Hugo+Touvron%2C+Thibaut+Lavril%2C+Gautier+Izacard%2C+Xavier+Martinet%2C+Marie%2DAnne+Lachaux%2C+Timothe%CC%81e+Lacroix%2C+Baptiste+Rozie%CC%80re%2C+Naman+Goyal%2C+Eric+Hambro%2C+Faisal+Azhar%2C+et+al%2E+LLaMA%3A+Open+and+efficient+found)
138. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton-Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… [find](https://scholar.google.com/scholar?q=Hugo+Touvron%2C+Louis+Martin%2C+Kevin+Stone%2C+Peter+Albert%2C+Amjad+Almahairi%2C+Yasmine+Babaei%2C+Nikolay+Bashlykov%2C+Soumya+Batra%2C+Prajjwal+Bhargava%2C+Shruti+Bhosale%2C+Dan+Bikel%2C+Lukas+Blecher%2C+Cristian+Canton%2DFe)
139. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. [find](https://scholar.google.com/scholar?q=Ashish+Vaswani%2C+Noam+Shazeer%2C+Niki+Parmar%2C+Jakob+Uszkoreit%2C+Llion+Jones%2C+Aidan+N+Gomez%2C+%C5%81ukasz+Kaiser%2C+and+Illia+Polosukhin%2E+Attention+is+all+you+need%2E+Advances+in+neural+information+processing+system)
140. Guanzhi Wang, Yuqi Xie, Yunfan Jiang, Ajay Mandlekar, Chaowei Xiao, Yuke Zhu, Linxi Fan, and Anima Anandkumar. Voyager: An open-ended embodied agent with large language models. arXiv preprint arXiv:2305.16291, 2023a. [find](https://scholar.google.com/scholar?q=Guanzhi+Wang%2C+Yuqi+Xie%2C+Yunfan+Jiang%2C+Ajay+Mandlekar%2C+Chaowei+Xiao%2C+Yuke+Zhu%2C+Linxi+Fan%2C+and+Anima+Anandkumar%2E+Voyager%3A+An+open%2Dended+embodied+agent+with+large+language+models%2E+arXiv+preprint+arXiv%3A23)
141. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Huai hsin Chi, and Denny Zhou. Self-consistency improves chain of thought reasoning in language models. ArXiv, abs/2203.11171, 2022. [find](https://scholar.google.com/scholar?q=Xuezhi+Wang%2C+Jason+Wei%2C+Dale+Schuurmans%2C+Quoc+Le%2C+Ed+Huai+hsin+Chi%2C+and+Denny+Zhou%2E+Self%2Dconsistency+improves+chain+of+thought+reasoning+in+language+models%2E+ArXiv%2C+abs%2F2203%2E11171%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/)
142. Yan Wang, Xiaojiang Liu, and Shuming Shi. Deep neural solver for math word problems. In Conference on Empirical Methods in Natural Language Processing, 2017. URL [https://api.semanticscholar.org/CorpusID:910689](https://api.semanticscholar.org/CorpusID:910689). [↗](https://api.semanticscholar.org/CorpusID:910689)
143. Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Raghavi Chandu, David Wadden, Kelsey MacMillan, Noah A Smith, Iz Beltagy, et al. How far can camels go? Exploring the state of instruction tuning on open resources. arXiv preprint arXiv:2306.04751, 2023b. [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Hamish+Ivison%2C+Pradeep+Dasigi%2C+Jack+Hessel%2C+Tushar+Khot%2C+Khyathi+Raghavi+Chandu%2C+David+Wadden%2C+Kelsey+MacMillan%2C+Noah+A+Smith%2C+Iz+Beltagy%2C+et+al%2E+How+far+can+camels+go%3F+Exploring+the+sta) [our card](https://md.research.elamur.ai/free/en/papers/wang-2023-far-camels-go-exploring-state/)
144. Yizhong Wang, Yeganeh Kordi, Swaroop Mishra, Alisa Liu, Noah A. Smith, Daniel Khashabi, and Hannaneh Hajishirzi. Self-Instruct: Aligning language models with self-generated instructions. In Anna Rogers, Jordan L. Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the A… [find](https://scholar.google.com/scholar?q=Yizhong+Wang%2C+Yeganeh+Kordi%2C+Swaroop+Mishra%2C+Alisa+Liu%2C+Noah+A%2E+Smith%2C+Daniel+Khashabi%2C+and+Hannaneh+Hajishirzi%2E+Self%2DInstruct%3A+Aligning+language+models+with+self%2Dgenerated+instructions%2E+In+Anna+Roger)
145. Yue Wang, Weishi Wang, Shafiq Joty, and Steven CH Hoi. CodeT5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. arXiv preprint arXiv:2109.00859, 2021. [find](https://scholar.google.com/scholar?q=Yue+Wang%2C+Weishi+Wang%2C+Shafiq+Joty%2C+and+Steven+CH+Hoi%2E+CodeT5%3A+Identifier%2Daware+unified+pre%2Dtrained+encoder%2Ddecoder+models+for+code+understanding+and+generation%2E+arXiv+preprint+arXiv%3A2109%2E00859%2C+2021%2E)
146. Yue Wang, Hung Le, Akhilesh Deepak Gotmare, Nghi D. Q. Bui, Junnan Li, and Steven C. H. Hoi. CodeT5+: Open code large language models for code understanding and generation. CoRR, abs/2305.07922, 2023d. doi: 10.48550/arXiv.2305.07922. URL https://doi.org/10. 48550/arXiv.2305.07922. [find](https://scholar.google.com/scholar?q=Yue+Wang%2C+Hung+Le%2C+Akhilesh+Deepak+Gotmare%2C+Nghi+D%2E+Q%2E+Bui%2C+Junnan+Li%2C+and+Steven+C%2E+H%2E+Hoi%2E+CodeT5%2B%3A+Open+code+large+language+models+for+code+understanding+and+generation%2E+CoRR%2C+abs%2F2305%2E07922%2C+2023d)
147. Jason Wei, Maarten Bosma, Vincent Y. Zhao, Kelvin Guu, Adams Wei Yu, Brian Lester, Nan Du, Andrew M. Dai, and Quoc V. Le. Finetuned language models are zero-shot learners. In The Tenth International Conference on Learning Representations, ICLR 2022, Virtual Event, April 25-29, 2022. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Maarten+Bosma%2C+Vincent+Y%2E+Zhao%2C+Kelvin+Guu%2C+Adams+Wei+Yu%2C+Brian+Lester%2C+Nan+Du%2C+Andrew+M%2E+Dai%2C+and+Quoc+V%2E+Le%2E+Finetuned+language+models+are+zero%2Dshot+learners%2E+In+The+Tenth+International+C)
148. OpenReview.net, 2022a. URL [https://openreview.net/forum?id=](https://openreview.net/forum?id=) gEZrGCozdqR. [↗](https://openreview.net/forum?id=)
149. Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed Huai hsin Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, and William Fedus. Emergent abilities of large language models. Trans. Mach. Lear… [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Yi+Tay%2C+Rishi+Bommasani%2C+Colin+Raffel%2C+Barret+Zoph%2C+Sebastian+Borgeaud%2C+Dani+Yogatama%2C+Maarten+Bosma%2C+Denny+Zhou%2C+Donald+Metzler%2C+Ed+Huai+hsin+Chi%2C+Tatsunori+Hashimoto%2C+Oriol+Vinyals%2C+Percy) [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-emergent-abilities-large-language-models/)
150. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Fei Xia, Ed Chi, Quoc V Le, Denny Zhou, et al. Chain-of-thought prompting elicits reasoning in large language models. Advances in Neural Information Processing Systems, 35:24824–24837, 2022c. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Xuezhi+Wang%2C+Dale+Schuurmans%2C+Maarten+Bosma%2C+Fei+Xia%2C+Ed+Chi%2C+Quoc+V+Le%2C+Denny+Zhou%2C+et+al%2E+Chain%2Dof%2Dthought+prompting+elicits+reasoning+in+large+language+models%2E+Advances+in+Neural+Informa) [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
151. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. HuggingFace’s transformers: State-of-the-art natural language processing. arXiv preprint arXiv:1910.03771, 2019. [find](https://scholar.google.com/scholar?q=Thomas+Wolf%2C+Lysandre+Debut%2C+Victor+Sanh%2C+Julien+Chaumond%2C+Clement+Delangue%2C+Anthony+Moi%2C+Pierric+Cistac%2C+Tim+Rault%2C+Re%CC%81mi+Louf%2C+Morgan+Funtowicz%2C+et+al%2E+HuggingFace%E2%80%99s+transformers%3A+State%2Dof%2Dthe%2Dart+n) [our card](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
152. Benfeng Xu, An Yang, Junyang Lin, Quan Wang, Chang Zhou, Yongdong Zhang, and Zhendong Mao. ExpertPrompting: Instructing large language models to be distinguished experts. arXiv preprint arXiv:2305.14688, 2023a. [find](https://scholar.google.com/scholar?q=Benfeng+Xu%2C+An+Yang%2C+Junyang+Lin%2C+Quan+Wang%2C+Chang+Zhou%2C+Yongdong+Zhang%2C+and+Zhendong+Mao%2E+ExpertPrompting%3A+Instructing+large+language+models+to+be+distinguished+experts%2E+arXiv+preprint+arXiv%3A2305%2E146)
153. Can Xu, Qingfeng Sun, Kai Zheng, Xiubo Geng, Pu Zhao, Jiazhan Feng, Chongyang Tao, and Daxin Jiang. WizardLM: Empowering large language models to follow complex instructions. arXiv preprint arXiv:2304.12244, 2023b. [find](https://scholar.google.com/scholar?q=Can+Xu%2C+Qingfeng+Sun%2C+Kai+Zheng%2C+Xiubo+Geng%2C+Pu+Zhao%2C+Jiazhan+Feng%2C+Chongyang+Tao%2C+and+Daxin+Jiang%2E+WizardLM%3A+Empowering+large+language+models+to+follow+complex+instructions%2E+arXiv+preprint+arXiv%3A2304) [our card](https://md.research.elamur.ai/free/en/papers/xu-2023-wizardlm-empowering-large-pre-trained/)
154. Canwen Xu, Daya Guo, Nan Duan, and Julian McAuley. Baize: An open-source chat model with parameter-efficient tuning on self-chat data. arXiv preprint arXiv:2304.01196, 2023c. [find](https://scholar.google.com/scholar?q=Canwen+Xu%2C+Daya+Guo%2C+Nan+Duan%2C+and+Julian+McAuley%2E+Baize%3A+An+open%2Dsource+chat+model+with+parameter%2Defficient+tuning+on+self%2Dchat+data%2E+arXiv+preprint+arXiv%3A2304%2E01196%2C+2023c%2E) [our card](https://md.research.elamur.ai/free/en/papers/xu-2023-baize-open-source-chat-model/)
155. Yuzhuang Xu, Shuo Wang, Peng Li, Fuwen Luo, Xiaolong Wang, Weidong Liu, and Yang Liu. Exploring large language models for communication games: An empirical study on werewolf. arXiv preprint arXiv:2309.04658, 2023d. [find](https://scholar.google.com/scholar?q=Yuzhuang+Xu%2C+Shuo+Wang%2C+Peng+Li%2C+Fuwen+Luo%2C+Xiaolong+Wang%2C+Weidong+Liu%2C+and+Yang+Liu%2E+Exploring+large+language+models+for+communication+games%3A+An+empirical+study+on+werewolf%2E+arXiv+preprint+arXiv%3A2309)
156. Aiyuan Yang, Bin Xiao, Bingning Wang, Borong Zhang, Chao Yin, Chenxu Lv, Da Pan, Dian Wang, Dong Yan, Fan Yang, Fei Deng, Feng Wang, Feng Liu, Guangwei Ai, Guosheng Dong, Haizhou Zhao, Hang Xu, Haoze Sun, Hongda Zhang, Hui Liu, Jiaming Ji, Jian Xie, Juntao Dai, Kun Fang, Lei Su, Liang Song, Lifeng L… [find](https://scholar.google.com/scholar?q=Aiyuan+Yang%2C+Bin+Xiao%2C+Bingning+Wang%2C+Borong+Zhang%2C+Chao+Yin%2C+Chenxu+Lv%2C+Da+Pan%2C+Dian+Wang%2C+Dong+Yan%2C+Fan+Yang%2C+Fei+Deng%2C+Feng+Wang%2C+Feng+Liu%2C+Guangwei+Ai%2C+Guosheng+Dong%2C+Haizhou+Zhao%2C+Hang+Xu%2C+Haoze+) [our card](https://md.research.elamur.ai/free/en/papers/yang-2023-baichuan-2-open-large-scale/)
157. Jianxin Yang. Firefly. [https://github.com/yangjianxin1/Firefly](https://github.com/yangjianxin1/Firefly), 2023. [↗](https://github.com/yangjianxin1/Firefly)
158. Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, and Yuan Cao. ReAct: Synergizing reasoning and acting in language models. arXiv preprint arXiv:2210.03629, 2022. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Jeffrey+Zhao%2C+Dian+Yu%2C+Nan+Du%2C+Izhak+Shafran%2C+Karthik+Narasimhan%2C+and+Yuan+Cao%2E+ReAct%3A+Synergizing+reasoning+and+acting+in+language+models%2E+arXiv+preprint+arXiv%3A2210%2E03629%2C+2022%2E) [our card](https://md.research.elamur.ai/free/en/papers/yao-2022-react-synergizing-reasoning-acting-language/)
159. Qinghao Ye, Haiyang Xu, Guohai Xu, Jiabo Ye, Ming Yan, Yiyang Zhou, Junyang Wang, Anwen Hu, Pengcheng Shi, Yaya Shi, et al. mPLUG-Owl: Modularization empowers large language models with multimodality. arXiv preprint arXiv:2304.14178, 2023. [find](https://scholar.google.com/scholar?q=Qinghao+Ye%2C+Haiyang+Xu%2C+Guohai+Xu%2C+Jiabo+Ye%2C+Ming+Yan%2C+Yiyang+Zhou%2C+Junyang+Wang%2C+Anwen+Hu%2C+Pengcheng+Shi%2C+Yaya+Shi%2C+et+al%2E+mPLUG%2DOwl%3A+Modularization+empowers+large+language+models+with+multimodality%2E) [our card](https://md.research.elamur.ai/free/en/papers/ye-2023-mplug-owl-modularization-empowers-large/)
160. Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T. Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu. Metamath: Bootstrap your own mathematical questions for large language models, 2023. [find](https://scholar.google.com/scholar?q=Longhui+Yu%2C+Weisen+Jiang%2C+Han+Shi%2C+Jincheng+Yu%2C+Zhengying+Liu%2C+Yu+Zhang%2C+James+T%2E+Kwok%2C+Zhenguo+Li%2C+Adrian+Weller%2C+and+Weiyang+Liu%2E+Metamath%3A+Bootstrap+your+own+mathematical+questions+for+large+langua) [our card](https://md.research.elamur.ai/free/en/papers/yu-2023-metamath-bootstrap-your-own-mathematical/)
161. Zheng Yuan, Hongyi Yuan, Chengpeng Li, Guanting Dong, Keming Lu, Chuanqi Tan, Chang Zhou, and Jingren Zhou. Scaling relationship on learning mathematical reasoning with large language models, 2023a. [find](https://scholar.google.com/scholar?q=Zheng+Yuan%2C+Hongyi+Yuan%2C+Chengpeng+Li%2C+Guanting+Dong%2C+Keming+Lu%2C+Chuanqi+Tan%2C+Chang+Zhou%2C+and+Jingren+Zhou%2E+Scaling+relationship+on+learning+mathematical+reasoning+with+large+language+models%2C+2023a%2E) [our card](https://md.research.elamur.ai/free/en/papers/yuan-2023-scaling-relationship-learning-mathematical-reasoning/)
162. Zheng Yuan, Hongyi Yuan, Chuanqi Tan, Wei Wang, and Songfang Huang. How well do large language models perform in arithmetic tasks? arXiv preprint arXiv:2304.02015, 2023b. [find](https://scholar.google.com/scholar?q=Zheng+Yuan%2C+Hongyi+Yuan%2C+Chuanqi+Tan%2C+Wei+Wang%2C+and+Songfang+Huang%2E+How+well+do+large+language+models+perform+in+arithmetic+tasks%3F+arXiv+preprint+arXiv%3A2304%2E02015%2C+2023b%2E)
163. Zheng Yuan, Hongyi Yuan, Chuanqi Tan, Wei Wang, Songfang Huang, and Fei Huang. RRHF: Rank responses to align language models with human feedback without tears, 2023c. [find](https://scholar.google.com/scholar?q=Zheng+Yuan%2C+Hongyi+Yuan%2C+Chuanqi+Tan%2C+Wei+Wang%2C+Songfang+Huang%2C+and+Fei+Huang%2E+RRHF%3A+Rank+responses+to+align+language+models+with+human+feedback+without+tears%2C+2023c%2E) [our card](https://md.research.elamur.ai/free/en/papers/yuan-2023-rrhf-rank-responses-align-language/)
164. Xiang Yue, Xingwei Qu, Ge Zhang, Yao Fu, Wenhao Huang, Huan Sun, Yu Su, and Wenhu Chen. MAmmoTH: Building math generalist models through hybrid instruction tuning. arXiv preprint arXiv:2309.05653, 2023. [find](https://scholar.google.com/scholar?q=Xiang+Yue%2C+Xingwei+Qu%2C+Ge+Zhang%2C+Yao+Fu%2C+Wenhao+Huang%2C+Huan+Sun%2C+Yu+Su%2C+and+Wenhu+Chen%2E+MAmmoTH%3A+Building+math+generalist+models+through+hybrid+instruction+tuning%2E+arXiv+preprint+arXiv%3A2309%2E05653%2C+202) [our card](https://md.research.elamur.ai/free/en/papers/yue-2023-mammoth-building-math-generalist-models/)
165. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. HellaSwag: Can a machine really finish your sentence? In Anna Korhonen, David R. Traum, and Lluı́s Màrquez (eds.), Proceedings of the 57th Conference of the Association for Computational Linguistics, ACL 2019, Florence, Italy,… [find](https://scholar.google.com/scholar?q=Rowan+Zellers%2C+Ari+Holtzman%2C+Yonatan+Bisk%2C+Ali+Farhadi%2C+and+Yejin+Choi%2E+HellaSwag%3A+Can+a+machine+really+finish+your+sentence%3F+In+Anna+Korhonen%2C+David+R%2E+Traum%2C+and+Llu%C4%B1%CC%81s+Ma%CC%80rquez+%28eds%2E%29%2C+Proceedings+)
166. Aohan Zeng, Xiao Liu, Zhengxiao Du, Zihan Wang, Hanyu Lai, Ming Ding, Zhuoyi Yang, Yifan Xu, Wendi Zheng, Xiao Xia, et al. GLM-130B: An open bilingual pre-trained model. arXiv preprint arXiv:2210.02414, 2022. [find](https://scholar.google.com/scholar?q=Aohan+Zeng%2C+Xiao+Liu%2C+Zhengxiao+Du%2C+Zihan+Wang%2C+Hanyu+Lai%2C+Ming+Ding%2C+Zhuoyi+Yang%2C+Yifan+Xu%2C+Wendi+Zheng%2C+Xiao+Xia%2C+et+al%2E+GLM%2D130B%3A+An+open+bilingual+pre%2Dtrained+model%2E+arXiv+preprint+arXiv%3A2210%2E0241) [our card](https://md.research.elamur.ai/free/en/papers/zeng-2022-glm-130b-open-bilingual-pre/)
167. Fengji Zhang, Bei Chen, Yue Zhang, Jin Liu, Daoguang Zan, Yi Mao, Jian-Guang Lou, and Weizhu Chen. RepoCoder: Repository-level code completion through iterative retrieval and generation. CoRR, abs/2303.12570, 2023a. doi: 10.48550/arXiv.2303.12570. URL https://doi.org/ 10.48550/arXiv.2303.12570. [find](https://scholar.google.com/scholar?q=Fengji+Zhang%2C+Bei+Chen%2C+Yue+Zhang%2C+Jin+Liu%2C+Daoguang+Zan%2C+Yi+Mao%2C+Jian%2DGuang+Lou%2C+and+Weizhu+Chen%2E+RepoCoder%3A+Repository%2Dlevel+code+completion+through+iterative+retrieval+and+generation%2E+CoRR%2C+abs%2F230)
168. Susan Zhang, Stephen Roller, Naman Goyal, Mikel Artetxe, Moya Chen, Shuohui Chen, Christopher Dewan, Mona Diab, Xian Li, Xi Victoria Lin, et al. OPT: Open pre-trained transformer language models. arXiv preprint arXiv:2205.01068, 2022. [find](https://scholar.google.com/scholar?q=Susan+Zhang%2C+Stephen+Roller%2C+Naman+Goyal%2C+Mikel+Artetxe%2C+Moya+Chen%2C+Shuohui+Chen%2C+Christopher+Dewan%2C+Mona+Diab%2C+Xian+Li%2C+Xi+Victoria+Lin%2C+et+al%2E+OPT%3A+Open+pre%2Dtrained+transformer+language+models%2E+arXi)
169. Xiaotian Zhang, Chunyang Li, Yi Zong, Zhengyu Ying, Liang He, and Xipeng Qiu. Evaluating the performance of large language models on GAOKAO benchmark. CoRR, abs/2305.12474, 2023b. doi: 10.48550/arXiv.2305.12474. URL https://doi.org/10.48550/arXiv. 2305.12474. [↗](https://doi.org/10.48550/arXiv) [our card](https://md.research.elamur.ai/free/en/papers/zhang-2023-evaluating-performance-large-language/)
170. Qinkai Zheng, Xiao Xia, Xu Zou, Yuxiao Dong, Shan Wang, Yufei Xue, Zihan Wang, Lei Shen, Andi Wang, Yang Li, Teng Su, Zhilin Yang, and Jie Tang. CodeGeeX: A pre-trained model for code generation with multilingual evaluations on humaneval-x. CoRR, abs/2303.17568, 2023. doi: 10.48550/arXiv.2303.17568.… [find](https://scholar.google.com/scholar?q=Qinkai+Zheng%2C+Xiao+Xia%2C+Xu+Zou%2C+Yuxiao+Dong%2C+Shan+Wang%2C+Yufei+Xue%2C+Zihan+Wang%2C+Lei+Shen%2C+Andi+Wang%2C+Yang+Li%2C+Teng+Su%2C+Zhilin+Yang%2C+and+Jie+Tang%2E+CodeGeeX%3A+A+pre%2Dtrained+model+for+code+generation+with+) [our card](https://md.research.elamur.ai/free/en/papers/zheng-2023-codegeex-pre-trained-model-code/)
171. Wanjun Zhong, Ruixiang Cui, Yiduo Guo, Yaobo Liang, Shuai Lu, Yanlin Wang, Amin Saied, Weizhu Chen, and Nan Duan. AGIEval: A human-centric benchmark for evaluating foundation models. CoRR, abs/2304.06364, 2023a. doi: 10.48550/arXiv.2304.06364. URL https://doi.org/ 10.48550/arXiv.2304.06364. [find](https://scholar.google.com/scholar?q=Wanjun+Zhong%2C+Ruixiang+Cui%2C+Yiduo+Guo%2C+Yaobo+Liang%2C+Shuai+Lu%2C+Yanlin+Wang%2C+Amin+Saied%2C+Weizhu+Chen%2C+and+Nan+Duan%2E+AGIEval%3A+A+human%2Dcentric+benchmark+for+evaluating+foundation+models%2E+CoRR%2C+abs%2F2304%2E06) [our card](https://md.research.elamur.ai/free/en/papers/zhong-2024-agieval-human-centric-benchmark-evaluating/)
172. Wanjun Zhong, Lianghong Guo, Qiqi Gao, and Yanlin Wang. MemoryBank: Enhancing large language models with long-term memory. arXiv preprint arXiv:2305.10250, 2023b. [find](https://scholar.google.com/scholar?q=Wanjun+Zhong%2C+Lianghong+Guo%2C+Qiqi+Gao%2C+and+Yanlin+Wang%2E+MemoryBank%3A+Enhancing+large+language+models+with+long%2Dterm+memory%2E+arXiv+preprint+arXiv%3A2305%2E10250%2C+2023b%2E)
173. Denny Zhou, Nathanael Scharli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet, Quoc Le, and Ed Huai hsin Chi. Least-to-most prompting enables complex reasoning in large language models. ArXiv, abs/2205.10625, 2022. [find](https://scholar.google.com/scholar?q=Denny+Zhou%2C+Nathanael+Scharli%2C+Le+Hou%2C+Jason+Wei%2C+Nathan+Scales%2C+Xuezhi+Wang%2C+Dale+Schuurmans%2C+Olivier+Bousquet%2C+Quoc+Le%2C+and+Ed+Huai+hsin+Chi%2E+Least%2Dto%2Dmost+prompting+enables+complex+reasoning+in+lar) [our card](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/)