---
title: "LLMs Are Biased Towards Output Formats! Systematically"
description: "The paper examines how strongly the quality of LLM answers depends on the output format requested… 57 references, cited by 8 papers here."
authors: ["Do Xuan Long", "Hai Nguyen Ngoc", "Tiviatis Sim", "Hieu Dao", "Shafiq Joty", "Kenji Kawaguchi", "Nancy F. Chen", "Min-Yen Kan"]
year: 2025
venue: "Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)"
source: "https://aclanthology.org/2025.naacl-long.15/"
doi: "10.18653/v1/2025.naacl-long.15"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/long-2025-llms-are-biased-output-formats/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/long-2025-llms-are-biased-output-formats/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/) / LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMs

# LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMs

**Authors:** [Do Xuan Long](https://md.research.elamur.ai/free/en/authors/do-xuan-long/), Hai Nguyen Ngoc, Tiviatis Sim, Hieu Dao, [Shafiq Joty](https://md.research.elamur.ai/free/en/authors/shafiq-joty/), [Kenji Kawaguchi](https://md.research.elamur.ai/free/en/authors/kenji-kawaguchi/), [Nancy F. Chen](https://md.research.elamur.ai/free/en/authors/nancy-chen/) and [Min-Yen Kan](https://md.research.elamur.ai/free/en/authors/min-yen-kan/) · **Year:** 2025 · **Venue:** [Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/naacl/) ([NAACL 2025](https://md.research.elamur.ai/free/en/venues/naacl/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.naacl-long.15/) · [PDF](https://aclanthology.org/2025.naacl-long.15.pdf) · [DOI 10.18653/v1/2025.naacl-long.15](https://doi.org/10.18653/v1/2025.naacl-long.15) · [OpenAlex](https://openalex.org/W4411119565)

**Citations:** 6 citations (OpenAlex, 2026-10-14) · [8](https://md.research.elamur.ai/free/en/papers/long-2025-llms-are-biased-output-formats/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
The paper examines how strongly the quality of LLM answers depends on the output format requested in the prompt. Earlier benchmarks only checked whether models obey format instructions. Here the authors separate two scores. SysE counts only answers that satisfy the format constraint. TrueE counts every correct answer regardless of formatting. Because TrueE would need an oracle extractor or human checking, they estimate it automatically: EstTrueE divides SysE by the format-instruction-following (FI) rate, and a theorem gives the minimum FI needed for that estimate to be reliable. Format bias (BiasFo) is the variance of performance across a set of formats. The evaluation covers 15 formats in four families (MCQ, wrapping, list, mapping), eight QA and reasoning tasks, and four models, using zero-shot and zero-shot chain-of-thought prompting. All models showed marked bias. The authors found that improving format compliance tends to reduce bias, and they test three remedies. This matters for any pipeline that parses LLM outputs automatically.

## Key points
- Metrics introduced: FI score, SysE, TrueE, the automatic estimator EstTrueE (consistent, with a reliability condition on FI), and BiasFo for cross-format variance.
- Scale of the evaluation: 15 formats, 4 format categories, 8 tasks, 4 LLMs (including ChatGPT), with MMLU among the benchmarks.
- Adding demonstrations raised ChatGPT's average FI on wrapping formats from 72.99% to 79.19% with one example and 85.31% with five.
- Stating the format instruction three times also lowered BiasFo. The exception was the "Placeholder" format, where the repeated instructions confused the model.
- The authors fine-tuned ChatGPT on 6,500 synthesized MMLU auxiliary samples, weighting weaker formats more heavily. Average FI rose to 99.58%, and wrapping-format variance fell from 235.33 to 0.71 (%²).

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [National University of Singapore](https://md.research.elamur.ai/free/en/universities/national-university-of-singapore/) — University, Singapore, Singapore · 180 works · 34,304 citations
- [Salesforce](https://md.research.elamur.ai/free/en/companies/salesforce/) — Company, San Francisco, United States · 56 works · 17,867 citations
- [Nanyang Technological University](https://md.research.elamur.ai/free/en/universities/nanyang-technological-university/) — University, Singapore, Singapore · 156 works · 25,644 citations
- [Institute for Infocomm Research](https://md.research.elamur.ai/free/en/labs/institute-for-infocomm-research/) — Research institute, Singapore, Singapore · 43 works · 14,598 citations
- [Institute of High Performance Computing](https://md.research.elamur.ai/free/en/labs/institute-of-high-performance-computing/) — Research institute, Singapore, Singapore · 5 works · 169 citations

📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,772 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 543 works

🧭 Topics (2): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,320 works · [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,220 works

## Cite

Cite as: Do Xuan Long, Hai Nguyen Ngoc, Tiviatis Sim, Hieu Dao, Shafiq Joty, Kenji Kawaguchi, Nancy F. Chen and Min-Yen Kan. 2025. LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMs. Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{long2025llms,
  title = {{LLMs Are Biased Towards Output Formats! Systematically Evaluating and Mitigating Output Format Bias of LLMs}},
  author = {Do Xuan Long, Hai Nguyen Ngoc, Tiviatis Sim, Hieu Dao, Shafiq Joty, Kenji Kawaguchi, Nancy F. Chen and Min-Yen Kan},
  booktitle = {Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)},
  year = {2025},
  doi = {10.18653/v1/2025.naacl-long.15},
  url = {https://doi.org/10.18653/v1/2025.naacl-long.15}
}
```

- [← previous Large Language Models for Automated Literature Review: An Evaluation of Reference Generation, Abstract Writing, and Review Composition](https://md.research.elamur.ai/free/en/papers/tang-2025-large-language-models-automated-literature/)
- [next → LLMs Know What They Need: Leveraging a Missing Information Guided Framework to Empower Retrieval-Augmented Generation](https://md.research.elamur.ai/free/en/papers/wang-2025-llms-know-what-they-need/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/6/)

## References

57 references, 29 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, et al. 2023. [Gpt-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). arXiv preprint arXiv:2303.08774.
2. Takeshi Amemiya. 1985. Advanced econometrics. Harvard university press. [find](https://scholar.google.com/scholar?q=Takeshi+Amemiya%2E+1985%2E+Advanced+econometrics%2E+Harvard+university+press%2E)
3. Isabelle Augenstein, Mrinal Das, Sebastian Riedel, Lakshmi Vikraman, and Andrew McCallum. 2017. [find](https://scholar.google.com/scholar?q=Isabelle+Augenstein%2C+Mrinal+Das%2C+Sebastian+Riedel%2C+Lakshmi+Vikraman%2C+and+Andrew+McCallum%2E+2017%2E)
4. [SemEval 2017 task 10: ScienceIE - extracting keyphrases and relations from scientific publications](https://md.research.elamur.ai/free/en/papers/augenstein-2017-semeval-2017-task-10-scienceie/). In Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pages 546– 555, Vancouver, Canada. Association for Computational Linguistics.
5. Yushi Bai, Xin Lv, Jiajie Zhang, Hongchang Lyu, Jiankai Tang, Zhidian Huang, Zhengxiao Du, Xiao Liu, Aohan Zeng, Lei Hou, Yuxiao Dong, Jie Tang, and Juanzi Li. 2024. LongBench: A bilingual, multi-task benchmark for long context understanding. In Proceedings of the 62nd Annual Meeting of the Associat… [find](https://scholar.google.com/scholar?q=Yushi+Bai%2C+Xin+Lv%2C+Jiajie+Zhang%2C+Hongchang+Lyu%2C+Jiankai+Tang%2C+Zhidian+Huang%2C+Zhengxiao+Du%2C+Xiao+Liu%2C+Aohan+Zeng%2C+Lei+Hou%2C+Yuxiao+Dong%2C+Jie+Tang%2C+and+Juanzi+Li%2E+2024%2E+LongBench%3A+A+bilingual%2C+multi%2Dtask)
6. Hutchinson, Reiner Pope, James Bradbury, Jacob Austin, Michael Isard, Guy Gur-Ari, Pengcheng Yin, Toju Duke, Anselm Levskaya, Sanjay Ghemawat, Sunipa Dev, Henryk Michalewski, Xavier García, Vedant Misra, Kevin Robinson, Liam Fedus, Denny Zhou, Daphne Ippolito, David Luan, Hyeontaek Lim, Barret Zoph,… — [PaLM: Scaling Language Modeling with Pathways](https://md.research.elamur.ai/free/en/papers/chowdhery-2022-palm-scaling-language-modeling-pathways/)
7. Nick Baumann, Alexander Brinkmann, and Christian Bizer. 2024. Using llms for the extraction and normalization of product attribute values. arXiv preprint arXiv:2403.02130. — [arXiv:2403.02130](https://arxiv.org/abs/2403.02130) [find](https://scholar.google.com/scholar?q=Nick+Baumann%2C+Alexander+Brinkmann%2C+and+Christian+Bizer%2E+2024%2E+Using+llms+for+the+extraction+and+normalization+of+product+attribute+values%2E+arXiv+preprint+arXiv%3A2403%2E02130%2E+%E2%80%94+%5BarXiv%3A2403%2E02130%5D%28https%3A%2F)
8. Ayham Boucher. 2023. Llm based context splitter for large documents. [find](https://scholar.google.com/scholar?q=Ayham+Boucher%2E+2023%2E+Llm+based+context+splitter+for+large+documents%2E)
9. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, Chri… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
10. Yupeng Chang, Xu Wang, Jindong Wang, Yuan Wu, Linyi Yang, Kaijie Zhu, Hao Chen, Xiaoyuan Yi, Cunxiang Wang, Yidong Wang, et al. 2024. [A survey on evaluation of large language models](https://md.research.elamur.ai/free/en/papers/chang-2023-survey-evaluation-large-language-models/). ACM Transactions on Intelligent Systems and Technology, 15(3):1–45.
11. Xinyun Chen, Renat Aksitov, Uri Alon, Jie Ren, Kefan Xiao, Pengcheng Yin, Sushant Prakash, Charles Sutton, Xuezhi Wang, and Denny Zhou. 2023. [Universal self-consistency for large language model generation](https://md.research.elamur.ai/free/en/papers/chen-2023-universal-self-consistency-large-language/). arXiv preprint arXiv:2311.17311.
12. Yihan Chen, Benfeng Xu, Quan Wang, Yi Liu, and Zhendong Mao. 2024. Benchmarking large language models on controllable generation under diversified instructions. arXiv preprint arXiv:2401.00690. — [arXiv:2401.00690](https://arxiv.org/abs/2401.00690) [find](https://scholar.google.com/scholar?q=Yihan+Chen%2C+Benfeng+Xu%2C+Quan+Wang%2C+Yi+Liu%2C+and+Zhendong+Mao%2E+2024%2E+Benchmarking+large+language+models+on+controllable+generation+under+diversified+instructions%2E+arXiv+preprint+arXiv%3A2401%2E00690%2E+%E2%80%94+%5BarX)
13. Liying Cheng, Xingxuan Li, and Lidong Bing. 2023. Is GPT-4 a good data analyst? In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 9496–9514, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Liying+Cheng%2C+Xingxuan+Li%2C+and+Lidong+Bing%2E+2023%2E+Is+GPT%2D4+a+good+data+analyst%3F+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2023%2C+pages+9496%E2%80%939514%2C+Singapore%2E+Association+for+Co)
14. Cheng-Han Chiang and Hung-yi Lee. 2023. Can large language models be an alternative to human evaluations? In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 15607–15631, Toronto, Canada. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Cheng%2DHan+Chiang+and+Hung%2Dyi+Lee%2E+2023%2E+Can+large+language+models+be+an+alternative+to+human+evaluations%3F+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+for+Computational+Linguistics+%28Vo)
15. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, Parker Schuh, Kensen Shi, Sasha Tsvyashchenko, Joshua Maynez, Abhishek Rao, Parker Barnes, Yi Tay, Noam M. Shazeer, Vinodkumar Prabhakaran,… [find](https://scholar.google.com/scholar?q=Aakanksha+Chowdhery%2C+Sharan+Narang%2C+Jacob+Devlin%2C+Maarten+Bosma%2C+Gaurav+Mishra%2C+Adam+Roberts%2C+Paul+Barham%2C+Hyung+Won+Chung%2C+Charles+Sutton%2C+Sebastian+Gehrmann%2C+Parker+Schuh%2C+Kensen+Shi%2C+Sasha+Tsvyashc)
16. Jan Clusmann, Fiona R Kolbinger, Hannah Sophie Muti, Zunamys I Carrero, Jan-Niklas Eckardt, Narmin Ghaffari Laleh, Chiara Maria Lavinia Löffler, Sophie-Caroline Schwarzkopf, Michaela Unger, Gregory P Veldhuizen, et al. 2023. The future landscape of large language models in medicine. Communications m… [find](https://scholar.google.com/scholar?q=Jan+Clusmann%2C+Fiona+R+Kolbinger%2C+Hannah+Sophie+Muti%2C+Zunamys+I+Carrero%2C+Jan%2DNiklas+Eckardt%2C+Narmin+Ghaffari+Laleh%2C+Chiara+Maria+Lavinia+L%C3%B6ffler%2C+Sophie%2DCaroline+Schwarzkopf%2C+Michaela+Unger%2C+Gregory+P+)
17. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al. 2021. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/). arXiv preprint arXiv:2110.14168.
18. Arman Cohan, Sergey Feldman, Iz Beltagy, Doug Downey, and Daniel Weld. 2020. [SPECTER: Document-level representation learning using citation-informed transformers](https://md.research.elamur.ai/free/en/papers/cohan-2020-specter-document-level-representation-learning/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 2270–2282, Online. Association for Comp…
19. Xuan Long Do, Kenji Kawaguchi, Min-Yen Kan, and Nancy Chen. 2025. [Aligning large language models with human opinions through persona selection and value–belief–norm reasoning](https://md.research.elamur.ai/free/en/papers/long-2023-aligning-large-language-models-human/). In Proceedings of the 31st International Conference on Computational Linguistics, pages 2526–2547, Abu Dhabi, UAE. Associati…
20. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony S. Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, Aurelien Rodrigu… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
21. Abbas Ghaddar, Phillippe Langlais, Mehdi Rezagholizadeh, and Ahmad Rashid. 2021. End-to-end self-debiasing framework for robust NLU training. In Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021, pages 1923–1929, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Abbas+Ghaddar%2C+Phillippe+Langlais%2C+Mehdi+Rezagholizadeh%2C+and+Ahmad+Rashid%2E+2021%2E+End%2Dto%2Dend+self%2Ddebiasing+framework+for+robust+NLU+training%2E+In+Findings+of+the+Association+for+Computational+Linguisti)
22. Akshay Goel, Almog Gueta, Omry Gilon, Chang Liu, Sofia Erell, Lan Huong Nguyen, Xiaohong Hao, Bolous Jaber, Shashir Reddy, Rupesh Kartha, Jean Steiner, Itay Laish, and Amir Feder. 2023. Llms accelerate annotation for medical information extraction. In Proceedings of the 3rd Machine Learning for Heal… [find](https://scholar.google.com/scholar?q=Akshay+Goel%2C+Almog+Gueta%2C+Omry+Gilon%2C+Chang+Liu%2C+Sofia+Erell%2C+Lan+Huong+Nguyen%2C+Xiaohong+Hao%2C+Bolous+Jaber%2C+Shashir+Reddy%2C+Rupesh+Kartha%2C+Jean+Steiner%2C+Itay+Laish%2C+and+Amir+Feder%2E+2023%2E+Llms+accelerat)
23. Zishan Guo, Renren Jin, Chuang Liu, Yufei Huang, Dan Shi, Linhao Yu, Yan Liu, Jiaxuan Li, Bojian Xiong, Deyi Xiong, et al. 2023. [Evaluating large language models: A comprehensive survey](https://md.research.elamur.ai/free/en/papers/guo-2023-evaluating-large-language-models-comprehensive/). arXiv preprint arXiv:2310.19736.
24. Shashank Gupta, Vaishnavi Shrivastava, Ameet Deshpande, Ashwin Kalyan, Peter Clark, Ashish Sabharwal, and Tushar Khot. 2024. Bias runs deep: Implicit reasoning biases in persona-assigned LLMs. In The Twelfth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Shashank+Gupta%2C+Vaishnavi+Shrivastava%2C+Ameet+Deshpande%2C+Ashwin+Kalyan%2C+Peter+Clark%2C+Ashish+Sabharwal%2C+and+Tushar+Khot%2E+2024%2E+Bias+runs+deep%3A+Implicit+reasoning+biases+in+persona%2Dassigned+LLMs%2E+In+The+)
25. Izzeddin Gur, Ofir Nachum, Yingjie Miao, Mustafa Safdari, Austin Huang, Aakanksha Chowdhery, Sharan Narang, Noah Fiedel, and Aleksandra Faust. 2023. Understanding HTML with large language models. In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 2803–2821, Singapore. As… [find](https://scholar.google.com/scholar?q=Izzeddin+Gur%2C+Ofir+Nachum%2C+Yingjie+Miao%2C+Mustafa+Safdari%2C+Austin+Huang%2C+Aakanksha+Chowdhery%2C+Sharan+Narang%2C+Noah+Fiedel%2C+and+Aleksandra+Faust%2E+2023%2E+Understanding+HTML+with+large+language+models%2E+In+F)
26. Dan Hendrycks, Collin Burns, Steven Basart, Andy Zou, Mantas Mazeika, Dawn Song, and Jacob Steinhardt. 2021. [Measuring massive multitask language understanding](https://md.research.elamur.ai/free/en/papers/hendrycks-2020-measuring-massive-multitask-language-understanding/). In International Conference on Learning Representations.
27. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. [The curious case of neural text degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/). In International Conference on Learning Representations.
28. Sarthak Jain, Madeleine van Zuylen, Hannaneh Hajishirzi, and Iz Beltagy. 2020. [SciREX: A challenge dataset for document-level information extraction](https://md.research.elamur.ai/free/en/papers/jain-2020-scirex-challenge-dataset-document-level/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7506– 7516, Online. Association for Computational Li…
29. Albert Qiaochu Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de Las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, L’elio Renard Lavaud, Marie-Anne Lachaux, Pierre Stock, Teven Le Scao, Thibaut Lavril, Thomas Wang, Timothée La… [find](https://scholar.google.com/scholar?q=Albert+Qiaochu+Jiang%2C+Alexandre+Sablayrolles%2C+Arthur+Mensch%2C+Chris+Bamford%2C+Devendra+Singh+Chaplot%2C+Diego+de+Las+Casas%2C+Florian+Bressand%2C+Gianna+Lengyel%2C+Guillaume+Lample%2C+Lucile+Saulnier%2C+L%E2%80%99elio+Rena)
30. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. [Large language models are zero-shot reasoners](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/). In Advances in Neural Information Processing Systems.
31. Ang Li, Jingqian Zhao, Bin Liang, Lin Gui, Hui Wang, Xi Zeng, Kam-Fai Wong, and Ruifeng Xu. 2024. Mitigating biases of large language models in stance detection with calibration. arXiv preprint arXiv:2402.14296. [find](https://scholar.google.com/scholar?q=Ang+Li%2C+Jingqian+Zhao%2C+Bin+Liang%2C+Lin+Gui%2C+Hui+Wang%2C+Xi+Zeng%2C+Kam%2DFai+Wong%2C+and+Ruifeng+Xu%2E+2024%2E+Mitigating+biases+of+large+language+models+in+stance+detection+with+calibration%2E+arXiv+preprint+arXiv%3A)
32. Stephanie Lin, Jacob Hilton, and Owain Evans. 2022. [TruthfulQA: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 3214–3252, Dublin, Ireland. Association for Computational Linguistics…
33. Michael Xieyang Liu, Frederick Liu, Alexander J Fiannaca, Terry Koo, Lucas Dixon, Michael Terry, and Carrie J Cai. 2024. "we need structured output": Towards user-centered constraints on large language model output. In Extended Abstracts of the CHI Conference on Human Factors in Computing Systems, p… [find](https://scholar.google.com/scholar?q=Michael+Xieyang+Liu%2C+Frederick+Liu%2C+Alexander+J+Fiannaca%2C+Terry+Koo%2C+Lucas+Dixon%2C+Michael+Terry%2C+and+Carrie+J+Cai%2E+2024%2E+%22we+need+structured+output%22%3A+Towards+user%2Dcentered+constraints+on+large+languag)
34. Shang-Ching Liu, ShengKun Wang, Tsungyao Chang, Wenqi Lin, Chung-Wei Hsiung, Yi-Chen Hsieh, Yu-Ping Cheng, Sian-Hong Luo, and Jianwei Zhang. 2023. JarviX: A LLM no code platform for tabular data analysis and optimization. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language… [find](https://scholar.google.com/scholar?q=Shang%2DChing+Liu%2C+ShengKun+Wang%2C+Tsungyao+Chang%2C+Wenqi+Lin%2C+Chung%2DWei+Hsiung%2C+Yi%2DChen+Hsieh%2C+Yu%2DPing+Cheng%2C+Sian%2DHong+Luo%2C+and+Jianwei+Zhang%2E+2023%2E+JarviX%3A+A+LLM+no+code+platform+for+tabular+data+analy)
35. Xiaoliang Luo, Akilles Rechardt, Guangzhi Sun, Kevin K. Nejad, Felipe Yáñez, Bati Yilmaz, Kangjoo Lee, Alexandra O. Cohen, Valentina Borghesani, Anton Pashkov, Daniele Marinazzo, Jonathan Nicholas, Alessandro Salatiello, Ilia Sucholutsky, Pasquale Minervini, Sepehr Razavi, Roberta Rocca, Elkhan Yusi… [find](https://scholar.google.com/scholar?q=Xiaoliang+Luo%2C+Akilles+Rechardt%2C+Guangzhi+Sun%2C+Kevin+K%2E+Nejad%2C+Felipe+Y%C3%A1%C3%B1ez%2C+Bati+Yilmaz%2C+Kangjoo+Lee%2C+Alexandra+O%2E+Cohen%2C+Valentina+Borghesani%2C+Anton+Pashkov%2C+Daniele+Marinazzo%2C+Jonathan+Nicholas%2C+Al)
36. Marcos Macedo, Yuan Tian, Filipe R Cogo, and Bram Adams. 2024. Exploring the impact of the output format on the evaluation of large language models for code translation. arXiv preprint arXiv:2403.17214. — [arXiv:2403.17214](https://arxiv.org/abs/2403.17214) [find](https://scholar.google.com/scholar?q=Marcos+Macedo%2C+Yuan+Tian%2C+Filipe+R+Cogo%2C+and+Bram+Adams%2E+2024%2E+Exploring+the+impact+of+the+output+format+on+the+evaluation+of+large+language+models+for+code+translation%2E+arXiv+preprint+arXiv%3A2403%2E1721)
37. Onkar Mishra. 2023. Using langchain for question answering on own data. [find](https://scholar.google.com/scholar?q=Onkar+Mishra%2E+2023%2E+Using+langchain+for+question+answering+on+own+data%2E)
38. Niklas Muennighoff, Nouamane Tazi, Loic Magne, and Nils Reimers. 2023. [MTEB: Massive text embedding benchmark](https://md.research.elamur.ai/free/en/papers/muennighoff-2022-mteb-massive-text-embedding-benchmark/). In Proceedings of the 17th Conference of the European Chapter of the Association for Computational Linguistics, pages 2014–2037, Dubrovnik, Croatia. Association for Computational Linguistic…
39. OpenAI. 2022. Introducing chatgpt. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2022%2E+Introducing+chatgpt%2E)
40. Zhi Rui Tam, Cheng-Kuang Wu, Yi-Lin Tsai, Chieh-Yen Lin, Hung-yi Lee, and Yun-Nung Chen. 2024. Let me speak freely? a study on the impact of format restrictions on large language model performance. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing: Industry Tr… [find](https://scholar.google.com/scholar?q=Zhi+Rui+Tam%2C+Cheng%2DKuang+Wu%2C+Yi%2DLin+Tsai%2C+Chieh%2DYen+Lin%2C+Hung%2Dyi+Lee%2C+and+Yun%2DNung+Chen%2E+2024%2E+Let+me+speak+freely%3F+a+study+on+the+impact+of+format+restrictions+on+large+language+model+performance%2E+In)
41. Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al. 2024. Gemma: Open models based on gemini research and technology. arXiv preprint arXiv:2403.08295. [find](https://scholar.google.com/scholar?q=Gemma+Team%2C+Thomas+Mesnard%2C+Cassidy+Hardin%2C+Robert+Dadashi%2C+Surya+Bhupatiraju%2C+Shreya+Pathak%2C+Laurent+Sifre%2C+Morgane+Rivi%C3%A8re%2C+Mihir+Sanjay+Kale%2C+Juliette+Love%2C+et+al%2E+2024%2E+Gemma%3A+Open+models+based+on)
42. Arun James Thirunavukarasu, Darren Shu Jeng Ting, Kabilan Elangovan, Laura Gutierrez, Ting Fang Tan, and Daniel Shu Wei Ting. 2023. Large language models in medicine. Nature medicine, 29(8):1930– 1940. [find](https://scholar.google.com/scholar?q=Arun+James+Thirunavukarasu%2C+Darren+Shu+Jeng+Ting%2C+Kabilan+Elangovan%2C+Laura+Gutierrez%2C+Ting+Fang+Tan%2C+and+Daniel+Shu+Wei+Ting%2E+2023%2E+Large+language+models+in+medicine%2E+Nature+medicine%2C+29%288%29%3A1930%E2%80%93+1940)
43. Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample. 2023. [Llama: Open and efficient foundation language models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-open-efficient-foundation-language/). ArXiv, a…
44. Ke Wang, Houxing Ren, Aojun Zhou, Zimu Lu, Sichun Luo, Weikang Shi, Renrui Zhang, Linqi Song, Mingjie Zhan, and Hongsheng Li. 2024. [Mathcoder: Seamless code integration in LLMs for enhanced mathematical reasoning](https://md.research.elamur.ai/free/en/papers/wang-2023-mathcoder-seamless-code-integration-llms/). In The Twelfth International Conference on Learning Representations.
45. Joshua Robinson and David Wingate. 2023. [Leveraging large language models for multiple choice question answering](https://md.research.elamur.ai/free/en/papers/robinson-2022-leveraging-large-language-models-multiple/). In The Eleventh International Conference on Learning Representations.
46. Rebecca Roelofs, Nicholas Cain, Jonathon Shlens, and Michael C Mozer. 2022. Mitigating bias in calibration error estimation. In International Conference on Artificial Intelligence and Statistics, pages 4036– 4054. PMLR. [find](https://scholar.google.com/scholar?q=Rebecca+Roelofs%2C+Nicholas+Cain%2C+Jonathon+Shlens%2C+and+Michael+C+Mozer%2E+2022%2E+Mitigating+bias+in+calibration+error+estimation%2E+In+International+Conference+on+Artificial+Intelligence+and+Statistics%2C+page)
47. Timo Schick, Sahana Udupa, and Hinrich Schütze. 2021. [Self-diagnosis and self-debiasing: A proposal for reducing corpus-based bias in NLP](https://md.research.elamur.ai/free/en/papers/schick-2021-self-diagnosis-self-debiasing-proposal/). Transactions of the Association for Computational Linguistics, 9:1408– 1424.
48. Ishika Singh, Valts Blukis, Arsalan Mousavian, Ankit Goyal, Danfei Xu, Jonathan Tremblay, Dieter Fox, Jesse Thomason, and Animesh Garg. 2023. [Progprompt: Generating situated robot task plans using large language models](https://md.research.elamur.ai/free/en/papers/singh-2022-progprompt-generating-situated-robot-task/). In 2023 IEEE International Conference on Robotics and Automation (ICRA), pages 1…
49. Mirac Suzgun, Nathan Scales, Nathanael Schärli, Sebastian Gehrmann, Yi Tay, Hyung Won Chung, Aakanksha Chowdhery, Quoc Le, Ed Chi, Denny Zhou, and Jason Wei. 2023. [Challenging BIG-bench tasks and whether chain-of-thought can solve them](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/). In Findings of the Association for Computational Linguistics: A…
50. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, brian ichter, Fei Xia, Ed Chi, Quoc V Le, and Denny Zhou. 2022. [Chain-of-thought prompting elicits reasoning in large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/). In Advances in Neural Information Processing Systems, volume 35, pages 24824–24837. Curran Associates, Inc.
51. Congying Xia, Chen Xing, Jiangshu Du, Xinyi Yang, Yihao Feng, Ran Xu, Wenpeng Yin, and Caiming Xiong. 2024. FOFO: A benchmark to evaluate LLMs’ format-following capability. In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 680–6… [find](https://scholar.google.com/scholar?q=Congying+Xia%2C+Chen+Xing%2C+Jiangshu+Du%2C+Xinyi+Yang%2C+Yihao+Feng%2C+Ran+Xu%2C+Wenpeng+Yin%2C+and+Caiming+Xiong%2E+2024%2E+FOFO%3A+A+benchmark+to+evaluate+LLMs%E2%80%99+format%2Dfollowing+capability%2E+In+Proceedings+of+the+62nd+)
52. Sang Michael Xie, Aditi Raghunathan, Percy Liang, and Tengyu Ma. 2022. [An explanation of in-context learning as implicit bayesian inference](https://md.research.elamur.ai/free/en/papers/xie-2021-explanation-context-learning-implicit-bayesian/). In International Conference on Learning Representations.
53. Ying Xu, Dakuo Wang, Mo Yu, Daniel Ritchie, Bingsheng Yao, Tongshuang Wu, Zheng Zhang, Toby Li, Nora Bradford, Branda Sun, Tran Hoang, Yisi Sang, Yufang Hou, Xiaojuan Ma, Diyi Yang, Nanyun Peng, Zhou Yu, and Mark Warschauer. 2022. [Fantastic questions and where to find them: FairytaleQA – an authenti…](https://md.research.elamur.ai/free/en/papers/xu-2022-fantastic-questions-where-find-them/)
54. Ziyang Xu, Keqin Peng, Liang Ding, Dacheng Tao, and Xiliang Lu. 2024. Take care of your prompt bias! investigating and mitigating prompt bias in factual knowledge extraction. arXiv preprint arXiv:2403.09963. [find](https://scholar.google.com/scholar?q=Ziyang+Xu%2C+Keqin+Peng%2C+Liang+Ding%2C+Dacheng+Tao%2C+and+Xiliang+Lu%2E+2024%2E+Take+care+of+your+prompt+bias%21+investigating+and+mitigating+prompt+bias+in+factual+knowledge+extraction%2E+arXiv+preprint+arXiv%3A2403)
55. Chengrun Yang, Xuezhi Wang, Yifeng Lu, Hanxiao Liu, Quoc V Le, Denny Zhou, and Xinyun Chen. 2024. [Large language models as optimizers](https://md.research.elamur.ai/free/en/papers/yang-2023-large-language-models-optimizers/). In The Twelfth International Conference on Learning Representations.
56. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. [HotpotQA: A dataset for diverse, explainable multi-hop question answering](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pages 2369…
57. Jeffrey Zhou, Tianjian Lu, Swaroop Mishra, Siddhartha Brahma, Sujoy Basu, Yi Luan, Denny Zhou, and Le Hou. 2023. [Instruction-following evaluation for large language models](https://md.research.elamur.ai/free/en/papers/zhou-2023-instruction-following-evaluation-large-language/). arXiv preprint arXiv:2311.07911.

*Updated 31 October 2026*