---
title: "Large Language Models are few(1)-shot Table — Chen 2023"
description: "This paper asks whether general-purpose large language models, which were never trained on table data, can handle table reasoning through in-context…"
authors: ["Wenhu Chen"]
year: 2023
venue: "Findings of the Association for Computational Linguistics: EACL 2023"
source: "https://aclanthology.org/2023.findings-eacl.83/"
doi: "10.18653/v1/2023.findings-eacl.83"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/chen-2023-large-language-models-are-few/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/chen-2023-large-language-models-are-few/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/4/) / Large Language Models are few(1)-shot Table Reasoners

# Large Language Models are few(1)-shot Table Reasoners

**Authors:** [Wenhu Chen](https://md.research.elamur.ai/free/en/authors/wenhu-chen/) · **Year:** 2023 · **Venue:** Findings of the Association for Computational Linguistics: EACL 2023 · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.findings-eacl.83/) · [PDF](https://aclanthology.org/2023.findings-eacl.83.pdf) · [DOI 10.18653/v1/2023.findings-eacl.83](https://doi.org/10.18653/v1/2023.findings-eacl.83) · [OpenAlex](https://openalex.org/W4386566488)

**Citations:** 97 citations (openalex-snapshot · 2026-10-14) · [85](#cited-by) in our library, of the 140884 papers whose bibliographies we read

## Summary
This paper asks whether general-purpose large language models, which were never trained on table data, can handle table reasoning through in-context learning instead of task-specific fine-tuning. The author flattens each table into text, adds one or a few worked demonstrations, and compares three prompting styles: direct answering, chain-of-thought (CoT), and CoT combined with self-consistency voting. Experiments use GPT-3 (mainly Davinci-text-002) and, on some datasets, Codex. The benchmarks are WikiTableQuestions, FetaQA, TabFact and FEVEROUS. With only one or two demonstrations, CoT prompting reaches scores close to some fine-tuned near-SoTA systems. On FetaQA, which requires long free-form answers, human judges rated GPT-3 well above a fine-tuned T5-large. A manual review found that most reasoning chains behind correct answers actually rely on table content. The main weakness is table size: accuracy falls steadily as tables grow longer. The study positions LLM prompting as a simple, general baseline for table tasks that complements symbolic methods.

## Key points
- **Data and metrics:**
  - WikiTableQuestions: about 4,000 test questions, scored by exact match.
  - FetaQA: 2,003 examples, scored with sacreBLEU and human evaluation.
  - TabFact: 12,779 test claims, scored by binary accuracy.
  - FEVEROUS: filtered to 2,295 table-supported claims, dev-set accuracy.
- **Input truncation:** tables are cut to the first 22 rows and 8 columns, and each cell to its first 10 words. This keeps prompts under roughly 2,000 tokens.
- **Self-consistency setup:** five reasoning paths are sampled at temperature 0.7, and the final answer is chosen by majority vote.
- **Headline results:**
  - 48.8% on WikiTableQuestions and 78.8% on TabFact with 1–2 demonstrations.
  - On FetaQA, human ratings of correctness and adequacy exceed fine-tuned T5-large by more than 30%.
- **Analysis findings:**
  - Going from 1 to 2 shots often helps, but more shots add little.
  - In a sample of 100 correct predictions, about 90% of reasoning chains were grounded in the table.
  - Once tables exceed about 1,000 tokens, accuracy drops to near chance.
  - Common errors are grounding to the wrong row or column and slips in simple arithmetic or comparisons.

## Interesting ideas
- **Budget-driven table truncation:** capping rows, columns and words per cell is a cheap way to fit semi-structured inputs into a fixed context window. It could be reused for any tabular data, such as trade statistics or geographic registries.
- **Breaking accuracy down by input length:** grouping test items into token-length bins and plotting accuracy per bin exposes where a prompting method stops working. This diagnostic transfers easily to any long-input task.
- **Checking whether correct answers are correct for the right reason:** manually labelling sampled reasoning chains as grounded or hallucinated tests whether a model's reasons are faithful, not just its final answers. This matters for any setting where explanations will be shown to users.
- **LLM rationales as cheap annotations:** reasoning chains generated by the model could serve as draft natural-language versions of semantic forms. This could cut the cost of expert annotation for semantic parsing or other structured labelling work.
- **Hybrid symbolic-plus-LLM pipeline:** a symbolic step first narrows a large table down to the relevant region, and the LLM then reasons only over that slice. This combines the size-independence of symbolic tools with the flexibility of language models.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/)

## Cited by

Cited by

85 papers of our library cite it (of the 140884 whose bibliographies we read); 5 of them have a page here, 80 not yet.

- [AraTable: Benchmarking LLMs' Reasoning and Understanding of Arabic Tabular Data](https://md.research.elamur.ai/free/en/papers/alshaikh-2025-aratable-benchmarking-llms-reasoning-understanding/)
- [Towards Reasoning in Large Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/huang-2023-reasoning-large-language-models-survey/)
- [DIN-SQL: Decomposed In-Context Learning of Text-to-SQL with Self-Correction](https://md.research.elamur.ai/free/en/papers/pourreza-2023-din-sql-decomposed-context-learning/)
- [Reasoning with Language Model Prompting: A Survey](https://md.research.elamur.ai/free/en/papers/qiao-2022-reasoning-language-model-prompting-survey/)
- [Challenging BIG-Bench Tasks and Whether Chain-of-Thought Can Solve Them](https://md.research.elamur.ai/free/en/papers/suzgun-2023-challenging-big-bench-tasks-whether/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{chen2023large,
  title = {{Large Language Models are few(1)-shot Table Reasoners}},
  author = {Chen, Wenhu},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2023},
  year = {2023},
  doi = {10.18653/v1/2023.findings-eacl.83},
  url = {https://doi.org/10.18653/v1/2023.findings-eacl.83}
}
```

- [← previous Large Language Models are Better Reasoners with Self-Verification](https://md.research.elamur.ai/free/en/papers/weng-2023-large-language-models-are-better/)
- [next → Large Language Models Are Reasoning Teachers](https://md.research.elamur.ai/free/en/papers/ho-2023-large-language-models-are-reasoning/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/4/)

## References

46 references, 24 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Rishabh Agarwal, Chen Liang, Dale Schuurmans, and Mohammad Norouzi. 2019. Learning to generalize from sparse and underspecified rewards. In International conference on machine learning, pages 130– 140. PMLR. [find](https://scholar.google.com/scholar?q=Rishabh+Agarwal%2C+Chen+Liang%2C+Dale+Schuurmans%2C+and+Mohammad+Norouzi%2E+2019%2E+Learning+to+generalize+from+sparse+and+underspecified+rewards%2E+In+International+conference+on+machine+learning%2C+pages+130%E2%80%93+140)
2. Rami Aly, Zhijiang Guo, Michael Sejr Schlichtkrull, James Thorne, Andreas Vlachos, Christos Christodoulopoulos, Oana Cocarascu, and Arpit Mittal. 2021. The fact extraction and verification over unstructured and structured information (feverous) shared task. In Proceedings of the Fourth Workshop on F… [find](https://scholar.google.com/scholar?q=Rami+Aly%2C+Zhijiang+Guo%2C+Michael+Sejr+Schlichtkrull%2C+James+Thorne%2C+Andreas+Vlachos%2C+Christos+Christodoulopoulos%2C+Oana+Cocarascu%2C+and+Arpit+Mittal%2E+2021%2E+The+fact+extraction+and+verification+over+unstru) [our card](https://md.research.elamur.ai/free/en/papers/aly-2021-fact-extraction-verification-over-unstructured/)
3. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al. 2020. Language models are few-shot learners. Advances in neural information processing systems, 33:1877–1901. [find](https://scholar.google.com/scholar?q=Tom+Brown%2C+Benjamin+Mann%2C+Nick+Ryder%2C+Melanie+Subbiah%2C+Jared+D+Kaplan%2C+Prafulla+Dhariwal%2C+Arvind+Neelakantan%2C+Pranav+Shyam%2C+Girish+Sastry%2C+Amanda+Askell%2C+et+al%2E+2020%2E+Language+models+are+few%2Dshot+lear)
4. Paweł Budzianowski, Tsung-Hsien Wen, Bo-Hsiang Tseng, Iñigo Casanueva, Stefan Ultes, Osman Ramadan, and Milica Gasic. 2018. Multiwoz-a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Pr… [find](https://scholar.google.com/scholar?q=Pawe%C5%82+Budzianowski%2C+Tsung%2DHsien+Wen%2C+Bo%2DHsiang+Tseng%2C+I%C3%B1igo+Casanueva%2C+Stefan+Ultes%2C+Osman+Ramadan%2C+and+Milica+Gasic%2E+2018%2E+Multiwoz%2Da+large%2Dscale+multi%2Ddomain+wizard%2Dof%2Doz+dataset+for+task%2Doriented+d)
5. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, et al. 2021a. Evaluating large language models trained on code. arXiv preprint arXiv:2107.03374. [find](https://scholar.google.com/scholar?q=Mark+Chen%2C+Jerry+Tworek%2C+Heewoo+Jun%2C+Qiming+Yuan%2C+Henrique+Ponde+de+Oliveira+Pinto%2C+Jared+Kaplan%2C+Harri+Edwards%2C+Yuri+Burda%2C+Nicholas+Joseph%2C+Greg+Brockman%2C+et+al%2E+2021a%2E+Evaluating+large+language+mod) [our card](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
6. Wenhu Chen, Hongmin Wang, Jianshu Chen, Yunkai Zhang, Hong Wang, Shiyang Li, Xiyou Zhou, and William Yang Wang. 2019. Tabfact: A large-scale dataset for table-based fact verification. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Wenhu+Chen%2C+Hongmin+Wang%2C+Jianshu+Chen%2C+Yunkai+Zhang%2C+Hong+Wang%2C+Shiyang+Li%2C+Xiyou+Zhou%2C+and+William+Yang+Wang%2E+2019%2E+Tabfact%3A+A+large%2Dscale+dataset+for+table%2Dbased+fact+verification%2E+In+International) [our card](https://md.research.elamur.ai/free/en/papers/chen-2019-tabfact-large-scale-dataset-table/)
7. Wenhu Chen, Hanwen Zha, Zhiyu Chen, Wenhan Xiong, Hong Wang, and William Yang Wang. 2020c. Hybridqa: A dataset of multi-hop question answering over tabular and textual data. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 1026–1036. [find](https://scholar.google.com/scholar?q=Wenhu+Chen%2C+Hanwen+Zha%2C+Zhiyu+Chen%2C+Wenhan+Xiong%2C+Hong+Wang%2C+and+William+Yang+Wang%2E+2020c%2E+Hybridqa%3A+A+dataset+of+multi%2Dhop+question+answering+over+tabular+and+textual+data%2E+In+Findings+of+the+Associa) [our card](https://md.research.elamur.ai/free/en/papers/chen-2020-hybridqa-dataset-multi-hop-question/)
8. Zhiyu Chen, Wenhu Chen, Charese Smiley, Sameena Shah, Iana Borova, Dylan Langdon, Reema Moussa, Matt Beane, Ting-Hao Huang, Bryan R Routledge, et al. 2021b. Finqa: A dataset of numerical reasoning over financial data. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Pro… [find](https://scholar.google.com/scholar?q=Zhiyu+Chen%2C+Wenhu+Chen%2C+Charese+Smiley%2C+Sameena+Shah%2C+Iana+Borova%2C+Dylan+Langdon%2C+Reema+Moussa%2C+Matt+Beane%2C+Ting%2DHao+Huang%2C+Bryan+R+Routledge%2C+et+al%2E+2021b%2E+Finqa%3A+A+dataset+of+numerical+reasoning+ove) [our card](https://md.research.elamur.ai/free/en/papers/chen-2021-finqa-dataset-numerical-reasoning-over/)
9. Zhoujun Cheng, Tianbao Xie, Peng Shi, Chengzu Li, Rahul Nadkarni, Yushi Hu, Caiming Xiong, Dragomir Radev, Mari Ostendorf, Luke Zettlemoyer, et al. 2022. Binding language models in symbolic languages. arXiv preprint arXiv:2210.02875. [find](https://scholar.google.com/scholar?q=Zhoujun+Cheng%2C+Tianbao+Xie%2C+Peng+Shi%2C+Chengzu+Li%2C+Rahul+Nadkarni%2C+Yushi+Hu%2C+Caiming+Xiong%2C+Dragomir+Radev%2C+Mari+Ostendorf%2C+Luke+Zettlemoyer%2C+et+al%2E+2022%2E+Binding+language+models+in+symbolic+languages%2E) [our card](https://md.research.elamur.ai/free/en/papers/cheng-2022-binding-language-models-symbolic-languages/)
10. Aakanksha Chowdhery, Sharan Narang, Jacob Devlin, Maarten Bosma, Gaurav Mishra, Adam Roberts, Paul Barham, Hyung Won Chung, Charles Sutton, Sebastian Gehrmann, et al. 2022. Palm: Scaling language modeling with pathways. arXiv preprint arXiv:2204.02311. [find](https://scholar.google.com/scholar?q=Aakanksha+Chowdhery%2C+Sharan+Narang%2C+Jacob+Devlin%2C+Maarten+Bosma%2C+Gaurav+Mishra%2C+Adam+Roberts%2C+Paul+Barham%2C+Hyung+Won+Chung%2C+Charles+Sutton%2C+Sebastian+Gehrmann%2C+et+al%2E+2022%2E+Palm%3A+Scaling+language+mode)
11. Xiang Deng, Huan Sun, Alyssa Lees, You Wu, and Cong Yu. 2022. Turl: Table understanding through representation learning. ACM SIGMOD Record, 51(1):33– 40. [find](https://scholar.google.com/scholar?q=Xiang+Deng%2C+Huan+Sun%2C+Alyssa+Lees%2C+You+Wu%2C+and+Cong+Yu%2E+2022%2E+Turl%3A+Table+understanding+through+representation+learning%2E+ACM+SIGMOD+Record%2C+51%281%29%3A33%E2%80%93+40%2E)
12. Andrew Drozdov, Nathanael Schärli, Ekin Akyürek, Nathan Scales, Xinying Song, Xinyun Chen, Olivier Bousquet, and Denny Zhou. 2022. Compositional semantic parsing with large language models. arXiv preprint arXiv:2209.15003. [find](https://scholar.google.com/scholar?q=Andrew+Drozdov%2C+Nathanael+Sch%C3%A4rli%2C+Ekin+Aky%C3%BCrek%2C+Nathan+Scales%2C+Xinying+Song%2C+Xinyun+Chen%2C+Olivier+Bousquet%2C+and+Denny+Zhou%2E+2022%2E+Compositional+semantic+parsing+with+large+language+models%2E+arXiv+prep)
13. Nan Du, Yanping Huang, Andrew M Dai, Simon Tong, Dmitry Lepikhin, Yuanzhong Xu, Maxim Krikun, Yanqi Zhou, Adams Wei Yu, Orhan Firat, et al. 2022. Glam: Efficient scaling of language models with mixture-of-experts. In International Conference on Machine Learning, pages 5547–5569. PMLR. [find](https://scholar.google.com/scholar?q=Nan+Du%2C+Yanping+Huang%2C+Andrew+M+Dai%2C+Simon+Tong%2C+Dmitry+Lepikhin%2C+Yuanzhong+Xu%2C+Maxim+Krikun%2C+Yanqi+Zhou%2C+Adams+Wei+Yu%2C+Orhan+Firat%2C+et+al%2E+2022%2E+Glam%3A+Efficient+scaling+of+language+models+with+mixtur) [our card](https://md.research.elamur.ai/free/en/papers/du-2021-glam-efficient-scaling-language-models/)
14. Wenhu Chen, Ming-Wei Chang, Eva Schlinger, William Yang Wang, and William W Cohen. 2020a. Open question answering over tables and text. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Wenhu+Chen%2C+Ming%2DWei+Chang%2C+Eva+Schlinger%2C+William+Yang+Wang%2C+and+William+W+Cohen%2E+2020a%2E+Open+question+answering+over+tables+and+text%2E+In+International+Conference+on+Learning+Representations%2E)
15. Julian Eisenschlos, Syrine Krichene, and Thomas Mueller. 2020. Understanding tables with intermediate pre-training. In Findings of the Association for Computational Linguistics: EMNLP 2020, pages 281–296. [find](https://scholar.google.com/scholar?q=Julian+Eisenschlos%2C+Syrine+Krichene%2C+and+Thomas+Mueller%2E+2020%2E+Understanding+tables+with+intermediate+pre%2Dtraining%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2020%2C+pages+281%E2%80%93)
16. Wenhu Chen, Jianshu Chen, Yu Su, Zhiyu Chen, and William Yang Wang. 2020b. Logical natural language generation from open-domain tables. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 7929– 7942. [find](https://scholar.google.com/scholar?q=Wenhu+Chen%2C+Jianshu+Chen%2C+Yu+Su%2C+Zhiyu+Chen%2C+and+William+Yang+Wang%2E+2020b%2E+Logical+natural+language+generation+from+open%2Ddomain+tables%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Association+for) [our card](https://md.research.elamur.ai/free/en/papers/chen-2020-logical-natural-language-generation-open/)
17. Jonathan Herzig, Pawel Krzysztof Nowak, Thomas Mueller, Francesco Piccinno, and Julian Eisenschlos. 2020. Tapas: Weakly supervised table parsing via pre-training. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4320–4333. [find](https://scholar.google.com/scholar?q=Jonathan+Herzig%2C+Pawel+Krzysztof+Nowak%2C+Thomas+Mueller%2C+Francesco+Piccinno%2C+and+Julian+Eisenschlos%2E+2020%2E+Tapas%3A+Weakly+supervised+table+parsing+via+pre%2Dtraining%2E+In+Proceedings+of+the+58th+Annual+Mee) [our card](https://md.research.elamur.ai/free/en/papers/herzig-2020-tapas-weakly-supervised-table-parsing/)
18. Takeshi Kojima, Shixiang Shane Gu, Machel Reid, Yutaka Matsuo, and Yusuke Iwasawa. 2022. Large language models are zero-shot reasoners. arXiv preprint arXiv:2205.11916. [find](https://scholar.google.com/scholar?q=Takeshi+Kojima%2C+Shixiang+Shane+Gu%2C+Machel+Reid%2C+Yutaka+Matsuo%2C+and+Yusuke+Iwasawa%2E+2022%2E+Large+language+models+are+zero%2Dshot+reasoners%2E+arXiv+preprint+arXiv%3A2205%2E11916%2E) [our card](https://md.research.elamur.ai/free/en/papers/kojima-2022-large-language-models-zero-shot/)
19. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2020. Bart: Denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension. In Proceedings of the 58th Annual Meeting of the… [find](https://scholar.google.com/scholar?q=Mike+Lewis%2C+Yinhan+Liu%2C+Naman+Goyal%2C+Marjan+Ghazvininejad%2C+Abdelrahman+Mohamed%2C+Omer+Levy%2C+Veselin+Stoyanov%2C+and+Luke+Zettlemoyer%2E+2020%2E+Bart%3A+Denoising+sequence%2Dto%2Dsequence+pre%2Dtraining+for+natural+l) [our card](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/)
20. Chen Liang, Mohammad Norouzi, Jonathan Berant, Quoc V Le, and Ni Lao. 2018. Memory augmented policy optimization for program synthesis and semantic parsing. Advances in Neural Information Processing Systems, 31. [find](https://scholar.google.com/scholar?q=Chen+Liang%2C+Mohammad+Norouzi%2C+Jonathan+Berant%2C+Quoc+V+Le%2C+and+Ni+Lao%2E+2018%2E+Memory+augmented+policy+optimization+for+program+synthesis+and+semantic+parsing%2E+Advances+in+Neural+Information+Processing+S)
21. Qian Liu, Bei Chen, Jiaqi Guo, Morteza Ziyadi, Zeqi Lin, Weizhu Chen, and Jian-Guang Lou. 2021. Tapex: Table pre-training via learning a neural sql executor. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Qian+Liu%2C+Bei+Chen%2C+Jiaqi+Guo%2C+Morteza+Ziyadi%2C+Zeqi+Lin%2C+Weizhu+Chen%2C+and+Jian%2DGuang+Lou%2E+2021%2E+Tapex%3A+Table+pre%2Dtraining+via+learning+a+neural+sql+executor%2E+In+International+Conference+on+Learning+Re) [our card](https://md.research.elamur.ai/free/en/papers/liu-2021-tapex-table-pre-training-learning/)
22. Kai Nakamura, Sharon Levy, Yi-Lin Tuan, Wenhu Chen, and William Yang Wang. 2022. Hybridialogue: An information-seeking dialogue dataset grounded on tabular and textual data. In Findings of the Association for Computational Linguistics: ACL 2022, pages 481–492. generation dataset. In Proceedings of t… [find](https://scholar.google.com/scholar?q=Kai+Nakamura%2C+Sharon+Levy%2C+Yi%2DLin+Tuan%2C+Wenhu+Chen%2C+and+William+Yang+Wang%2E+2022%2E+Hybridialogue%3A+An+information%2Dseeking+dialogue+dataset+grounded+on+tabular+and+textual+data%2E+In+Findings+of+the+Associa)
23. Panupong Pasupat and Percy Liang. 2015. Compositional semantic parsing on semi-structured tables. In Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 1470–… [find](https://scholar.google.com/scholar?q=Panupong+Pasupat+and+Percy+Liang%2E+2015%2E+Compositional+semantic+parsing+on+semi%2Dstructured+tables%2E+In+Proceedings+of+the+53rd+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+7th)
24. Jack W Rae, Sebastian Borgeaud, Trevor Cai, Katie Millican, Jordan Hoffmann, Francis Song, John Aslanides, Sarah Henderson, Roman Ring, Susannah Young, et al. 2021. Scaling language models: Methods, analysis & insights from training gopher. arXiv preprint arXiv:2112.11446. [find](https://scholar.google.com/scholar?q=Jack+W+Rae%2C+Sebastian+Borgeaud%2C+Trevor+Cai%2C+Katie+Millican%2C+Jordan+Hoffmann%2C+Francis+Song%2C+John+Aslanides%2C+Sarah+Henderson%2C+Roman+Ring%2C+Susannah+Young%2C+et+al%2E+2021%2E+Scaling+language+models%3A+Methods%2C+a)
25. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, Peter J Liu, et al. 2020. Exploring the limits of transfer learning with a unified text-to-text transformer. J. Mach. Learn. Res., 21(140):1–67. [find](https://scholar.google.com/scholar?q=Colin+Raffel%2C+Noam+Shazeer%2C+Adam+Roberts%2C+Katherine+Lee%2C+Sharan+Narang%2C+Michael+Matena%2C+Yanqi+Zhou%2C+Wei+Li%2C+Peter+J+Liu%2C+et+al%2E+2020%2E+Exploring+the+limits+of+transfer+learning+with+a+unified+text%2Dto%2Dt)
26. Victor Sanh, Albert Webson, Colin Raffel, Stephen Bach, Lintang Sutawika, Zaid Alyafeai, Antoine Chaffin, Arnaud Stiegler, Teven Le Scao, Arun Raja, et al. 2022. Multitask prompted training enables zero-shot task generalization. In The Tenth International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Victor+Sanh%2C+Albert+Webson%2C+Colin+Raffel%2C+Stephen+Bach%2C+Lintang+Sutawika%2C+Zaid+Alyafeai%2C+Antoine+Chaffin%2C+Arnaud+Stiegler%2C+Teven+Le+Scao%2C+Arun+Raja%2C+et+al%2E+2022%2E+Multitask+prompted+training+enables+ze) [our card](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)
27. Linyong Nan, Chiachun Hsieh, Ziming Mao, Xi Victoria Lin, Neha Verma, Rui Zhang, Wojciech Kryściński, Nick Schoelkopf, Riley Kong, Xiangru Tang, et al. 2022. Fetaqa: Free-form table question answering. Transactions of the Association for Computational Linguistics, 10:35–49. [find](https://scholar.google.com/scholar?q=Linyong+Nan%2C+Chiachun+Hsieh%2C+Ziming+Mao%2C+Xi+Victoria+Lin%2C+Neha+Verma%2C+Rui+Zhang%2C+Wojciech+Krys%CC%81cin%CC%81ski%2C+Nick+Schoelkopf%2C+Riley+Kong%2C+Xiangru+Tang%2C+et+al%2E+2022%2E+Fetaqa%3A+Free%2Dform+table+question+answeri) [our card](https://md.research.elamur.ai/free/en/papers/nan-2022-fetaqa-free-form-table-question/)
28. Shaden Smith, Mostofa Patwary, Brandon Norick, Patrick LeGresley, Samyam Rajbhandari, Jared Casper, Zhun Liu, Shrimai Prabhumoye, George Zerveas, Vijay Korthikanti, et al. 2022. Using deepspeed and megatron to train megatron-turing nlg 530b, a large-scale generative language model. arXiv preprint ar… [find](https://scholar.google.com/scholar?q=Shaden+Smith%2C+Mostofa+Patwary%2C+Brandon+Norick%2C+Patrick+LeGresley%2C+Samyam+Rajbhandari%2C+Jared+Casper%2C+Zhun+Liu%2C+Shrimai+Prabhumoye%2C+George+Zerveas%2C+Vijay+Korthikanti%2C+et+al%2E+2022%2E+Using+deepspeed+and+me) [our card](https://md.research.elamur.ai/free/en/papers/smith-2022-using-deepspeed-megatron-train-megatron/)
29. Linyong Nan, Dragomir Radev, Rui Zhang, Amrit Rau, Abhinand Sivaprasad, Chiachun Hsieh, Xiangru Tang, Aadit Vyas, Neha Verma, Pranav Krishna, et al. 2021. Dart: Open-domain structured data record to text generation. In Proceedings of the 2021 Conference of the North American Chapter of the Associati… [find](https://scholar.google.com/scholar?q=Linyong+Nan%2C+Dragomir+Radev%2C+Rui+Zhang%2C+Amrit+Rau%2C+Abhinand+Sivaprasad%2C+Chiachun+Hsieh%2C+Xiangru+Tang%2C+Aadit+Vyas%2C+Neha+Verma%2C+Pranav+Krishna%2C+et+al%2E+2021%2E+Dart%3A+Open%2Ddomain+structured+data+record+to+t) [our card](https://md.research.elamur.ai/free/en/papers/nan-2020-dart-open-domain-structured-data/)
30. Alon Talmor, Ori Yoran, Amnon Catav, Dan Lahav, Yizhong Wang, Akari Asai, Gabriel Ilharco, Hannaneh Hajishirzi, and Jonathan Berant. 2020. Multimodalqa: complex question answering over text, tables and images. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Alon+Talmor%2C+Ori+Yoran%2C+Amnon+Catav%2C+Dan+Lahav%2C+Yizhong+Wang%2C+Akari+Asai%2C+Gabriel+Ilharco%2C+Hannaneh+Hajishirzi%2C+and+Jonathan+Berant%2E+2020%2E+Multimodalqa%3A+complex+question+answering+over+text%2C+tables+an)
31. Long Ouyang, Jeff Wu, Xu Jiang, Diogo Almeida, Carroll L Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Ray, et al. 2022. Training language models to follow instructions with human feedback. arXiv preprint arXiv:2203.02155. [find](https://scholar.google.com/scholar?q=Long+Ouyang%2C+Jeff+Wu%2C+Xu+Jiang%2C+Diogo+Almeida%2C+Carroll+L+Wainwright%2C+Pamela+Mishkin%2C+Chong+Zhang%2C+Sandhini+Agarwal%2C+Katarina+Slama%2C+Alex+Ray%2C+et+al%2E+2022%2E+Training+language+models+to+follow+instructio)
32. Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. Bleu: a method for automatic evaluation of machine translation. In Proceedings of the 40th annual meeting of the Association for Computational Linguistics, pages 311–318. [find](https://scholar.google.com/scholar?q=Kishore+Papineni%2C+Salim+Roukos%2C+Todd+Ward%2C+and+Wei%2DJing+Zhu%2E+2002%2E+Bleu%3A+a+method+for+automatic+evaluation+of+machine+translation%2E+In+Proceedings+of+the+40th+annual+meeting+of+the+Association+for+Comp) [our card](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/)
33. Ankur Parikh, Xuezhi Wang, Sebastian Gehrmann, Manaal Faruqui, Bhuwan Dhingra, Diyi Yang, and Dipanjan Das. 2020. Totto: A controlled table-to-text Bailin Wang, Ivan Titov, and Mirella Lapata. 2019. Learning semantic parsers from denotations with latent structured alignments and abstract programs. I… [find](https://scholar.google.com/scholar?q=Ankur+Parikh%2C+Xuezhi+Wang%2C+Sebastian+Gehrmann%2C+Manaal+Faruqui%2C+Bhuwan+Dhingra%2C+Diyi+Yang%2C+and+Dipanjan+Das%2E+2020%2E+Totto%3A+A+controlled+table%2Dto%2Dtext+Bailin+Wang%2C+Ivan+Titov%2C+and+Mirella+Lapata%2E+2019%2E+L)
34. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, and Denny Zhou. 2022. Self-consistency improves chain of thought reasoning in language models. arXiv preprint arXiv:2203.11171. [find](https://scholar.google.com/scholar?q=Xuezhi+Wang%2C+Jason+Wei%2C+Dale+Schuurmans%2C+Quoc+Le%2C+Ed+Chi%2C+and+Denny+Zhou%2E+2022%2E+Self%2Dconsistency+improves+chain+of+thought+reasoning+in+language+models%2E+arXiv+preprint+arXiv%3A2203%2E11171%2E) [our card](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/)
35. Jason Wei, Xuezhi Wang, Dale Schuurmans, Maarten Bosma, Ed Chi, Quoc Le, and Denny Zhou. 2022. Chain of thought prompting elicits reasoning in large language models. arXiv preprint arXiv:2201.11903. [find](https://scholar.google.com/scholar?q=Jason+Wei%2C+Xuezhi+Wang%2C+Dale+Schuurmans%2C+Maarten+Bosma%2C+Ed+Chi%2C+Quoc+Le%2C+and+Denny+Zhou%2E+2022%2E+Chain+of+thought+prompting+elicits+reasoning+in+large+language+models%2E+arXiv+preprint+arXiv%3A2201%2E11903%2E) [our card](https://md.research.elamur.ai/free/en/papers/wei-2022-chain-thought-prompting-elicits-reasoning/)
36. Tianbao Xie, Chen Henry Wu, Peng Shi, Ruiqi Zhong, Torsten Scholak, Michihiro Yasunaga, Chien-Sheng Wu, Ming Zhong, Pengcheng Yin, Sida I. Wang, Victor Zhong, Bailin Wang, Chengzu Li, Connor Boyle, Ansong Ni, Ziyu Yao, Dragomir Radev, Caiming Xiong, Lingpeng Kong, Rui Zhang, Noah A. Smith, Luke Zett… [find](https://scholar.google.com/scholar?q=Tianbao+Xie%2C+Chen+Henry+Wu%2C+Peng+Shi%2C+Ruiqi+Zhong%2C+Torsten+Scholak%2C+Michihiro+Yasunaga%2C+Chien%2DSheng+Wu%2C+Ming+Zhong%2C+Pengcheng+Yin%2C+Sida+I%2E+Wang%2C+Victor+Zhong%2C+Bailin+Wang%2C+Chengzu+Li%2C+Connor+Boyle%2C+An) [our card](https://md.research.elamur.ai/free/en/papers/xie-2022-unifiedskg-unifying-multi-tasking-structured/)
37. Xiaoyu Yang, Feng Nie, Yufei Feng, Quan Liu, Zhigang Chen, and Xiaodan Zhu. 2020. Program enhanced fact verification with verbalization and graph attention network. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 7810–7825. [find](https://scholar.google.com/scholar?q=Xiaoyu+Yang%2C+Feng+Nie%2C+Yufei+Feng%2C+Quan+Liu%2C+Zhigang+Chen%2C+and+Xiaodan+Zhu%2E+2020%2E+Program+enhanced+fact+verification+with+verbalization+and+graph+attention+network%2E+In+Proceedings+of+the+2020+Conferen)
38. Denny Zhou, Nathanael Schärli, Le Hou, Jason Wei, Nathan Scales, Xuezhi Wang, Dale Schuurmans, Olivier Bousquet, Quoc Le, and Ed Chi. 2022. Least-to-most prompting enables complex reasoning in large language models. arXiv preprint arXiv:2205.10625. [find](https://scholar.google.com/scholar?q=Denny+Zhou%2C+Nathanael+Sch%C3%A4rli%2C+Le+Hou%2C+Jason+Wei%2C+Nathan+Scales%2C+Xuezhi+Wang%2C+Dale+Schuurmans%2C+Olivier+Bousquet%2C+Quoc+Le%2C+and+Ed+Chi%2E+2022%2E+Least%2Dto%2Dmost+prompting+enables+complex+reasoning+in+large+l) [our card](https://md.research.elamur.ai/free/en/papers/al-2022-least-most-prompting-enables-complex/)
39. Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua. 2021. Tat-qa: A question answering benchmark on a hybrid of tabular and textual content in finance. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Fengbin+Zhu%2C+Wenqiang+Lei%2C+Youcheng+Huang%2C+Chao+Wang%2C+Shuo+Zhang%2C+Jiancheng+Lv%2C+Fuli+Feng%2C+and+Tat%2DSeng+Chua%2E+2021%2E+Tat%2Dqa%3A+A+question+answering+benchmark+on+a+hybrid+of+tabular+and+textual+content+in) [our card](https://md.research.elamur.ai/free/en/papers/zhu-2021-tat-qa-question-answering-benchmark/)
40. Pengcheng Yin, Graham Neubig, Wen-tau Yih, and Sebastian Riedel. 2020. Tabert: Pretraining for joint understanding of textual and tabular data. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 8413–8426. [find](https://scholar.google.com/scholar?q=Pengcheng+Yin%2C+Graham+Neubig%2C+Wen%2Dtau+Yih%2C+and+Sebastian+Riedel%2E+2020%2E+Tabert%3A+Pretraining+for+joint+understanding+of+textual+and+tabular+data%2E+In+Proceedings+of+the+58th+Annual+Meeting+of+the+Associa) [our card](https://md.research.elamur.ai/free/en/papers/yin-2020-tabert-pretraining-joint-understanding-textual/)
41. Tao Yu, Chien-Sheng Wu, Xi Victoria Lin, Bailin Wang, Yi Chern Tan, Xinyi Yang, Dragomir R Radev, Richard Socher, and Caiming Xiong. 2021. Grappa: Grammar-augmented pre-training for table semantic parsing. In ICLR. [find](https://scholar.google.com/scholar?q=Tao+Yu%2C+Chien%2DSheng+Wu%2C+Xi+Victoria+Lin%2C+Bailin+Wang%2C+Yi+Chern+Tan%2C+Xinyi+Yang%2C+Dragomir+R+Radev%2C+Richard+Socher%2C+and+Caiming+Xiong%2E+2021%2E+Grappa%3A+Grammar%2Daugmented+pre%2Dtraining+for+table+semantic+par)
42. Tao Yu, Rui Zhang, Kai Yang, Michihiro Yasunaga, Dongxu Wang, Zifan Li, James Ma, Irene Li, Qingning Yao, Shanelle Roman, et al. 2018. Spider: A large-scale human-labeled dataset for complex and cross-domain semantic parsing and text-to-sql task. In Proceedings of the 2018 Conference on Empirical Me… [find](https://scholar.google.com/scholar?q=Tao+Yu%2C+Rui+Zhang%2C+Kai+Yang%2C+Michihiro+Yasunaga%2C+Dongxu+Wang%2C+Zifan+Li%2C+James+Ma%2C+Irene+Li%2C+Qingning+Yao%2C+Shanelle+Roman%2C+et+al%2E+2018%2E+Spider%3A+A+large%2Dscale+human%2Dlabeled+dataset+for+complex+and+cross) [our card](https://md.research.elamur.ai/free/en/papers/yu-2018-spider-large-scale-human-labeled/)
43. Hongzhi Zhang, Yingyao Wang, Sirui Wang, Xuezhi Cao, Fuzheng Zhang, and Zhongyuan Wang. 2020. Table fact verification with structure-aware transformer. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1624–1629. [find](https://scholar.google.com/scholar?q=Hongzhi+Zhang%2C+Yingyao+Wang%2C+Sirui+Wang%2C+Xuezhi+Cao%2C+Fuzheng+Zhang%2C+and+Zhongyuan+Wang%2E+2020%2E+Table+fact+verification+with+structure%2Daware+transformer%2E+In+Proceedings+of+the+2020+Conference+on+Empiric)
44. Yuchen Zhang, Panupong Pasupat, and Percy Liang. 2017. Macro grammars and holistic triggering for efficient semantic parsing. In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 1214–1223. [find](https://scholar.google.com/scholar?q=Yuchen+Zhang%2C+Panupong+Pasupat%2C+and+Percy+Liang%2E+2017%2E+Macro+grammars+and+holistic+triggering+for+efficient+semantic+parsing%2E+In+Proceedings+of+the+2017+Conference+on+Empirical+Methods+in+Natural+Lang)
45. Victor Zhong, Caiming Xiong, and Richard Socher. 2017. Seq2sql: Generating structured queries from natural language using reinforcement learning. arXiv preprint arXiv:1709.00103. [find](https://scholar.google.com/scholar?q=Victor+Zhong%2C+Caiming+Xiong%2C+and+Richard+Socher%2E+2017%2E+Seq2sql%3A+Generating+structured+queries+from+natural+language+using+reinforcement+learning%2E+arXiv+preprint+arXiv%3A1709%2E00103%2E)
46. Wanjun Zhong, Duyu Tang, Zhangyin Feng, Nan Duan, Ming Zhou, Ming Gong, Linjun Shou, Daxin Jiang, Jiahai Wang, and Jian Yin. 2020. Logicalfactchecker: Leveraging logical operations for fact checking with graph module network. In Proceedings of the 58th Annual Meeting of the Association for Computati… [find](https://scholar.google.com/scholar?q=Wanjun+Zhong%2C+Duyu+Tang%2C+Zhangyin+Feng%2C+Nan+Duan%2C+Ming+Zhou%2C+Ming+Gong%2C+Linjun+Shou%2C+Daxin+Jiang%2C+Jiahai+Wang%2C+and+Jian+Yin%2E+2020%2E+Logicalfactchecker%3A+Leveraging+logical+operations+for+fact+checking+w)