--- title: "FeTaQA: Free-form Table Question Answering — Nan 2022" description: "Most table question answering benchmarks expect a short entity or value as the answer." authors: ["Linyong Nan", "Chiachun Hsieh", "Ziming Mao", "Xi Victoria Lin", "Neha Verma", "Rui Zhang", "Wojciech Kryściński", "Hailey Schoelkopf", "Riley Kong", "Xiangru Tang", "Mutethia Mutuma", "Ben Rosand", "Isabel Trindade", "Renusree Bandaru", "Jacob Cunningham", "Caiming Xiong", "Dragomir Radev"] year: 2022 venue: "Transactions of the Association for Computational Linguistics" source: "https://aclanthology.org/2022.tacl-1.3/" doi: "10.1162/tacl_a_00446" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/nan-2022-fetaqa-free-form-table-question/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/nan-2022-fetaqa-free-form-table-question/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/2/) / FeTaQA: Free-form Table Question Answering # FeTaQA: Free-form Table Question Answering **Authors:** [Linyong Nan](https://md.research.elamur.ai/free/en/authors/linyong-nan/), [Chiachun Hsieh](https://md.research.elamur.ai/free/en/authors/chiachun-hsieh/), Ziming Mao, [Xi Victoria Lin](https://md.research.elamur.ai/free/en/authors/xi-victoria-lin/), [Neha Verma](https://md.research.elamur.ai/free/en/authors/neha-verma/), [Rui Zhang](https://md.research.elamur.ai/free/en/authors/rui-zhang/), [Wojciech Kryściński](https://md.research.elamur.ai/free/en/authors/wojciech-kryscinski/), [Hailey Schoelkopf](https://md.research.elamur.ai/free/en/authors/hailey-schoelkopf/), Riley Kong, [Xiangru Tang](https://md.research.elamur.ai/free/en/authors/xiangru-tang/), [Mutethia Mutuma](https://md.research.elamur.ai/free/en/authors/mutethia-mutuma/), Ben Rosand et al. (17 authors) · **Year:** 2022 · **Venue:** Transactions of the Association for Computational Linguistics · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2022.tacl-1.3/) · [PDF](https://aclanthology.org/2022.tacl-1.3.pdf) · [DOI 10.1162/tacl\_a\_00446](https://doi.org/10.1162/tacl_a_00446) · [OpenAlex](https://openalex.org/W4210451781) **Citations:** 111 citations (openalex-snapshot · 2026-10-14) · [123](#cited-by) in our library, of the 140673 papers whose bibliographies we read ## Summary Most table question answering benchmarks expect a short entity or value as the answer. Real users, however, often ask questions that require combining several facts from a table into a full sentence. To address this gap, the authors built FeTaQA, a dataset of 10,330 Wikipedia table questions with free-form answers. They did not have annotators write both questions and answers. Instead, they took table-grounded sentences from ToTTo, kept tables of moderate size whose highlighted cells span multiple rows or columns, and asked annotators to write questions that those sentences answer. The highlighted cells also serve as weak supervision. Two baselines were tested: - a pipeline in which TAPAS selects cells and a separate generator writes the answer; - an end-to-end T5 model that reads a linearized table. The end-to-end model clearly outperformed the pipeline on automatic metrics and in human evaluation, but it still fell well short of the reference answers. FeTaQA is the first generative QA benchmark over tables, and it tests whether a system can retrieve, reason over, and integrate structured data. ## Key points - **Dataset composition:** FeTaQA has 10,330 instances. Internal experts produced 3,039 (30%) and Amazon Mechanical Turk workers produced 7,291 (70%). Reviewers rejected 12% of crowd annotations and edited only 16.7% of those they approved. - **Answer length:** Answers average 18.9 words, compared with 1.7 in WikiTableQuestions and 2.1 in HybridQA. - **Human evaluation (share of answers rated 4–5):** | Answer source | Correct | Adequate | Faithful | |---|---|---|---| | End-to-End T5 | 54.8% | 48.4% | 50.4% | | Pipeline | 25.4% | 8.4% | 23.6% | | References | 92.4% | 90.6% | 95.6% | - **Pipeline weakness:** Gold annotations highlight a median of six cells, but the TAPAS parser predicts only about two cells, or about three after fine-tuning. Fine-tuning raises the pipeline by nearly 2 BLEU. With gold cells, the pipeline beats End-to-End. - **Availability:** The data and code are released at github.com/Yale-LILY/FeTaQA. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Natural language processing](https://md.research.elamur.ai/free/en/shelves/nlp/) ## Cited by Cited by 123 papers of our library cite it (of the 140673 whose bibliographies we read); 7 of them have a page here, 116 not yet. - [Large Language Models are few(1)-shot Table Reasoners](https://md.research.elamur.ai/free/en/papers/chen-2023-large-language-models-are-few/) - [Towards Reasoning in Large Language Models: A Survey](https://md.research.elamur.ai/free/en/papers/huang-2023-reasoning-large-language-models-survey/) - [AgentBench: Evaluating LLMs as Agents](https://md.research.elamur.ai/free/en/papers/liu-2023-agentbench-evaluating-llms-agents/) - [Dynamic Prompt Learning via Policy Gradient for Semi-structured Mathematical Reasoning](https://md.research.elamur.ai/free/en/papers/lu-2022-dynamic-prompt-learning-policy-gradient/) - [ChartQA: A Benchmark for Question Answering about Charts with Visual and Logical Reasoning](https://md.research.elamur.ai/free/en/papers/masry-2022-chartqa-benchmark-question-answering-about/) - [UL2: Unifying Language Learning Paradigms](https://md.research.elamur.ai/free/en/papers/tay-2022-ul2-unifying-language-learning-paradigms/) - [UnifiedSKG: Unifying and Multi-Tasking Structured Knowledge Grounding with Text-to-Text Language Models](https://md.research.elamur.ai/free/en/papers/xie-2022-unifiedskg-unifying-multi-tasking-structured/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @article{nan2022fetaqa, title = {{FeTaQA: Free-form Table Question Answering}}, author = {Nan, Linyong and Hsieh, Chiachun and Mao, Ziming and Lin, Xi Victoria and Verma, Neha and Zhang, Rui and Kryściński, Wojciech and Schoelkopf, Hailey and Kong, Riley and Tang, Xiangru and Mutuma, Mutethia and Rosand, Ben and Trindade, Isabel and Bandaru, Renusree and Cunningham, Jacob and Xiong, Caiming and Radev, Dragomir}, journal = {Transactions of the Association for Computational Linguistics}, year = {2022}, doi = {10.1162/tacl\_a\_00446}, url = {https://doi.org/10.1162/tacl_a_00446} } ``` - [← previous Fantastically Ordered Prompts and Where to Find Them: Overcoming Few-Shot Prompt Order Sensitivity](https://md.research.elamur.ai/free/en/papers/lu-2022-fantastically-ordered-prompts-where-find/) - [next → Few-Shot Anaphora Resolution in Scientific Protocols via Mixtures of In-Context Experts](https://md.research.elamur.ai/free/en/papers/le-2022-few-shot-anaphora-resolution-scientific/) - [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/2/) ## References 1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Sumit Asthana and Aaron Halfaker. 2018. With few eyes, all hoaxes are deep. In Proceedings of the ACM on Human Computer Interaction 2(CSCW). https://doi.org/10 .1145/3274290 [find](https://scholar.google.com/scholar?q=Sumit+Asthana+and+Aaron+Halfaker%2E+2018%2E+With+few+eyes%2C+all+hoaxes+are+deep%2E+In+Proceedings+of+the+ACM+on+Human+Computer+Interaction+2%28CSCW%29%2E+https%3A%2F%2Fdoi%2Eorg%2F10+%2E1145%2F3274290)