--- title: "Synthetic Function Demonstrations Improve — McKenna 2026" description: "LLMs struggle with programming languages that have little natural training data, such as Excel formulas." authors: ["Nick McKenna", "Xinnuo Xu", "Jack Williams", "Nicholas C. Wilson", "Benjamin Van Durme", "Christian Poelitz"] year: 2026 venue: "Proceedings of the Fifteenth Language Resources and Evaluation Conference" source: "https://aclanthology.org/2026.lrec-1.242/" doi: "10.63317/4tutdq38ch4b" licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)" licence_url: "https://creativecommons.org/licenses/by-nc/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/papers/mckenna-2026-synthetic-function-demonstrations-improve-generation/" --- [Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/mckenna-2026-synthetic-function-demonstrations-improve-generation/) [home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2026](https://md.research.elamur.ai/free/papers/2026/page/4/) / Synthetic Function Demonstrations Improve Generation in Low-Resource Programming Languages # Synthetic Function Demonstrations Improve Generation in Low-Resource Programming Languages **Authors:** Nick McKenna, Xinnuo Xu, Jack Williams, Nicholas C. Wilson, [Benjamin Van Durme](https://md.research.elamur.ai/free/authors/benjamin-van-durme/), Christian Poelitz · **Year:** 2026 · **Venue:** Proceedings of the Fifteenth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.lrec-1.242/) · [PDF](https://aclanthology.org/2026.lrec-1.242.pdf) · [DOI 10.63317/4tutdq38ch4b](https://doi.org/10.63317/4tutdq38ch4b) · [OpenAlex](https://openalex.org/W4417174056) **Citations:** 0 citations (openalex-snapshot · 2026-10-14) · 1 in our library, of the 140673 papers whose bibliographies we read ## Summary LLMs struggle with programming languages that have little natural training data, such as Excel formulas. Even GPT-4o scores only 58.2% on a central Excel test set. The authors propose a recipe for building finetuning data for such low-resource languages that does not require a teacher already skilled in them. They select the 100 most used Excel functions, estimated from a sample of public web spreadsheets, out of 505 in total. They gather each function's official Microsoft documentation and have GPT-4o pair it with a fitting real table from WikiTableQuestions. GPT-4o then writes tutorial-like examples, each with a question, a step-by-step explanation and an executable formula. The examples cover the function's arguments one at a time, including optional ones. Qwen 2.5 and Llama 2 students of varying size, with and without code pretraining, are finetuned on this data and tested on WikiTQ and TAT-QA. Most students improve table-QA accuracy by over 10%, while RAG gives only small gains. Documentation alone can therefore bootstrap model skill in niche languages. ## Key points - Pipeline inputs are function documentation, unlabeled grounding data (Wikipedia tables) and an instruction-following teacher (GPT-4o). The teacher does not need to be an Excel expert. - For each of the top 100 functions, 10 random training tables are sampled and GPT-4o picks the most suitable one. The pipeline is repeated to grow the curriculum across many tables. - Code pretraining on high-resource languages does not by itself boost Excel generation, but it makes finetuning on Excel data more effective. - RAG-only baselines barely help, even with oracle-retrieved documentation. The authors attribute this to the domain being unfamiliar to student models. - Code, prompts and data are released at github.com/microsoft/Synthetic-Function-Demonstrations. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/) ## Cite The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{mckenna2026synthetic, title = {{Synthetic Function Demonstrations Improve Generation in Low-Resource Programming Languages}}, author = {McKenna, Nick and Xu, Xinnuo and Williams, Jack and Wilson, Nicholas C. and Durme, Benjamin Van and Poelitz, Christian}, booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference}, year = {2026}, doi = {10.63317/4tutdq38ch4b}, url = {https://doi.org/10.63317/4tutdq38ch4b} } ``` - [← previous Syntax is the Key to Semantics: Combining Universal Dependencies and Abstract Meaning Representation](https://md.research.elamur.ai/free/papers/heinecke-2026-syntax-is-key-semantics-combining/) - [next → Synthetic Instruction Generation for Low-Resource Nordic Languages: Viability and Limitations in LLM Instruction-Tuning](https://md.research.elamur.ai/free/papers/stenlund-2026-synthetic-instruction-generation-low-resource/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/papers/2026/page/4/) ## References 24 references, 8 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Marah Abdin, Jyoti Aneja, Hany Awadalla, Ahmed Awadallah, Ammar Ahmad Awan, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Jianmin Bao, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Qin Cai, Vishrav Chaudhary, Dong Chen, Dongdong Chen, Weizhu Chen, Yen-Chun Chen… [find](https://scholar.google.com/scholar?q=Marah+Abdin%2C+Jyoti+Aneja%2C+Hany+Awadalla%2C+Ahmed+Awadallah%2C+Ammar+Ahmad+Awan%2C+Nguyen+Bach%2C+Amit+Bahree%2C+Arash+Bakhtiari%2C+Jianmin+Bao%2C+Harkirat+Behl%2C+Alon+Benhaim%2C+Misha+Bilenko%2C+Johan+Bjorck%2C+S%C3%A9bastien+) [our card](https://md.research.elamur.ai/free/papers/abdin-2024-phi-3-technical-report-highly/) 2. DeepSeek-AI, Daya Guo, Dejian Yang, Haowei Zhang, Junxiao Song, Ruoyu Zhang, Runxin Xu, Qihao Zhu, Shirong Ma, Peiyi Wang, Xiao Bi, Xiaokang Zhang, Xingkai Yu, Yu Wu, Z. F. Wu, Zhibin Gou, Zhihong Shao, Zhuoshu Li, Ziyi Gao, Aixin Liu, Bing Xue, Bingxuan Wang, Bochao Wu, Bei Feng, Chengda Lu, Chengg… [find](https://scholar.google.com/scholar?q=DeepSeek%2DAI%2C+Daya+Guo%2C+Dejian+Yang%2C+Haowei+Zhang%2C+Junxiao+Song%2C+Ruoyu+Zhang%2C+Runxin+Xu%2C+Qihao+Zhu%2C+Shirong+Ma%2C+Peiyi+Wang%2C+Xiao+Bi%2C+Xiaokang+Zhang%2C+Xingkai+Yu%2C+Yu+Wu%2C+Z%2E+F%2E+Wu%2C+Zhibin+Gou%2C+Zhihong+Sha) [our card](https://md.research.elamur.ai/free/papers/deepseek-2025-ai-deepseek-r1-incentivizing-reasoning-capability/) 3. Marc Fisher and Gregg Rothermel. 2005. The euses spreadsheet corpus: a shared resource for supporting experimentation with spreadsheet dependability mechanisms. SIGSOFT Softw. Eng. Notes, 30(4):1–5. [find](https://scholar.google.com/scholar?q=Marc+Fisher+and+Gregg+Rothermel%2E+2005%2E+The+euses+spreadsheet+corpus%3A+a+shared+resource+for+supporting+experimentation+with+spreadsheet+dependability+mechanisms%2E+SIGSOFT+Softw%2E+Eng%2E+Notes%2C+30%284%29%3A1%E2%80%935%2E) 4. Suriya Gunasekar, Yi Zhang, Jyoti Aneja, Caio César Teodoro Mendes, Allie Del Giorno, Sivakanth Gopi, Mojan Javaheripi, Piero Kauffmann, Gustavo de Rosa, Olli Saarikivi, Adil Salim, Shital Shah, Harkirat Singh Behl, Xin Wang, Sébastien Bubeck, Ronen Eldan, Adam Tauman Kalai, Yin Tat Lee, and Yuanzhi… [find](https://scholar.google.com/scholar?q=Suriya+Gunasekar%2C+Yi+Zhang%2C+Jyoti+Aneja%2C+Caio+C%C3%A9sar+Teodoro+Mendes%2C+Allie+Del+Giorno%2C+Sivakanth+Gopi%2C+Mojan+Javaheripi%2C+Piero+Kauffmann%2C+Gustavo+de+Rosa%2C+Olli+Saarikivi%2C+Adil+Salim%2C+Shital+Shah%2C+Harki) 5. Or Honovich, Thomas Scialom, Omer Levy, and Timo Schick. 2023. Unnatural instructions: Tuning language models with (almost) no human labor. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 14409–14428, Toronto, Canada. Associat… [find](https://scholar.google.com/scholar?q=Or+Honovich%2C+Thomas+Scialom%2C+Omer+Levy%2C+and+Timo+Schick%2E+2023%2E+Unnatural+instructions%3A+Tuning+language+models+with+%28almost%29+no+human+labor%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association) [our card](https://md.research.elamur.ai/free/papers/honovich-2023-unnatural-instructions-tuning-language-models/) 6. Sathvik Joel, Jie JW Wu, and Fatemeh H. Fard. 2024. A survey on llm-based code generation for low-resource and domain-specific programming languages. [find](https://scholar.google.com/scholar?q=Sathvik+Joel%2C+Jie+JW+Wu%2C+and+Fatemeh+H%2E+Fard%2E+2024%2E+A+survey+on+llm%2Dbased+code+generation+for+low%2Dresource+and+domain%2Dspecific+programming+languages%2E) 7. Denis Kocetkov, Raymond Li, Loubna Ben Allal, Jia Li, Chenghao Mou, Carlos Muñoz Ferrandis, Yacine Jernite, Margaret Mitchell, Sean Hughes, Thomas Wolf, Dzmitry Bahdanau, Leandro von Werra, and Harm de Vries. 2022. The stack: 3 tb of permissively licensed source code. Preprint. [find](https://scholar.google.com/scholar?q=Denis+Kocetkov%2C+Raymond+Li%2C+Loubna+Ben+Allal%2C+Jia+Li%2C+Chenghao+Mou%2C+Carlos+Mu%C3%B1oz+Ferrandis%2C+Yacine+Jernite%2C+Margaret+Mitchell%2C+Sean+Hughes%2C+Thomas+Wolf%2C+Dzmitry+Bahdanau%2C+Leandro+von+Werra%2C+and+Harm+d) 8. Patrick Lewis, Ethan Perez, Aleksandra Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, and Douwe Kiela. 2020. Retrieval-augmented generation for knowledge-intensive nlp tasks. In Advances in Neural Information Proc… [find](https://scholar.google.com/scholar?q=Patrick+Lewis%2C+Ethan+Perez%2C+Aleksandra+Piktus%2C+Fabio+Petroni%2C+Vladimir+Karpukhin%2C+Naman+Goyal%2C+Heinrich+K%C3%BCttler%2C+Mike+Lewis%2C+Wen%2Dtau+Yih%2C+Tim+Rockt%C3%A4schel%2C+Sebastian+Riedel%2C+and+Douwe+Kiela%2E+2020%2E+Retr) 9. Anton Lozhkov, Raymond Li, Loubna Ben Allal, Federico Cassano, Joel Lamy-Poirier, Nouamane Tazi, Ao Tang, Dmytro Pykhtar, Jiawei Liu, Yuxiang Wei, Tianyang Liu, Max Tian, Denis Kocetkov, Arthur Zucker, Younes Belkada, Zijian Wang, Qian Liu, Dmitry Abulkhanov, Indraneil Paul, Zhuang Li, Wen-Ding Li,… [find](https://scholar.google.com/scholar?q=Anton+Lozhkov%2C+Raymond+Li%2C+Loubna+Ben+Allal%2C+Federico+Cassano%2C+Joel+Lamy%2DPoirier%2C+Nouamane+Tazi%2C+Ao+Tang%2C+Dmytro+Pykhtar%2C+Jiawei+Liu%2C+Yuxiang+Wei%2C+Tianyang+Liu%2C+Max+Tian%2C+Denis+Kocetkov%2C+Arthur+Zucker) 10. Aman Madaan, Shuyan Zhou, Uri Alon, Yiming Yang, and Graham Neubig. 2022. Language models of code are few-shot commonsense learners. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 1384–1403, Abu Dhabi, United Arab Emirates. Association for Computatio… [find](https://scholar.google.com/scholar?q=Aman+Madaan%2C+Shuyan+Zhou%2C+Uri+Alon%2C+Yiming+Yang%2C+and+Graham+Neubig%2E+2022%2E+Language+models+of+code+are+few%2Dshot+commonsense+learners%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+in+Natur) 11. Alexandre Magueresse, Vincent Carles, and Evan Heetderks. 2020. Low-resource languages: A review of past work and future challenges. [find](https://scholar.google.com/scholar?q=Alexandre+Magueresse%2C+Vincent+Carles%2C+and+Evan+Heetderks%2E+2020%2E+Low%2Dresource+languages%3A+A+review+of+past+work+and+future+challenges%2E) 12. Microsoft Support. 2024. Excel functions. https://support.microsoft.com/en-gb/office/excel-functions-alphabetical-b3944572-255d-4efb-[Accessed 17-05-bb96-c6d90033e188. 2025]. [find](https://scholar.google.com/scholar?q=Microsoft+Support%2E+2024%2E+Excel+functions%2E+https%3A%2F%2Fsupport%2Emicrosoft%2Ecom%2Fen%2Dgb%2Foffice%2Fexcel%2Dfunctions%2Dalphabetical%2Db3944572%2D255d%2D4efb%2D%5BAccessed+17%2D05%2Dbb96%2Dc6d90033e188%2E+2025%5D%2E) 13. OpenAI, :, Aaron Jaech, Adam Kalai, Adam Lerer, Adam Richardson, Ahmed El-Kishky, Aiden Low, Alec Helyar, Aleksander Madry, Alex Beutel, Alex Carney, Alex Iftimie, Alex Karpenko, Alex Tachard Passos, Alexander Neitz, Alexander Prokofiev, Alexander Wei, Allison Tam, Ally Bennett, Ananya Kumar, Andre… [find](https://scholar.google.com/scholar?q=OpenAI%2C+%3A%2C+Aaron+Jaech%2C+Adam+Kalai%2C+Adam+Lerer%2C+Adam+Richardson%2C+Ahmed+El%2DKishky%2C+Aiden+Low%2C+Alec+Helyar%2C+Aleksander+Madry%2C+Alex+Beutel%2C+Alex+Carney%2C+Alex+Iftimie%2C+Alex+Karpenko%2C+Alex+Tachard+Passos%2C+) [our card](https://md.research.elamur.ai/free/papers/openai-2024-openai-o1-system-card/) 14. OpenAI, Josh Achiam, Steven Adler, Sandhini Agarwal, Lama Ahmad, Ilge Akkaya, Florencia Leoni Aleman, Diogo Almeida, Janko Altenschmidt, Sam Altman, Shyamal Anadkat, Red Avila, Igor Babuschkin, Suchir Balaji, Valerie Balcom, Paul Baltescu, Haiming Bao, Mohammad Bavarian, Jeff Belgum, Irwan Bello, Ja… [find](https://scholar.google.com/scholar?q=OpenAI%2C+Josh+Achiam%2C+Steven+Adler%2C+Sandhini+Agarwal%2C+Lama+Ahmad%2C+Ilge+Akkaya%2C+Florencia+Leoni+Aleman%2C+Diogo+Almeida%2C+Janko+Altenschmidt%2C+Sam+Altman%2C+Shyamal+Anadkat%2C+Red+Avila%2C+Igor+Babuschkin%2C+Suchir) [our card](https://md.research.elamur.ai/free/papers/openai-2023-gpt-4-technical-report/) 15. Panupong Pasupat and Percy Liang. 2015. Compositional semantic parsing on semi-structured tables. [find](https://scholar.google.com/scholar?q=Panupong+Pasupat+and+Percy+Liang%2E+2015%2E+Compositional+semantic+parsing+on+semi%2Dstructured+tables%2E) 16. Justin Payan, Swaroop Mishra, Mukul Singh, Carina Negreanu, Christian Poelitz, Chitta Baral, Subhro Roy, Rasika Chakravarthy, Benjamin Van Durme, and Elnaz Nouri. 2023. InstructExcel: A benchmark for natural language instruction in excel. In Findings of the Association for Computational Linguistics:… [find](https://scholar.google.com/scholar?q=Justin+Payan%2C+Swaroop+Mishra%2C+Mukul+Singh%2C+Carina+Negreanu%2C+Christian+Poelitz%2C+Chitta+Baral%2C+Subhro+Roy%2C+Rasika+Chakravarthy%2C+Benjamin+Van+Durme%2C+and+Elnaz+Nouri%2E+2023%2E+InstructExcel%3A+A+benchmark+for+) 17. Haritz Puerto, Martin Tutek, Somak Aditya, Xiaodan Zhu, and Iryna Gurevych. 2024. Code prompting elicits conditional reasoning abilities in Text+Code LLMs. In Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing, pages 11234–11258, Miami, Florida, USA. Association f… [find](https://scholar.google.com/scholar?q=Haritz+Puerto%2C+Martin+Tutek%2C+Somak+Aditya%2C+Xiaodan+Zhu%2C+and+Iryna+Gurevych%2E+2024%2E+Code+prompting+elicits+conditional+reasoning+abilities+in+Text%2BCode+LLMs%2E+In+Proceedings+of+the+2024+Conference+on+Emp) 18. Timo Schick, Jane Dwivedi-Yu, Roberto Dessì, Roberta Raileanu, Maria Lomeli, Luke Zettlemoyer, Nicola Cancedda, and Thomas Scialom. 2023. Toolformer: Language models can teach themselves to use tools. [find](https://scholar.google.com/scholar?q=Timo+Schick%2C+Jane+Dwivedi%2DYu%2C+Roberto+Dess%C3%AC%2C+Roberta+Raileanu%2C+Maria+Lomeli%2C+Luke+Zettlemoyer%2C+Nicola+Cancedda%2C+and+Thomas+Scialom%2E+2023%2E+Toolformer%3A+Language+models+can+teach+themselves+to+use+tools%2E) [our card](https://md.research.elamur.ai/free/papers/schick-2023-toolformer-language-models-can-teach/) 19. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… [find](https://scholar.google.com/scholar?q=Hugo+Touvron%2C+Louis+Martin%2C+Kevin+Stone%2C+Peter+Albert%2C+Amjad+Almahairi%2C+Yasmine+Babaei%2C+Nikolay+Bashlykov%2C+Soumya+Batra%2C+Prajjwal+Bhargava%2C+Shruti+Bhosale%2C+Dan+Bikel%2C+Lukas+Blecher%2C+Cristian+Canton+Fe) 20. An Yang, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chengyuan Li, Dayiheng Liu, Fei Huang, Haoran Wei, Huan Lin, Jian Yang, Jianhong Tu, Jianwei Zhang, Jianxin Yang, Jiaxi Yang, Jingren Zhou, Junyang Lin, Kai Dang, Keming Lu, Keqin Bao, Kexin Yang, Le Yu, Mei Li, Mingfeng Xue, Pei… [find](https://scholar.google.com/scholar?q=An+Yang%2C+Baosong+Yang%2C+Beichen+Zhang%2C+Binyuan+Hui%2C+Bo+Zheng%2C+Bowen+Yu%2C+Chengyuan+Li%2C+Dayiheng+Liu%2C+Fei+Huang%2C+Haoran+Wei%2C+Huan+Lin%2C+Jian+Yang%2C+Jianhong+Tu%2C+Jianwei+Zhang%2C+Jianxin+Yang%2C+Jiaxi+Yang%2C+Jin) 21. Kang Min Yoo, Dongju Park, Jaewook Kang, Sang-Woo Lee, and Woomyoung Park. 2021. GPT3Mix: Leveraging large-scale language models for text augmentation. In Findings of the Association for Computational Linguistics: EMNLP 2021, pages 2225–2239, Punta Cana, Dominican Republic. Association for Computati… [find](https://scholar.google.com/scholar?q=Kang+Min+Yoo%2C+Dongju+Park%2C+Jaewook+Kang%2C+Sang%2DWoo+Lee%2C+and+Woomyoung+Park%2E+2021%2E+GPT3Mix%3A+Leveraging+large%2Dscale+language+models+for+text+augmentation%2E+In+Findings+of+the+Association+for+Computational) [our card](https://md.research.elamur.ai/free/papers/yoo-2021-gpt3mix-leveraging-large-scale-language/) 22. Yueheng Zhang, Xiaoyuan Liu, Yiyou Sun, Atheer Alharbi, Hend Alzahrani, Basel Alomair, and Dawn Song. 2025. Can llms design good questions based on context? [find](https://scholar.google.com/scholar?q=Yueheng+Zhang%2C+Xiaoyuan+Liu%2C+Yiyou+Sun%2C+Atheer+Alharbi%2C+Hend+Alzahrani%2C+Basel+Alomair%2C+and+Dawn+Song%2E+2025%2E+Can+llms+design+good+questions+based+on+context%3F) 23. Wei Zhao, Zhitao Hou, Siyuan Wu, Yan Gao, Haoyu Dong, Yao Wan, Hongyu Zhang, Yulei Sui, and Haidong Zhang. 2024. NL2Formula: Generating spreadsheet formulas from natural language queries. In Findings of the Association for Computational Linguistics: EACL 2024, pages 2377– 2388, St. Julian’s, Malta.… [find](https://scholar.google.com/scholar?q=Wei+Zhao%2C+Zhitao+Hou%2C+Siyuan+Wu%2C+Yan+Gao%2C+Haoyu+Dong%2C+Yao+Wan%2C+Hongyu+Zhang%2C+Yulei+Sui%2C+and+Haidong+Zhang%2E+2024%2E+NL2Formula%3A+Generating+spreadsheet+formulas+from+natural+language+queries%2E+In+Findings+) 24. Fengbin Zhu, Wenqiang Lei, Youcheng Huang, Chao Wang, Shuo Zhang, Jiancheng Lv, Fuli Feng, and Tat-Seng Chua. 2021. TAT-QA: A question answering benchmark on a hybrid of tabular and textual content in finance. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics… [find](https://scholar.google.com/scholar?q=Fengbin+Zhu%2C+Wenqiang+Lei%2C+Youcheng+Huang%2C+Chao+Wang%2C+Shuo+Zhang%2C+Jiancheng+Lv%2C+Fuli+Feng%2C+and+Tat%2DSeng+Chua%2E+2021%2E+TAT%2DQA%3A+A+question+answering+benchmark+on+a+hybrid+of+tabular+and+textual+content+in) [our card](https://md.research.elamur.ai/free/papers/zhu-2021-tat-qa-question-answering-benchmark/)