---
title: "Learning from Near-Misses — paper by Shuai et al. (2026)"
description: "The authors therefore define a deterministic 9-type error taxonomy. 75.32% EM. 25 references."
authors: ["Zhihao Shuai", "Yiyun Chen", "Maolin Ma", "Yutong Chen", "Hanjia Qiu", "Jing Xu", "Ziye Chen", "Weikai Yang"]
year: 2026
venue: "Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)"
source: "https://aclanthology.org/2026.acl-long.1368/"
doi: "10.18653/v1/2026.acl-long.1368"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/shuai-2026-learning-near-misses-error-aware/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/shuai-2026-learning-near-misses-error-aware/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/) / Learning from Near-Misses: Error-Aware Contrastive Few-Shot Learning for NL2Formula

# Learning from Near-Misses: Error-Aware Contrastive Few-Shot Learning for NL2Formula

**Authors:** Zhihao Shuai, Yiyun Chen, Maolin Ma, Yutong Chen, Hanjia Qiu, [Jing Xu](https://md.research.elamur.ai/free/en/authors/jing-xu/), Ziye Chen and Weikai Yang · **Year:** 2026 · **Venue:** [Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)](https://md.research.elamur.ai/free/en/venues/acl/) ([ACL 2026](https://md.research.elamur.ai/free/en/venues/acl/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.acl-long.1368/) · [PDF](https://aclanthology.org/2026.acl-long.1368.pdf) · [DOI 10.18653/v1/2026.acl-long.1368](https://doi.org/10.18653/v1/2026.acl-long.1368) · [OpenAlex](https://openalex.org/W7166847644)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
The paper addresses NL2Formula, the task of turning natural-language requests into Excel formulas. Its focus is "near-miss" outputs: formulas that parse correctly but give wrong results because one function, operator, reference or value is wrong. An error analysis showed these mistakes cluster around a few structural decision points in the formula's AST.

The authors therefore define a deterministic 9-type error taxonomy. It uses a skeleton-first comparison: function and operator nodes are checked before leaf-level references and values. On top of this they build ECFL (Error-Aware Contrastive Few-Shot Learning):
- **Error bank:** Monte Carlo sampling collects the model's wrong predictions, each typed and paired with its correct formula.
- **Training:** LoRA fine-tuning uses contrastive prompts with demonstrations matched by error type.
- **Inference:** a lightweight predictor guesses which error types are likely and retrieves one demonstration per type before a single decoding pass.

The authors also clean the NL2Formula benchmark and release a new expert-built benchmark, TFBench. ECFL gains 6.4 EM points over SFT and reaches SC@5-level accuracy at much lower cost. This suggests typed error supervision can replace expensive multi-sample decoding.

## Key points
- The taxonomy has 6 skeleton-level types (Func/Op × Mismatch/Missing/Redundant), 2 fill-level types (RefMismatch, ValMismatch) and a Miscellaneous class. Of roughly 10,000 errors, only 19 involved missing or redundant references or values.
- TFBench is expert-curated and covers 215 functions and 12 operators, including nested compositions.
- In the ablation on NL2Formula, full ECFL reaches 75.32% EM. Using demonstrations only in training gives 68.52%, and only at inference gives 62.71%, which is below the SFT baseline.
- Retrieval by predicted error type beats both similarity-only and fixed-type retrieval. ECFL also improves Llama-3.1-8B-Instruct, and error banks transferred to Qwen3-8B from another model family still beat SFT.
- Code and data are available at github.com/VICA-Lab-HKUST-GZ/ECFL.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University College London](https://md.research.elamur.ai/free/en/universities/university-college-london/) — University, London, United Kingdom · 88 works · 19,432 citations
- [The Hong Kong University of Science and Technology (Guangzhou)](https://md.research.elamur.ai/free/en/universities/the-hong-kong-university-of-science-and-technology-guangzhou/) — University, Guangzhou, China · 53 works · 1,367 citations
- [Southwest University](https://md.research.elamur.ai/free/en/universities/southwest-university/) — University, Chongqing, China · 2 works · 19 citations

📚 Shelves (5): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [🧮 Computational Theory and Mathematics](https://md.research.elamur.ai/free/en/shelves/computer-science/computational-theory-and-mathematics/) · 252 works

## Cite

Cite as: Zhihao Shuai, Yiyun Chen, Maolin Ma, Yutong Chen, Hanjia Qiu, Jing Xu, Ziye Chen and Weikai Yang. 2026. Learning from Near-Misses: Error-Aware Contrastive Few-Shot Learning for NL2Formula. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{shuai2026learning,
  title = {{Learning from Near-Misses: Error-Aware Contrastive Few-Shot Learning for NL2Formula}},
  author = {Zhihao Shuai, Yiyun Chen, Maolin Ma, Yutong Chen, Hanjia Qiu, Jing Xu, Ziye Chen and Weikai Yang},
  booktitle = {Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)},
  year = {2026},
  doi = {10.18653/v1/2026.acl-long.1368},
  url = {https://doi.org/10.18653/v1/2026.acl-long.1368}
}
```

- [← previous Learning from Emptiness: De-biasing Listwise Rerankers with Content-Agnostic Probability Calibration](https://md.research.elamur.ai/free/en/papers/lv-2026-learning-emptiness-de-biasing-listwise/)
- [next → Learning Invariant Modality Representation for Robust Multimodal Learning from a Causal Inference Perspective](https://md.research.elamur.ai/free/en/papers/mai-2026-learning-invariant-modality-representation-robust/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/34/)

## References

25 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Efthimia Aivaloglou, David Hoepelman, and Felienne Hermans. 2017. Parsing excel formulas: A grammar and its application on 4 large datasets. Journal of Software: Evolution and Process, 29(12):e1895. [find](https://scholar.google.com/scholar?q=Efthimia+Aivaloglou%2C+David+Hoepelman%2C+and+Felienne+Hermans%2E+2017%2E+Parsing+excel+formulas%3A+A+grammar+and+its+application+on+4+large+datasets%2E+Journal+of+Software%3A+Evolution+and+Process%2C+29%2812%29%3Ae1895%2E)
2. Qing Chang and Zhiming Hu. 2026. Gazeinterpreter: Parsing eye gaze to generate eye-body-coordinated narrations. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 17410– 17418. [find](https://scholar.google.com/scholar?q=Qing+Chang+and+Zhiming+Hu%2E+2026%2E+Gazeinterpreter%3A+Parsing+eye+gaze+to+generate+eye%2Dbody%2Dcoordinated+narrations%2E+In+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+volume+40%2C+pages+17410)
3. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, and 39 others. 202… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
4. Sibei Chen, Yeye He, Weiwei Cui, Ju Fan, Song Ge, Haidong Zhang, Dongmei Zhang, and Surajit Chaudhuri. 2024. Auto-formula: Recommend formulas in spreadsheets using contrastive learning for table representations. Proceedings of the ACM on Management of Data, 2(3):1–27. [find](https://scholar.google.com/scholar?q=Sibei+Chen%2C+Yeye+He%2C+Weiwei+Cui%2C+Ju+Fan%2C+Song+Ge%2C+Haidong+Zhang%2C+Dongmei+Zhang%2C+and+Surajit+Chaudhuri%2E+2024%2E+Auto%2Dformula%3A+Recommend+formulas+in+spreadsheets+using+contrastive+learning+for+table+repre)
5. Xinyun Chen, Maxwell Lin, Nathanael Schärli, and Denny Zhou. 2023. Teaching large language models to self-debug. Preprint, arXiv:2304.05128. [find](https://scholar.google.com/scholar?q=Xinyun+Chen%2C+Maxwell+Lin%2C+Nathanael+Sch%C3%A4rli%2C+and+Denny+Zhou%2E+2023%2E+Teaching+large+language+models+to+self%2Ddebug%2E+Preprint%2C+arXiv%3A2304%2E05128%2E)
6. Xinyun Chen, Petros Maniatis, Rishabh Singh, Charles Sutton, Hanjun Dai, Max Lin, and Denny Zhou. 2021b. Spreadsheetcoder: Formula prediction from semi-structured context. In International Conference on Machine Learning, pages 1661–1672. PMLR. [find](https://scholar.google.com/scholar?q=Xinyun+Chen%2C+Petros+Maniatis%2C+Rishabh+Singh%2C+Charles+Sutton%2C+Hanjun+Dai%2C+Max+Lin%2C+and+Denny+Zhou%2E+2021b%2E+Spreadsheetcoder%3A+Formula+prediction+from+semi%2Dstructured+context%2E+In+International+Conference+)
7. Tianyu Gao, Xingcheng Yao, and Danqi Chen. 2021. [Simcse: Simple contrastive learning of sentence embeddings](https://md.research.elamur.ai/free/en/papers/gao-2021-simcse-simple-contrastive-learning-sentence/). In Proceedings of the 2021 conference on empirical methods in natural language processing, pages 6894–6910.
8. Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2021. [Lora: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). Preprint, arXiv:2106.09685.
9. Harshit Joshi, Abishai Ebenezer, José Cambronero Sanchez, Sumit Gulwani, Aditya Kanade, Vu Le, Ivan Radiček, and Gust Verbruggen. 2024. Flame: a small language model for spreadsheet formulas. In Proceedings of the Thirty-Eighth AAAI Conference on Artificial Intelligence and Thirty-Sixth Conference… [find](https://scholar.google.com/scholar?q=Harshit+Joshi%2C+Abishai+Ebenezer%2C+Jos%C3%A9+Cambronero+Sanchez%2C+Sumit+Gulwani%2C+Aditya+Kanade%2C+Vu+Le%2C+Ivan+Radic%CC%8Cek%2C+and+Gust+Verbruggen%2E+2024%2E+Flame%3A+a+small+language+model+for+spreadsheet+formulas%2E+In+Proc)
10. Boyan Li, Jiayi Zhang, Ju Fan, Yanwei Xu, Chong Chen, Nan Tang, and Yuyu Luo. 2025. Alpha-SQL: Zero-shot text-to-SQL using monte carlo tree search. In Forty-second International Conference on Machine Learning. [find](https://scholar.google.com/scholar?q=Boyan+Li%2C+Jiayi+Zhang%2C+Ju+Fan%2C+Yanwei+Xu%2C+Chong+Chen%2C+Nan+Tang%2C+and+Yuyu+Luo%2E+2025%2E+Alpha%2DSQL%3A+Zero%2Dshot+text%2Dto%2DSQL+using+monte+carlo+tree+search%2E+In+Forty%2Dsecond+International+Conference+on+Machine+)
11. Tianqi Luo, Chuhan Huang, Leixian Shen, Boyan Li, Shuyu Shen, Wei Zeng, Nan Tang, and Yuyu Luo. 2025. nvbench 2.0: Resolving ambiguity in text-to-visualization through stepwise reasoning. Preprint, arXiv:2503.12880. — [arXiv:2503.12880](https://arxiv.org/abs/2503.12880) [find](https://scholar.google.com/scholar?q=Tianqi+Luo%2C+Chuhan+Huang%2C+Leixian+Shen%2C+Boyan+Li%2C+Shuyu+Shen%2C+Wei+Zeng%2C+Nan+Tang%2C+and+Yuyu+Luo%2E+2025%2E+nvbench+2%2E0%3A+Resolving+ambiguity+in+text%2Dto%2Dvisualization+through+stepwise+reasoning%2E+Preprint%2C+ar)
12. Ansong Ni, Srini Iyer, Dragomir Radev, Veselin Stoyanov, Wen-tau Yih, Sida Wang, and Xi Victoria Lin. 2023. [Lever: Learning to verify language-to-code generation with execution](https://md.research.elamur.ai/free/en/papers/ni-2023-lever-learning-verify-language-code/). In International Conference on Machine Learning, pages 26106–26128. PMLR.
13. Stephen G Powell, Kenneth R Baker, and Barry Lawson. 2008. A critical review of the literature on spreadsheet errors. Decision Support Systems, 46(1):128– 138. [find](https://scholar.google.com/scholar?q=Stephen+G+Powell%2C+Kenneth+R+Baker%2C+and+Barry+Lawson%2E+2008%2E+A+critical+review+of+the+literature+on+spreadsheet+errors%2E+Decision+Support+Systems%2C+46%281%29%3A128%E2%80%93+138%2E)
14. Rafael Rafailov, Archit Sharma, Eric Mitchell, Stefano Ermon, Christopher D Manning, and Chelsea Finn. 2023. [Direct preference optimization: your language model is secretly a reward model](https://md.research.elamur.ai/free/en/papers/rafailov-2023-direct-preference-optimization-language-model/). In Proceedings of the 37th International Conference on Neural Information Processing Systems, pages 53728–53741…
15. Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Romain Sauvestre, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, a… — [Code Llama: Open Foundation Models for Code](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/)
16. Freda Shi, Daniel Fried, Marjan Ghazvininejad, Luke Zettlemoyer, and Sida I Wang. 2022. Natural language to code translation with execution. In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 3533–3546. [find](https://scholar.google.com/scholar?q=Freda+Shi%2C+Daniel+Fried%2C+Marjan+Ghazvininejad%2C+Luke+Zettlemoyer%2C+and+Sida+I+Wang%2E+2022%2E+Natural+language+to+code+translation+with+execution%2E+In+Proceedings+of+the+2022+Conference+on+Empirical+Methods+)
17. Zhihao Shuai, Boyan Li, Siyu Yan, Yuyu Luo, and Weikai Yang. 2026. Deepvis: Bridging natural language and data visualization through step-wise reasoning. IEEE Transactions on Visualization and Computer Graphics, 32(1):868–878. [find](https://scholar.google.com/scholar?q=Zhihao+Shuai%2C+Boyan+Li%2C+Siyu+Yan%2C+Yuyu+Luo%2C+and+Weikai+Yang%2E+2026%2E+Deepvis%3A+Bridging+natural+language+and+data+visualization+through+step%2Dwise+reasoning%2E+IEEE+Transactions+on+Visualization+and+Compute)
18. Sruti Srinivasa Ragavan, Zhitao Hou, Yun Wang, Andrew D Gordon, Haidong Zhang, and Dongmei Zhang. 2022. Gridbook: Natural language formulas for the spreadsheet grid. In Proceedings of the 27th International Conference on Intelligent User Interfaces, pages 345–368. [find](https://scholar.google.com/scholar?q=Sruti+Srinivasa+Ragavan%2C+Zhitao+Hou%2C+Yun+Wang%2C+Andrew+D+Gordon%2C+Haidong+Zhang%2C+and+Dongmei+Zhang%2E+2022%2E+Gridbook%3A+Natural+language+formulas+for+the+spreadsheet+grid%2E+In+Proceedings+of+the+27th+Interna)
19. Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou. 2020. Minilm: deep self-attention distillation for task-agnostic compression of pre-trained transformers. In Proceedings of the 34th International Conference on Neural Information Processing Systems, pages 5776–5788. [find](https://scholar.google.com/scholar?q=Wenhui+Wang%2C+Furu+Wei%2C+Li+Dong%2C+Hangbo+Bao%2C+Nan+Yang%2C+and+Ming+Zhou%2E+2020%2E+Minilm%3A+deep+self%2Dattention+distillation+for+task%2Dagnostic+compression+of+pre%2Dtrained+transformers%2E+In+Proceedings+of+the+34t)
20. Xuezhi Wang, Jason Wei, Dale Schuurmans, Quoc Le, Ed Chi, Sharan Narang, Aakanksha Chowdhery, and Denny Zhou. 2023. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). Preprint, arXiv:2203.11171.
21. Yue Wang, Weishi Wang, Shafiq Joty, and Steven CH Hoi. 2021. Codet5: Identifier-aware unified pre-trained encoder-decoder models for code understanding and generation. In Proceedings of the 2021 conference on empirical methods in natural language processing, pages 8696–8708. [find](https://scholar.google.com/scholar?q=Yue+Wang%2C+Weishi+Wang%2C+Shafiq+Joty%2C+and+Steven+CH+Hoi%2E+2021%2E+Codet5%3A+Identifier%2Daware+unified+pre%2Dtrained+encoder%2Ddecoder+models+for+code+understanding+and+generation%2E+In+Proceedings+of+the+2021+confe)
22. Shunyu Yao, Dian Yu, Jeffrey Zhao, Izhak Shafran, Thomas L Griffiths, Yuan Cao, and Karthik Narasimhan. 2023. Tree of thoughts: deliberate problem solving with large language models. In Proceedings of the 37th International Conference on Neural Information Processing Systems, pages 11809–11822. [find](https://scholar.google.com/scholar?q=Shunyu+Yao%2C+Dian+Yu%2C+Jeffrey+Zhao%2C+Izhak+Shafran%2C+Thomas+L+Griffiths%2C+Yuan+Cao%2C+and+Karthik+Narasimhan%2E+2023%2E+Tree+of+thoughts%3A+deliberate+problem+solving+with+large+language+models%2E+In+Proceedings+of)
23. Haocheng Yuan, Jing Xu, Hao Pan, Adrien Bousseau, Niloy J Mitra, and Changjian Li. 2024. CADTalk: An algorithm and benchmark for semantic commenting of cad programs. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pages 3753–3762. [find](https://scholar.google.com/scholar?q=Haocheng+Yuan%2C+Jing+Xu%2C+Hao+Pan%2C+Adrien+Bousseau%2C+Niloy+J+Mitra%2C+and+Changjian+Li%2E+2024%2E+CADTalk%3A+An+algorithm+and+benchmark+for+semantic+commenting+of+cad+programs%2E+In+Proceedings+of+the+IEEE%2FCVF+Con)
24. Kechi Zhang, Zhuo Li, Jia Li, Ge Li, and Zhi Jin. 2023. Self-edit: Fault-aware code editor for code generation. In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 769–787. [find](https://scholar.google.com/scholar?q=Kechi+Zhang%2C+Zhuo+Li%2C+Jia+Li%2C+Ge+Li%2C+and+Zhi+Jin%2E+2023%2E+Self%2Dedit%3A+Fault%2Daware+code+editor+for+code+generation%2E+In+Proceedings+of+the+61st+Annual+Meeting+of+the+Association+for+Computational+Linguisti)
25. Wei Zhao, Zhitao Hou, Siyuan Wu, Yan Gao, Haoyu Dong, Yao Wan, Hongyu Zhang, Yulei Sui, and Haidong Zhang. 2024. Nl2formula: Generating spreadsheet formulas from natural language queries. In Findings of the Association for Computational Linguistics: EACL 2024, pages 2377–2388. [find](https://scholar.google.com/scholar?q=Wei+Zhao%2C+Zhitao+Hou%2C+Siyuan+Wu%2C+Yan+Gao%2C+Haoyu+Dong%2C+Yao+Wan%2C+Hongyu+Zhang%2C+Yulei+Sui%2C+and+Haidong+Zhang%2E+2024%2E+Nl2formula%3A+Generating+spreadsheet+formulas+from+natural+language+queries%2E+In+Findings+)

*Updated 1 November 2026*