---
title: "Compositional Exemplars for In-context Learning — Ye 2023"
description: "In-context learning (ICL) lets a frozen language model solve a new task from a few input–output… 61 references, cited by 96 papers here."
authors: ["Jiacheng Ye", "Zhiyong Wu", "Jiangtao Feng", "Tao Yu", "Lingpeng Kong"]
year: 2023
venue: "arXiv:2302.05698"
source: "https://arxiv.org/abs/2302.05698"
doi: "10.48550/arxiv.2302.05698"
arxiv: "2302.05698"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ye-2023-compositional-exemplars-context-learning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ye-2023-compositional-exemplars-context-learning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/) / Compositional Exemplars for In-context Learning

# Compositional Exemplars for In-context Learning

**Authors:** [Jiacheng Ye](https://md.research.elamur.ai/free/en/authors/jiacheng-ye/), [Zhiyong Wu](https://md.research.elamur.ai/free/en/authors/zhiyong-wu/), [Jiangtao Feng](https://md.research.elamur.ai/free/en/authors/jiangtao-feng/), [Tao Yu](https://md.research.elamur.ai/free/en/authors/tao-yu/) and [Lingpeng Kong](https://md.research.elamur.ai/free/en/authors/lingpeng-kong/) · **Year:** 2023 · **Venue:** arXiv:2302.05698 · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2302.05698) · [PDF](https://arxiv.org/pdf/2302.05698) · [DOI 10.48550/arxiv.2302.05698](https://doi.org/10.48550/arxiv.2302.05698) · [arXiv 2302.05698](https://arxiv.org/abs/2302.05698) · [OpenAlex](https://openalex.org/W4320839455)

**Citations:** 24 citations (OpenAlex, 2026-10-14) · [96](https://md.research.elamur.ai/free/en/papers/ye-2023-compositional-exemplars-context-learning/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
In-context learning (ICL) lets a frozen language model solve a new task from a few input–output demonstrations, but results swing widely depending on which demonstrations go into the prompt. Earlier selectors scored each candidate against the test input on its own, so they often picked near-duplicate examples. Ye et al. treat the choice of demonstrations as a subset selection problem and introduce CEIL. CEIL uses a conditional Determinantal Point Process (DPP) whose kernel rewards both relevance to the query and diversity within the set. Two BERT-based encoders produce the embeddings. They are trained with a contrastive objective so that subsets the LM scores higher also receive higher DPP probability. At inference, a fast greedy MAP algorithm picks the final set from KNN-prefiltered candidates. Across 12 datasets covering 7 classification and generation tasks, CEIL outperforms both heuristic and learned retrievers. It also transfers across LMs and datasets and handles compositional tasks well. This makes it a practical way to adapt black-box LMs without fine-tuning them.

## Key points
- The benchmark covers 12 datasets across 7 task types: sentiment analysis, paraphrase detection, NLI, commonsense reasoning, open-domain QA, code generation and semantic parsing. Code is available at github.com/HKUNLP/icl-ceil (ICML 2023).
- In the kernel, a weight λ sets the balance between relevance and diversity. Diversity mattered more for harder tasks (NLI, semantic parsing), while relevance mattered more for sentiment analysis.
- Inference first narrows the pool to n candidates with a KNN retriever. It then runs the exact greedy MAP algorithm of Chen et al. (2018), which costs O(nK²) for K examples.
- CEIL is most useful with small prompts. With only 4 examples it mostly beats EPR using 32, and with 1 example it mostly beats TopK-BERT using 32. This matters given context limits such as 1,024 tokens in GPT2-XL and 2,048 in GPT2-Neo.
- A retriever trained once can be reused for other LMs and datasets, so the approach can be applied efficiently without retraining for each new setting.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu and Lingpeng Kong. 2023. Compositional Exemplars for In-context Learning. arXiv:2302.05698.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{ye2023compositional,
  title = {{Compositional Exemplars for In-context Learning}},
  author = {Jiacheng Ye, Zhiyong Wu, Jiangtao Feng, Tao Yu and Lingpeng Kong},
  year = {2023},
  doi = {10.48550/arxiv.2302.05698},
  eprint = {2302.05698},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2302.05698}
}
```

- [← previous Automated evaluation of written discourse coherence using GPT-4](https://md.research.elamur.ai/free/en/papers/naismith-2023-automated-evaluation-written-discourse-coherence/)
- [next → Knowledge of cultural moral norms in large language models](https://md.research.elamur.ai/free/en/papers/ramezani-2023-knowledge-cultural-moral-norms-large/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/7/)

## References

61 references, 33 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Agrawal, S., Zhou, C., Lewis, M., Zettlemoyer, L., and Ghazvininejad, M. [In-context examples selection for machine translation](https://md.research.elamur.ai/free/en/papers/agrawal-2023-context-examples-selection-machine-translation/). arXiv preprint arXiv:2212.02437, 2022.
2. An, C., Feng, J., Lv, K., Kong, L., Qiu, X., and Huang, X. Cont: Contrastive neural text generation. NeurIPS, 2022. [find](https://scholar.google.com/scholar?q=An%2C+C%2E%2C+Feng%2C+J%2E%2C+Lv%2C+K%2E%2C+Kong%2C+L%2E%2C+Qiu%2C+X%2E%2C+and+Huang%2C+X%2E+Cont%3A+Contrastive+neural+text+generation%2E+NeurIPS%2C+2022%2E)
3. Andreas, J., Bufe, J., Burkett, D., Chen Jr, C., Clausman, J., Crawford, J., Crim, K., DeLoach, J., Dorner, L., Eisner, J., et al. [Task-oriented dialogue as dataflow synthesis](https://md.research.elamur.ai/free/en/papers/andreas-2020-task-oriented-dialogue-as-dataflow/). Transactions of the Association for Computational Linguistics, 8:556–571, 2020.
4. Azadi, S., Feng, J., and Darrell, T. Learning detection with diverse proposals. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7149– 7157, 2017. [find](https://scholar.google.com/scholar?q=Azadi%2C+S%2E%2C+Feng%2C+J%2E%2C+and+Darrell%2C+T%2E+Learning+detection+with+diverse+proposals%2E+In+Proceedings+of+the+IEEE+Conference+on+Computer+Vision+and+Pattern+Recognition%2C+pp%2E+7149%E2%80%93+7157%2C+2017%2E)
5. Berant, J., Chou, A., Frostig, R., and Liang, P. [Semantic parsing on Freebase from question-answer pairs](https://md.research.elamur.ai/free/en/papers/berant-2013-semantic-parsing-freebase-question-answer/). In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1533–1544, Seattle, Washington, USA, October 2013. Association for Computational Linguistics. URL h…
6. Black, S., Gao, L., Wang, P., Leahy, C., and Biderman, S. GPT-Neo: Large Scale Autoregressive Language Modeling with Mesh-Tensorflow, March 2021. URL https://doi.org/10.5281/zenodo.5297715. — [doi:10.5281/zenodo.5297715](https://doi.org/10.5281/zenodo.5297715) [↗](https://doi.org/10.5281/zenodo.5297715)
7. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Ch… [find](https://scholar.google.com/scholar?q=Brown%2C+T%2E+B%2E%2C+Mann%2C+B%2E%2C+Ryder%2C+N%2E%2C+Subbiah%2C+M%2E%2C+Kaplan%2C+J%2E%2C+Dhariwal%2C+P%2E%2C+Neelakantan%2C+A%2E%2C+Shyam%2C+P%2E%2C+Sastry%2C+G%2E%2C+Askell%2C+A%2E%2C+Agarwal%2C+S%2E%2C+Herbert%2DVoss%2C+A%2E%2C+Krueger%2C+G%2E%2C+Henighan%2C+T%2E%2C+Child%2C+R%2E%2C+Rames)
8. Systems 2020, [NeurIPS 2020, December 6-12, 2020, 00027](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). virtual, 2020. URL [https://proceedings.neurips.cc/paper/2020/hash/](https://proceedings.neurips.cc/paper/2020/hash/) Gao, T., Yao, X., and Chen, D. Simcse: Simple contrastive 1457c0d6bfcb4967418bfb8ac142f64a-Abstract. learning of sentence embeddings. In Proceedings of html. the 2021 Conferenc… [↗](https://proceedings.neurips.cc/paper/2020/hash/)
9. Dolan, W. B., Quirk, C., and Brockett, C. [Unsupervised construction of large paraphrase corpora: Exploiting massively parallel news sources](https://md.research.elamur.ai/free/en/papers/dolan-2004-unsupervised-construction-large-paraphrase-corpora/). In COLING 2004: Proceedings of the 20th International Conference on Computational Linguistics, pp. 350–356, 2004.
10. FAIR, Bakhtin, A., Brown, N., Dinan, E., Farina, G., Flaherty, C., Fried, D., Goff, A., Gray, J., Hu, H., et al. Human-level play in the game of diplomacy by combining language models with strategic reasoning. Science (New York, NY), 378(6624):1067–1074, 2022. [find](https://scholar.google.com/scholar?q=FAIR%2C+Bakhtin%2C+A%2E%2C+Brown%2C+N%2E%2C+Dinan%2C+E%2E%2C+Farina%2C+G%2E%2C+Flaherty%2C+C%2E%2C+Fried%2C+D%2E%2C+Goff%2C+A%2E%2C+Gray%2C+J%2E%2C+Hu%2C+H%2E%2C+et+al%2E+Human%2Dlevel+play+in+the+game+of+diplomacy+by+combining+language+models+with+strategic+r)
11. Finegan-Dollak, C., Kummerfeld, J. K., Zhang, L., Ramanathan, K., Sadasivam, S., Zhang, R., and Radev, D. [Improving text-to-SQL evaluation methodology](https://md.research.elamur.ai/free/en/papers/dollak-2018-improving-text-sql-evaluation-methodology/). In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 351–360, Melbourne, Australi…
12. International Conference on Machine Learning, pp. 1384– 1393. PMLR, 2017. [find](https://scholar.google.com/scholar?q=International+Conference+on+Machine+Learning%2C+pp%2E+1384%E2%80%93+1393%2E+PMLR%2C+2017%2E)
13. Hasson, M. and Berant, J. Question decomposition with dependency graphs. In 3rd Conference on Automated Knowledge Base Construction, 2021. [find](https://scholar.google.com/scholar?q=Hasson%2C+M%2E+and+Berant%2C+J%2E+Question+decomposition+with+dependency+graphs%2E+In+3rd+Conference+on+Automated+Knowledge+Base+Construction%2C+2021%2E)
14. He, K., Fan, H., Wu, Y., Xie, S., and Girshick, R. [Momentum contrast for unsupervised visual representation learning](https://md.research.elamur.ai/free/en/papers/he-2019-momentum-contrast-unsupervised-visual-representation/). In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 9729–9738, 2020.
15. Heilbron, F. C., Escorcia, V., Ghanem, B., and Niebles, J. C. Activitynet: A large-scale video benchmark for human activity understanding. In 2015 IEEE conference on computer vision and pattern recognition (CVPR), pp. 961–970. [find](https://scholar.google.com/scholar?q=Heilbron%2C+F%2E+C%2E%2C+Escorcia%2C+V%2E%2C+Ghanem%2C+B%2E%2C+and+Niebles%2C+J%2E+C%2E+Activitynet%3A+A+large%2Dscale+video+benchmark+for+human+activity+understanding%2E+In+2015+IEEE+conference+on+computer+vision+and+pattern+recogn)
16. Holtzman, A., Buys, J., Du, L., Forbes, M., and Choi, Y. [The curious case of neural text degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/). In International Conference on Learning Representations, 2019.
17. Izacard, G., Caron, M., Hosseini, L., Riedel, S., Bojanowski, P., Joulin, A., and Grave, E. Towards [unsupervised dense information retrieval with contrastive learning](https://md.research.elamur.ai/free/en/papers/izacard-2021-unsupervised-dense-information-retrieval-contrastive/). arXiv preprint arXiv:2112.09118, 2021.
18. Karpukhin, V., Oguz, B., Min, S., Lewis, P., Wu, L., Edunov, S., Chen, D., and Yih, W.-t. [Dense passage retrieval for open-domain question answering](https://md.research.elamur.ai/free/en/papers/karpukhin-2020-dense-passage-retrieval-open-domain/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 6769–6781, 2020.
19. Kingma, D. P. and Ba, J. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). In Proceedings of ICLR, 2015.
20. Ko, C.-W., Lee, J., and Queyranne, M. An exact algorithm for maximum entropy sampling. Operations Research, 43(4):684–691, 1995. [find](https://scholar.google.com/scholar?q=Ko%2C+C%2E%2DW%2E%2C+Lee%2C+J%2E%2C+and+Queyranne%2C+M%2E+An+exact+algorithm+for+maximum+entropy+sampling%2E+Operations+Research%2C+43%284%29%3A684%E2%80%93691%2C+1995%2E)
21. Kulesza, A. and Taskar, B. k-dpps: Fixed-size determinantal point processes. In ICML, 2011. [find](https://scholar.google.com/scholar?q=Kulesza%2C+A%2E+and+Taskar%2C+B%2E+k%2Ddpps%3A+Fixed%2Dsize+determinantal+point+processes%2E+In+ICML%2C+2011%2E)
22. Kulesza, A., Taskar, B., et al. Determinantal point processes for machine learning. Foundations and Trends® in Machine Learning, 5(2–3):123–286, 2012. [find](https://scholar.google.com/scholar?q=Kulesza%2C+A%2E%2C+Taskar%2C+B%2E%2C+et+al%2E+Determinantal+point+processes+for+machine+learning%2E+Foundations+and+Trends%C2%AE+in+Machine+Learning%2C+5%282%E2%80%933%29%3A123%E2%80%93286%2C+2012%2E)
23. Kulis, B. et al. Metric learning: A survey. Foundations and Trends® in Machine Learning, 5(4):287–364, 2013. [find](https://scholar.google.com/scholar?q=Kulis%2C+B%2E+et+al%2E+Metric+learning%3A+A+survey%2E+Foundations+and+Trends%C2%AE+in+Machine+Learning%2C+5%284%29%3A287%E2%80%93364%2C+2013%2E)
24. Levy, I., Bogin, B., and Berant, J. [Diverse demonstrations improve in-context compositional generalization](https://md.research.elamur.ai/free/en/papers/levy-2023-diverse-demonstrations-improve-context-compositional/). arXiv preprint arXiv:2212.06800, 2022.
25. Li, H., Arora, A., Chen, S., Gupta, A., Gupta, S., and Mehdad, Y. [Mtop: A comprehensive multilingual task-oriented semantic parsing benchmark](https://md.research.elamur.ai/free/en/papers/li-2021-mtop-comprehensive-multilingual-task-oriented/). In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 2950–2962, 2021.
26. Li, Y., Lin, Z., Zhang, S., Fu, Q., Chen, B., Lou, J.-G., and Chen, W. On the advance of making language models better reasoners. arXiv preprint arXiv:2206.02336, 2022. — [arXiv:2206.02336](https://arxiv.org/abs/2206.02336) [find](https://scholar.google.com/scholar?q=Li%2C+Y%2E%2C+Lin%2C+Z%2E%2C+Zhang%2C+S%2E%2C+Fu%2C+Q%2E%2C+Chen%2C+B%2E%2C+Lou%2C+J%2E%2DG%2E%2C+and+Chen%2C+W%2E+On+the+advance+of+making+language+models+better+reasoners%2E+arXiv+preprint+arXiv%3A2206%2E02336%2C+2022%2E+%E2%80%94+%5BarXiv%3A2206%2E02336%5D%28https%3A%2F%2Far)
27. Lin, X. V., Wang, C., Zettlemoyer, L., and Ernst, M. D. Nl2bash: A corpus and semantic parser for natural language interface to the linux operating system. In Proceedings of the Eleventh International Conference on Language Resources and Evaluation LREC 2018, Miyazaki (Japan), 7-12 May, 2018., 2018. [find](https://scholar.google.com/scholar?q=Lin%2C+X%2E+V%2E%2C+Wang%2C+C%2E%2C+Zettlemoyer%2C+L%2E%2C+and+Ernst%2C+M%2E+D%2E+Nl2bash%3A+A+corpus+and+semantic+parser+for+natural+language+interface+to+the+linux+operating+system%2E+In+Proceedings+of+the+Eleventh+International)
28. Liu, J., Shen, D., Zhang, Y., Dolan, W. B., Carin, L., and Chen, W. [What makes good in-context examples for gpt-3?](https://md.research.elamur.ai/free/en/papers/liu-2022-what-makes-good-context-examples/) In Proceedings of Deep Learning Inside Out (DeeLIO 2022): The 3rd Workshop on Knowledge Extraction and Integration for Deep Learning Architectures, pp. 100– 114, 2022.
29. Liu, T.-Y. et al. Learning to rank for information retrieval. Foundations and Trends® in Information Retrieval, 3(3): 225–331, 2009. [find](https://scholar.google.com/scholar?q=Liu%2C+T%2E%2DY%2E+et+al%2E+Learning+to+rank+for+information+retrieval%2E+Foundations+and+Trends%C2%AE+in+Information+Retrieval%2C+3%283%29%3A+225%E2%80%93331%2C+2009%2E)
30. Lu, Y., Bartolo, M., Moore, A., Riedel, S., and Stenetorp, P. [Fantastically ordered prompts and where to find them: Overcoming few-shot prompt order sensitivity](https://md.research.elamur.ai/free/en/papers/lu-2022-fantastically-ordered-prompts-where-find/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 8086–8098, 2022.
31. Nakano, R., Hilton, J., Balaji, S., Wu, J., Ouyang, L., Kim, C., Hesse, C., Jain, S., Kosaraju, V., Saunders, W., et al. [Webgpt: Browser-assisted question-answering with human feedback](https://md.research.elamur.ai/free/en/papers/nakano-2021-webgpt-browser-assisted-question-answering/). arXiv preprint arXiv:2112.09332, 2021.
32. Oord, A. v. d., Li, Y., and Vinyals, O. [Representation learning with contrastive predictive coding](https://md.research.elamur.ai/free/en/papers/oord-2018-representation-learning-contrastive-predictive-coding/). arXiv preprint arXiv:1807.03748, 2018.
33. OpenAI, T. Chatgpt: Optimizing language models for dialogue. OpenAI, 2022. [find](https://scholar.google.com/scholar?q=OpenAI%2C+T%2E+Chatgpt%3A+Optimizing+language+models+for+dialogue%2E+OpenAI%2C+2022%2E)
34. Qiu, L., Shaw, P., Pasupat, P., Nowak, P., Linzen, T., Sha, F., and Toutanova, K. [Improving compositional generalization with latent structure and data augmentation](https://md.research.elamur.ai/free/en/papers/qiu-2022-improving-compositional-generalization-latent-structure/). In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language T…
35. Qiu, L., Shaw, P., Pasupat, P., Shi, T., Herzig, J., Pitler, E., Sha, F., and Toutanova, K. Evaluating the impact of model scale for compositional generalization in semantic parsing. arXiv preprint arXiv:2205.12253, 2022b. [find](https://scholar.google.com/scholar?q=Qiu%2C+L%2E%2C+Shaw%2C+P%2E%2C+Pasupat%2C+P%2E%2C+Shi%2C+T%2E%2C+Herzig%2C+J%2E%2C+Pitler%2C+E%2E%2C+Sha%2C+F%2E%2C+and+Toutanova%2C+K%2E+Evaluating+the+impact+of+model+scale+for+compositional+generalization+in+semantic+parsing%2E+arXiv+preprint+ar)
36. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. Language models are unsupervised multitask learners. 2019. [find](https://scholar.google.com/scholar?q=Radford%2C+A%2E%2C+Wu%2C+J%2E%2C+Child%2C+R%2E%2C+Luan%2C+D%2E%2C+Amodei%2C+D%2E%2C+and+Sutskever%2C+I%2E+Language+models+are+unsupervised+multitask+learners%2E+2019%2E)
37. Robertson, S. and Zaragoza, H. The probabilistic relevance framework: Bm25 and beyond. Foundations and Trends in Information Retrieval, 3:333–389, 01 2009. doi: 10.1561/1500000019. [find](https://scholar.google.com/scholar?q=Robertson%2C+S%2E+and+Zaragoza%2C+H%2E+The+probabilistic+relevance+framework%3A+Bm25+and+beyond%2E+Foundations+and+Trends+in+Information+Retrieval%2C+3%3A333%E2%80%93389%2C+01+2009%2E+doi%3A+10%2E1561%2F1500000019%2E)
38. Rohrbach, A., Torabi, A., Rohrbach, M., Tandon, N., Pal, C., Larochelle, H., Courville, A., and Schiele, B. Movie description. International Journal of Computer Vision, 123:94–120, 2017. [find](https://scholar.google.com/scholar?q=Rohrbach%2C+A%2E%2C+Torabi%2C+A%2E%2C+Rohrbach%2C+M%2E%2C+Tandon%2C+N%2E%2C+Pal%2C+C%2E%2C+Larochelle%2C+H%2E%2C+Courville%2C+A%2E%2C+and+Schiele%2C+B%2E+Movie+description%2E+International+Journal+of+Computer+Vision%2C+123%3A94%E2%80%93120%2C+2017%2E)
39. Rubin, O., Herzig, J., and Berant, J. [Learning to retrieve prompts for in-context learning](https://md.research.elamur.ai/free/en/papers/rubin-2021-learning-retrieve-prompts-context-learning/). In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 2655–2671, Seattle, United States, July 2022. Association…
40. Schölkopf, B., Smola, A. J., Bach, F., et al. Learning with kernels: support vector machines, regularization, optimization, and beyond. MIT press, 2002. [find](https://scholar.google.com/scholar?q=Scho%CC%88lkopf%2C+B%2E%2C+Smola%2C+A%2E+J%2E%2C+Bach%2C+F%2E%2C+et+al%2E+Learning+with+kernels%3A+support+vector+machines%2C+regularization%2C+optimization%2C+and+beyond%2E+MIT+press%2C+2002%2E)
41. Shaw, P., Chang, M.-W., Pasupat, P., and Toutanova, K. Compositional generalization and natural language variation: Can a semantic parsing approach handle both? In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on N… [find](https://scholar.google.com/scholar?q=Shaw%2C+P%2E%2C+Chang%2C+M%2E%2DW%2E%2C+Pasupat%2C+P%2E%2C+and+Toutanova%2C+K%2E+Compositional+generalization+and+natural+language+variation%3A+Can+a+semantic+parsing+approach+handle+both%3F+In+Proceedings+of+the+59th+Annual+Meeti)
42. Shi, F., Fried, D., Ghazvininejad, M., Zettlemoyer, L., and Wang, S. I. [Natural language to code translation with execution](https://md.research.elamur.ai/free/en/papers/shi-2022-natural-language-code-translation-execution/). arXiv preprint arXiv:2204.11454, 2022.
43. Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A. Y., and Potts, C. [Recursive deep models for semantic compositionality over a sentiment treebank](https://md.research.elamur.ai/free/en/papers/socher-2013-recursive-deep-models-semantic-compositionality/). In Proceedings of the 2013 conference on empirical methods in natural language processing, pp. 1631–1642, 2013.
44. Su, H., Kasai, J., Wu, C. H., Shi, W., Wang, T., Xin, J., Zhang, R., Ostendorf, M., Zettlemoyer, L., Smith, N. A., et al. Selective annotation makes language models better few-shot learners. arXiv preprint arXiv:2209.01975, 2022. [find](https://scholar.google.com/scholar?q=Su%2C+H%2E%2C+Kasai%2C+J%2E%2C+Wu%2C+C%2E+H%2E%2C+Shi%2C+W%2E%2C+Wang%2C+T%2E%2C+Xin%2C+J%2E%2C+Zhang%2C+R%2E%2C+Ostendorf%2C+M%2E%2C+Zettlemoyer%2C+L%2E%2C+Smith%2C+N%2E+A%2E%2C+et+al%2E+Selective+annotation+makes+language+models+better+few%2Dshot+learners%2E+arXiv+pre)
45. Talmor, A., Herzig, J., Lourie, N., and Berant, J. [CommonsenseQA: A question answering challenge targeting commonsense knowledge](https://md.research.elamur.ai/free/en/papers/talmor-2019-commonsenseqa-question-answering-challenge-targeting/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Shor…
46. Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A. N., Kaiser, Ł., and Polosukhin, I. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). Advances in neural information processing systems, 30, 2017.
47. Wang, A., Singh, A., Michael, J., Hill, F., Levy, O., and Bowman, S. [Glue: A multi-task benchmark and analysis platform for natural language understanding](https://md.research.elamur.ai/free/en/papers/wang-2018-glue-multi-task-benchmark-analysis/). In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pp. 353–355, 2018.
48. Wang, X., Wei, J., Schuurmans, D., Le, Q., Chi, E., and Zhou, D. [Self-consistency improves chain of thought reasoning in language models](https://md.research.elamur.ai/free/en/papers/wang-2022-self-consistency-improves-chain-thought/). arXiv preprint arXiv:2203.11171, 2022.
49. Wei, J., Tay, Y., Bommasani, R., Raffel, C., Zoph, B., Borgeaud, S., Yogatama, D., Bosma, M., Zhou, D., Metzler, D., et al. [Emergent abilities of large language models](https://md.research.elamur.ai/free/en/papers/wei-2022-emergent-abilities-large-language-models/). Transactions on Machine Learning Research, 2022.
50. Williams, A., Nangia, N., and Bowman, S. [A broad-coverage challenge corpus for sentence understanding through inference](https://md.research.elamur.ai/free/en/papers/williams-2018-broad-coverage-challenge-corpus-sentence/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pp. 1112…
51. Wolf, T., Debut, L., Sanh, V., Chaumond, J., Delangue, C., Moi, A., Cistac, P., Rault, T., Louf, R., Funtowicz, M., Davison, J., Shleifer, S., von Platen, P., Ma, C., Jernite, Y., Plu, J., Xu, C., Scao, T. L., Gugger, S., Drame, M., Lhoest, Q., and Rush, A. M. Transformers: State-ofthe-art natural l… — [Transformers: State-of-the-Art Natural Language Processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/)
52. Wolfson, T., Geva, M., Gupta, A., Gardner, M., Goldberg, Y., Deutch, D., and Berant, J. [Break it down: A question understanding benchmark](https://md.research.elamur.ai/free/en/papers/wolfson-2020-break-it-down-question-understanding/). Transactions of the Association for Computational Linguistics, 8:183–198, 2020.
53. Wu, Z., Wang, Y., Ye, J., and Kong, L. Self-adaptive in-context learning. arXiv preprint arXiv:2212.[10375, 2022](https://md.research.elamur.ai/free/en/papers/wu-2022-self-adaptive-context-learning-information/).
54. Xie, P., Salakhutdinov, R., Mou, L., and Xing, E. P. Deep determinantal point process for large-scale multi-label classification. In Proceedings of the IEEE International Conference on Computer Vision (ICCV), Oct 2017. [find](https://scholar.google.com/scholar?q=Xie%2C+P%2E%2C+Salakhutdinov%2C+R%2E%2C+Mou%2C+L%2E%2C+and+Xing%2C+E%2E+P%2E+Deep+determinantal+point+process+for+large%2Dscale+multi%2Dlabel+classification%2E+In+Proceedings+of+the+IEEE+International+Conference+on+Computer+Vision)
55. Ye, J., Gao, J., Wu, Z., Feng, J., Yu, T., and Kong, L. ProGen: Progressive zero-shot dataset generation via in-context feedback. In Findings of the Association for Computational Linguistics: EMNLP 2022, pp. 3671–3683, Abu Dhabi, United Arab Emirates, December 2022a. Association for Computational Li… [find](https://scholar.google.com/scholar?q=Ye%2C+J%2E%2C+Gao%2C+J%2E%2C+Wu%2C+Z%2E%2C+Feng%2C+J%2E%2C+Yu%2C+T%2E%2C+and+Kong%2C+L%2E+ProGen%3A+Progressive+zero%2Dshot+dataset+generation+via+in%2Dcontext+feedback%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+20)
56. Ye, J., Li, C., Kong, L., and Yu, T. Generating data for symbolic language with large language models. 2023. [find](https://scholar.google.com/scholar?q=Ye%2C+J%2E%2C+Li%2C+C%2E%2C+Kong%2C+L%2E%2C+and+Yu%2C+T%2E+Generating+data+for+symbolic+language+with+large+language+models%2E+2023%2E)
57. Ye, X., Iyer, S., Celikyilmaz, A., Stoyanov, V., Durrett, G., and Pasunuru, R. [Complementary explanations for effective in-context learning](https://md.research.elamur.ai/free/en/papers/ye-2023-complementary-explanations-effective-context-learning/). arXiv preprint arXiv:2211.13892, 2022b.
58. Yin, P., Fang, H., Neubig, G., Pauls, A., Platanios, E. A., Su, Y., Thomson, S., and Andreas, J. Compositional generalization for neural semantic parsing via span-level supervised attention. In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Ling… [find](https://scholar.google.com/scholar?q=Yin%2C+P%2E%2C+Fang%2C+H%2E%2C+Neubig%2C+G%2E%2C+Pauls%2C+A%2E%2C+Platanios%2C+E%2E+A%2E%2C+Su%2C+Y%2E%2C+Thomson%2C+S%2E%2C+and+Andreas%2C+J%2E+Compositional+generalization+for+neural+semantic+parsing+via+span%2Dlevel+supervised+attention%2E+In+Procee)
59. Zelle, J. M. and Mooney, R. J. [Learning to parse database queries using inductive logic programming](https://md.research.elamur.ai/free/en/papers/zelle-1996-learning-parse-database-queries-using/). In AAAI/IAAI, pp. 1050–1055, Portland, OR, August 1996. AAAI Press/MIT Press. URL [http://www.cs.utexas.edu/users/ai-lab?zelle:aaai96](http://www.cs.utexas.edu/users/ai-lab?zelle:aaai96). [↗](http://www.cs.utexas.edu/users/ai-lab?zelle:aaai96)
60. Zellers, R., Holtzman, A., Bisk, Y., Farhadi, A., and Choi, Y. Hellaswag: Can a machine really finish your sentence? In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, 2019. [find](https://scholar.google.com/scholar?q=Zellers%2C+R%2E%2C+Holtzman%2C+A%2E%2C+Bisk%2C+Y%2E%2C+Farhadi%2C+A%2E%2C+and+Choi%2C+Y%2E+Hellaswag%3A+Can+a+machine+really+finish+your+sentence%3F+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for+Computational+Ling)
61. Zhong, M., Liu, P., Chen, Y., Wang, D., Qiu, X., and Huang, X. [Extractive summarization as text matching](https://md.research.elamur.ai/free/en/papers/zhong-2020-extractive-summarization-as-text-matching/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6197–6208, Online, July 2020. Association for Computational Linguistics. doi: 10.18653/v1/2020.acl…

*Updated 2 November 2026*