---
title: "Tuning Language Models by Proxy — paper by Liu et al. (2024)"
description: "The authors propose proxy-tuning, a decoding-time technique that needs only a model's output logits… 52 references, cited by 76 papers here."
authors: ["Alisa Liu", "Xiaochuang Han", "Yizhong Wang", "Yulia Tsvetkov", "Yejin Choi", "Noah A. Smith"]
year: 2024
venue: "arXiv:2401.08565"
source: "https://arxiv.org/abs/2401.08565"
doi: "10.48550/arxiv.2401.08565"
arxiv: "2401.08565"
licence: "CC BY 4.0 (arXiv)"
licence_url: "http://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/liu-2024-tuning-language-models-proxy/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/liu-2024-tuning-language-models-proxy/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2024](https://md.research.elamur.ai/free/en/papers/2024/page/6/) / Tuning Language Models by Proxy

# Tuning Language Models by Proxy

**Authors:** [Alisa Liu](https://md.research.elamur.ai/free/en/authors/alisa-liu/), [Xiaochuang Han](https://md.research.elamur.ai/free/en/authors/xiaochuang-han/), [Yizhong Wang](https://md.research.elamur.ai/free/en/authors/yizhong-wang/), [Yulia Tsvetkov](https://md.research.elamur.ai/free/en/authors/yulia-tsvetkov/), [Yejin Choi](https://md.research.elamur.ai/free/en/authors/yejin-choi/) and [Noah A. Smith](https://md.research.elamur.ai/free/en/authors/noah-a-smith/) · **Year:** 2024 · **Venue:** arXiv:2401.08565 · **Licence:** [CC BY 4.0 (arXiv)](http://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2401.08565) · [PDF](https://arxiv.org/pdf/2401.08565) · [DOI 10.48550/arxiv.2401.08565](https://doi.org/10.48550/arxiv.2401.08565) · [arXiv 2401.08565](https://arxiv.org/abs/2401.08565) · [OpenAlex](https://openalex.org/W4390963130)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · [76](https://md.research.elamur.ai/free/en/papers/liu-2024-tuning-language-models-proxy/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Fine-tuning large language models is expensive, and it is impossible for closed models whose weights are hidden. The authors propose proxy-tuning, a decoding-time technique that needs only a model's output logits, not its parameters. A small model is fine-tuned to act as an "expert." At each generation step, the difference between its logits and those of its untuned counterpart (the "anti-expert") is added to the logits of a large base model, using the DExperts formula. The models only need to share a vocabulary, and no hyperparameters are required. Experiments use the LLaMA 2 family: LLaMA 2-7B-CHAT and LLaMA 2-7B steer the 13B and 70B base models. Evaluation covers GSM, AlpacaFarm, ToxiGen and TruthfulQA, plus code adaptation, task-specific fine-tuning and a GPT-3.5 case study. Proxy-tuning recovers most of the gain from genuine instruction tuning and sometimes exceeds it on knowledge-heavy tasks. This offers a cheap way to customize large or proprietary models, provided their output probabilities are exposed.

## Key points
- For instruction tuning, proxy-tuning closes on average 91.1% (13B) and 88.1% (70B) of the gap between LLaMA 2 base models and their LLaMA 2-CHAT versions. On AlpacaFarm, 70B rises from a 3.7% to an 88.0% win rate (CHAT: 90.4%). On GSM, it rises from 9.6% to 32.0% (CHAT: 51.8%).
- Steering LLaMA 2-13B with CodeLlama-7B as the expert yields 17–32% absolute gains on coding benchmarks over the base model.
- For task-specific fine-tuning (question answering and math), proxy-tuned LLaMA 2-70B gains 31% absolute over the untuned 70B model and 9% over the tuned 7B expert.
- With access to only the top-5 log probabilities, proxy-tuning GPT-3.5 for temporal adaptation improves its accuracy on recent-event questions by 2.3% (t-test, p \< 0.0001). The 7B (anti-)experts are weaker than GPT-3.5, making this an example of weak-to-strong generalization.
- Token-level analysis shows that the guidance mostly promotes reasoning and stylistic tokens. Code is released at github.com/alisawuffles/proxy-tuning.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Washington](https://md.research.elamur.ai/free/en/universities/university-of-washington/) — University, Seattle, United States · 402 works · 212,762 citations
- [Allen Institute for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/allen-institute-for-artificial-intelligence/) — Research institute, Seattle, United States · 304 works · 106,138 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

🧭 Topics (1): [🧠 Language models & pretraining](https://md.research.elamur.ai/free/en/topics/language-models/) · 3,894 works

## Cite

Cite as: Alisa Liu, Xiaochuang Han, Yizhong Wang, Yulia Tsvetkov, Yejin Choi and Noah A. Smith. 2024. Tuning Language Models by Proxy. arXiv:2401.08565.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{liu2024tuning,
  title = {{Tuning Language Models by Proxy}},
  author = {Alisa Liu, Xiaochuang Han, Yizhong Wang, Yulia Tsvetkov, Yejin Choi and Noah A. Smith},
  year = {2024},
  doi = {10.48550/arxiv.2401.08565},
  eprint = {2401.08565},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2401.08565}
}
```

- [← previous Model Merging in LLMs, MLLMs, and Beyond: Methods, Theories, Applications and Opportunities](https://md.research.elamur.ai/free/en/papers/yang-2024-model-merging-llms-mllms-methods/)
- [next → DeepSpeed-FastGen: High-throughput Text Generation for LLMs via MII and DeepSpeed-Inference](https://md.research.elamur.ai/free/en/papers/holmes-2024-deepspeed-fastgen-high-throughput-text/)
- [↑ back to papers of 2024](https://md.research.elamur.ai/free/en/papers/2024/page/6/)

## References

52 references, 35 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Iz Beltagy, Kyle Lo, and Arman Cohan. [SciBERT: A pretrained language model for scientific text](https://md.research.elamur.ai/free/en/papers/beltagy-2019-scibert-pretrained-language-model-scientific/). In Kentaro Inui, Jing Jiang, Vincent Ng, and Xiaojun Wan (eds.), Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natura…
2. Collin Burns, Pavel Izmailov, Jan Hendrik Kirchner, Bowen Baker, Leo Gao, Leopold Aschenbrenner, Yining Chen, Adrien Ecoffet, Manas Joglekar, Jan Leike, Ilya Sutskever, and Jeff Wu. [Weak-to-strong generalization: Eliciting strong capabilities with weak supervision](https://md.research.elamur.ai/free/en/papers/burns-2023-weak-strong-generalization-eliciting-strong/), 2023. URL https://arxiv.org/abs/23… [↗](https://arxiv.org/abs/23…)
3. Mark Chen, Jerry Tworek, Heewoo Jun, Qiming Yuan, Henrique Ponde de Oliveira Pinto, Jared Kaplan, Harri Edwards, Yuri Burda, Nicholas Joseph, Greg Brockman, Alex Ray, Raul Puri, Gretchen Krueger, Michael Petrov, Heidy Khlaaf, Girish Sastry, Pamela Mishkin, Brooke Chan, Scott Gray, Nick Ryder, Mikhai… — [Evaluating Large Language Models Trained on Code](https://md.research.elamur.ai/free/en/papers/chen-2021-evaluating-large-language-models-trained/)
4. Yung-Sung Chuang, Yujia Xie, Hongyin Luo, Yoon Kim, James Glass, and Pengcheng He. [Dola: Decoding by contrasting layers improves factuality in large language models](https://md.research.elamur.ai/free/en/papers/chuang-2023-dola-decoding-contrasting-layers-improves/), 2023. URL https://arxiv.org/abs/2309.03883. [↗](https://arxiv.org/abs/2309.03883)
5. Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, Christopher Hesse, and John Schulman. [Training verifiers to solve math word problems](https://md.research.elamur.ai/free/en/papers/cobbe-2021-training-verifiers-solve-math-word/), 2021. URL https://arxiv.org/abs/2110.14168. [↗](https://arxiv.org/abs/2110.14168)
6. Yiming Cui, Ziqing Yang, and Xin Yao. [Efficient and effective text encoding for chinese llama and alpaca](https://md.research.elamur.ai/free/en/papers/cui-2023-efficient-effective-text-encoding-chinese/), 2023. URL https://arxiv.org/abs/2304.08177. [↗](https://arxiv.org/abs/2304.08177)
7. Haikang Deng and Colin Raffel. Reward-augmented decoding: Efficient controlled text generation with a unidirectional reward model. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 11781–11791, Singapore,… [find](https://scholar.google.com/scholar?q=Haikang+Deng+and+Colin+Raffel%2E+Reward%2Daugmented+decoding%3A+Efficient+controlled+text+generation+with+a+unidirectional+reward+model%2E+In+Houda+Bouamor%2C+Juan+Pino%2C+and+Kalika+Bali+%28eds%2E%29%2C+Proceedings+of+t)
8. Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer. [QLoRA: Efficient finetuning of quantized LLMs](https://md.research.elamur.ai/free/en/papers/dettmers-2023-qlora-efficient-finetuning-quantized-llms/). In Thirty-seventh Conference on Neural Information Processing Systems, 2023. URL [https://openreview.net/forum?id=OUIFPHEgJU](https://openreview.net/forum?id=OUIFPHEgJU). [↗](https://openreview.net/forum?id=OUIFPHEgJU)
9. Zi-Yi Dou, Xinyi Wang, Junjie Hu, and Graham Neubig. Domain differential adaptation for neural machine translation. In Alexandra Birch, Andrew Finch, Hiroaki Hayashi, Ioannis Konstas, Thang Luong, Graham Neubig, Yusuke Oda, and Katsuhito Sudoh (eds.), Proceedings of the 3rd Workshop on Neural Genera… [find](https://scholar.google.com/scholar?q=Zi%2DYi+Dou%2C+Xinyi+Wang%2C+Junjie+Hu%2C+and+Graham+Neubig%2E+Domain+differential+adaptation+for+neural+machine+translation%2E+In+Alexandra+Birch%2C+Andrew+Finch%2C+Hiroaki+Hayashi%2C+Ioannis+Konstas%2C+Thang+Luong%2C+Gra)
10. Yann Dubois, Xuechen Li, Rohan Taori, Tianyi Zhang, Ishaan Gulrajani, Jimmy Ba, Carlos Guestrin, Percy Liang, and Tatsunori B. Hashimoto. [Alpacafarm: A simulation framework for methods that learn from human feedback](https://md.research.elamur.ai/free/en/papers/dubois-2023-alpacafarm-simulation-framework-methods-that/), 2023. URL https://arxiv.org/abs/2305. 14387. [↗](https://arxiv.org/abs/2305)
11. Ariel Gera, Roni Friedman, Ofir Arviv, Chulaka Gunasekara, Benjamin Sznajder, Noam Slonim, and Eyal Shnarch. The benefits of bad advice: Autocontrastive decoding across model layers. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the Associat… [find](https://scholar.google.com/scholar?q=Ariel+Gera%2C+Roni+Friedman%2C+Ofir+Arviv%2C+Chulaka+Gunasekara%2C+Benjamin+Sznajder%2C+Noam+Slonim%2C+and+Eyal+Shnarch%2E+The+benefits+of+bad+advice%3A+Autocontrastive+decoding+across+model+layers%2E+In+Anna+Rogers%2C+J)
12. Arnav Gudibande, Eric Wallace, Charlie Snell, Xinyang Geng, Hao Liu, Pieter Abbeel, Sergey Levine, and Dawn Song. [The false promise of imitating proprietary LLMs](https://md.research.elamur.ai/free/en/papers/gudibande-2023-false-promise-imitating-proprietary-llms/), 2023. URL https://arxiv.org/abs/2305.15717. [↗](https://arxiv.org/abs/2305.15717)
13. Suchin Gururangan, Ana Marasović, Swabha Swayamdipta, Kyle Lo, Iz Beltagy, Doug Downey, and Noah A. Smith. Don’t stop pretraining: Adapt language models to domains and tasks. In Dan Jurafsky, Joyce Chai, Natalie Schluter, and Joel Tetreault (eds.), Proceedings of the 58th Annual Meeting of the Asso… [find](https://scholar.google.com/scholar?q=Suchin+Gururangan%2C+Ana+Marasovic%CC%81%2C+Swabha+Swayamdipta%2C+Kyle+Lo%2C+Iz+Beltagy%2C+Doug+Downey%2C+and+Noah+A%2E+Smith%2E+Don%E2%80%99t+stop+pretraining%3A+Adapt+language+models+to+domains+and+tasks%2E+In+Dan+Jurafsky%2C+Joyce+C)
14. Xiaochuang Han. In-context alignment: Chat with vanilla language models before fine-tuning, 2023. URL https://arxiv.org/abs/2308.04275. — [arXiv:2308.04275](https://arxiv.org/abs/2308.04275) [↗](https://arxiv.org/abs/2308.04275)
15. Xiaochuang Han, Sachin Kumar, Yulia Tsvetkov, and Marjan Ghazvininejad. David helps Goliath: Inference-time collaboration between small specialized and large general diffusion lms. In Proceedings of the 2024 Conference of the North American Chapter of the Association for Computational Linguistics: H… [find](https://scholar.google.com/scholar?q=Xiaochuang+Han%2C+Sachin+Kumar%2C+Yulia+Tsvetkov%2C+and+Marjan+Ghazvininejad%2E+David+helps+Goliath%3A+Inference%2Dtime+collaboration+between+small+specialized+and+large+general+diffusion+lms%2E+In+Proceedings+of+t)
16. Thomas Hartvigsen, Saadia Gabriel, Hamid Palangi, Maarten Sap, Dipankar Ray, and Ece Kamar. [ToxiGen: A large-scale machine-generated dataset for adversarial and implicit hate speech detection](https://md.research.elamur.ai/free/en/papers/hartvigsen-2022-toxigen-large-scale-machine-generated/). In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting…
17. Ari Holtzman, Peter West, Vered Shwartz, Yejin Choi, and Luke Zettlemoyer. Surface form competition: Why the highest probability answer isn’t always right. In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 [Conference on Empirical Methods in…](https://md.research.elamur.ai/free/en/papers/holtzman-2021-surface-form-competition-why-highest/)
18. Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly. Parameter-efficient transfer learning for nlp. ArXiv, abs/1902.00751, 2019. [find](https://scholar.google.com/scholar?q=Neil+Houlsby%2C+Andrei+Giurgiu%2C+Stanislaw+Jastrzebski%2C+Bruna+Morrone%2C+Quentin+de+Laroussilhe%2C+Andrea+Gesmundo%2C+Mona+Attariyan%2C+and+Sylvain+Gelly%2E+Parameter%2Defficient+transfer+learning+for+nlp%2E+ArXiv%2C+ab)
19. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. [LoRA: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In International Conference on Learning Representations, 2022. URL [https://openreview.net/](https://openreview.net/) forum?id=nZeVKeeFYf9. [↗](https://openreview.net/)
20. Hamish Ivison, Yizhong Wang, Valentina Pyatkin, Nathan Lambert, Matthew Peters, Pradeep Dasigi, Joel Jang, David Wadden, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. [Camels in a changing climate: Enhancing lm adaptation with tulu 2](https://md.research.elamur.ai/free/en/papers/ivison-2023-camels-changing-climate-enhancing-lm/), 2023. URL https://arxiv.org/abs/2311.10702. [↗](https://arxiv.org/abs/2311.10702)
21. Mandar Joshi, Eunsol Choi, Daniel Weld, and Luke Zettlemoyer. [TriviaQA: A large scale distantly supervised challenge dataset for reading comprehension](https://md.research.elamur.ai/free/en/papers/joshi-2017-triviaqa-large-scale-distantly-supervised/). In Regina Barzilay and Min-Yen Kan (eds.), Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long P…
22. Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Alec Radford, Jeffrey Wu, and Dario Amodei. [Scaling laws for neural language models](https://md.research.elamur.ai/free/en/papers/kaplan-2020-scaling-laws-neural-language-models/), 2020. URL https://arxiv.org/abs/2001.08361. [↗](https://arxiv.org/abs/2001.08361)
23. Jungo Kasai, Keisuke Sakaguchi, Ronan Le Bras, Hao Peng, Ximing Lu, Dragomir Radev, Yejin Choi, and Noah A. Smith. Twist decoding: Diverse generators guide each other. In Yoav Goldberg, Zornitsa Kozareva, and Yue Zhang (eds.), Proceedings of the 2022 Conference on Empirical Methods in Natural Langua… [find](https://scholar.google.com/scholar?q=Jungo+Kasai%2C+Keisuke+Sakaguchi%2C+Ronan+Le+Bras%2C+Hao+Peng%2C+Ximing+Lu%2C+Dragomir+Radev%2C+Yejin+Choi%2C+and+Noah+A%2E+Smith%2E+Twist+decoding%3A+Diverse+generators+guide+each+other%2E+In+Yoav+Goldberg%2C+Zornitsa+Kozar)
24. Jungo Kasai, Keisuke Sakaguchi, yoichi takahashi, Ronan Le Bras, Akari Asai, Xinyan Velocity Yu, Dragomir Radev, Noah A. Smith, Yejin Choi, and Kentaro Inui. Realtime QA: What’s the answer right now? In Thirty-seventh Conference on Neural Information Processing Systems Datasets and Benchmarks Track,… — [RealTime QA: What's the Answer Right Now?](https://md.research.elamur.ai/free/en/papers/kasai-2022-realtime-qa-s-answer-right/)
25. Ben Krause, Akhilesh Deepak Gotmare, Bryan McCann, Nitish Shirish Keskar, Shafiq Joty, Richard Socher, and Nazneen Fatema Rajani. [GeDi: Generative discriminator guided sequence generation](https://md.research.elamur.ai/free/en/papers/krause-2021-gedi-generative-discriminator-guided-sequence/). In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Findings of the Associatio…
26. Yuhang Lai, Chengxi Li, Yiming Wang, Tianyi Zhang, Ruiqi Zhong, Luke Zettlemoyer, Scott Wen tau Yih, Daniel Fried, Sida Wang, and Tao Yu. Ds-1000: A natural and reliable benchmark for data science code generation. 2022. URL https://arxiv.org/abs/2211. 11501. [↗](https://arxiv.org/abs/2211)
27. Angeliki Lazaridou, Adhiguna Kuncoro, Elena Gribovskaya, Devang Agrawal, Adam Liska, Tayfun Terzi, Mai Gimenez, Cyprien de Masson d’Autume, Tomáš Kočiský, Sebastian Ruder, Dani Yogatama, Kris Cao, Susannah Young, and Phil Blunsom. Mind the gap: Assessing temporal generalization in neural languag… [find](https://scholar.google.com/scholar?q=Angeliki+Lazaridou%2C+Adhiguna+Kuncoro%2C+Elena+Gribovskaya%2C+Devang+Agrawal%2C+Adam+Liska%2C+Tayfun+Terzi%2C+Mai+Gimenez%2C+Cyprien+de+Masson+d%E2%80%99Autume%2C+Toma%CC%81s%CC%8C+Koc%CC%8Cisky%CC%81%2C+Sebastian+Ruder%2C+Dani+Yogatama%2C+Kris+Cao%2C)
28. Sicong Leng, Hang Zhang, Guanzheng Chen, Xin Li, Shijian Lu, Chunyan Miao, and Lidong Bing. [Mitigating object hallucinations in large vision-language models through visual contrastive decoding](https://md.research.elamur.ai/free/en/papers/leng-2023-mitigating-object-hallucinations-large-vision/), 2023. URL https://arxiv.org/abs/2311.16922. [↗](https://arxiv.org/abs/2311.16922)
29. Brian Lester, Rami Al-Rfou, and Noah Constant. [The power of scale for parameter-efficient prompt tuning](https://md.research.elamur.ai/free/en/papers/lester-2021-power-scale-parameter-efficient-prompt/). In Marie-Francine Moens, Xuanjing Huang, Lucia Specia, and Scott Wen-tau Yih (eds.), Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 3045–3059, Online…
30. Xiang Lisa Li and Percy Liang. Prefix-tuning: Optimizing continuous prompts for generation. In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natura… [find](https://scholar.google.com/scholar?q=Xiang+Lisa+Li+and+Percy+Liang%2E+Prefix%2Dtuning%3A+Optimizing+continuous+prompts+for+generation%2E+In+Chengqing+Zong%2C+Fei+Xia%2C+Wenjie+Li%2C+and+Roberto+Navigli+%28eds%2E%29%2C+Proceedings+of+the+59th+Annual+Meeting+of)
31. Xiang Lisa Li, Ari Holtzman, Daniel Fried, Percy Liang, Jason Eisner, Tatsunori Hashimoto, Luke Zettlemoyer, and Mike Lewis. Contrastive decoding: Open-ended text generation as optimization. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Proceedings of the 61st Annual Meeting of the… [find](https://scholar.google.com/scholar?q=Xiang+Lisa+Li%2C+Ari+Holtzman%2C+Daniel+Fried%2C+Percy+Liang%2C+Jason+Eisner%2C+Tatsunori+Hashimoto%2C+Luke+Zettlemoyer%2C+and+Mike+Lewis%2E+Contrastive+decoding%3A+Open%2Dended+text+generation+as+optimization%2E+In+Anna+R)
32. Bill Yuchen Lin, Abhilasha Ravichander, Ximing Lu, Nouha Dziri, Melanie Sclar, Khyathi Chandu, Chandra Bhagavatula, and Yejin Choi. [The unlocking spell on base llms: Rethinking alignment via in-context learning](https://md.research.elamur.ai/free/en/papers/lin-2023-unlocking-spell-base-llms-rethinking/). 2023. URL https://arxiv.org/abs/2312.01552. [↗](https://arxiv.org/abs/2312.01552)
33. Stephanie Lin, Jacob Hilton, and Owain Evans. [TruthfulQA: Measuring how models mimic human falsehoods](https://md.research.elamur.ai/free/en/papers/lin-2022-truthfulqa-measuring-how-models-mimic/). In Smaranda Muresan, Preslav Nakov, and Aline Villavicencio (eds.), Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 3214–3252, D…
34. Alisa Liu, Maarten Sap, Ximing Lu, Swabha Swayamdipta, Chandra Bhagavatula, Noah A. Smith, and Yejin Choi. [DExperts: Decoding-time controlled text generation with experts and anti-experts](https://md.research.elamur.ai/free/en/papers/liu-2021-dexperts-decoding-time-controlled-text/). In Chengqing Zong, Fei Xia, Wenjie Li, and Roberto Navigli (eds.), Proceedings of the 59th Annual Meeting of th…
35. Ximing Lu, Faeze Brahman, Peter West, Jaehun Jung, Khyathi Chandu, Abhilasha Ravichander, Prithviraj Ammanabrolu, Liwei Jiang, Sahana Ramnath, Nouha Dziri, Jillian Fisher, Bill Lin, Skyler Hallinan, Lianhui Qin, Xiang Ren, Sean Welleck, and Yejin Choi. Inference-time policy adapters (IPA): Tailoring… [find](https://scholar.google.com/scholar?q=Ximing+Lu%2C+Faeze+Brahman%2C+Peter+West%2C+Jaehun+Jung%2C+Khyathi+Chandu%2C+Abhilasha+Ravichander%2C+Prithviraj+Ammanabrolu%2C+Liwei+Jiang%2C+Sahana+Ramnath%2C+Nouha+Dziri%2C+Jillian+Fisher%2C+Bill+Lin%2C+Skyler+Hallinan%2C+L)
36. Kelvin Luu, Daniel Khashabi, Suchin Gururangan, Karishma Mandyam, and Noah A. Smith. [Time waits for no one! analysis and challenges of temporal misalignment](https://md.research.elamur.ai/free/en/papers/luu-2022-time-waits-no-one-analysis/). In Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz (eds.), Proceedings of the 2022 Conference of the North American C…
37. Michael McCloskey and Neal J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. volume 24 of Psychology of Learning and Motivation, pp. 109–165. Academic Press, 1989. doi: https://doi.org/10.1016/S0079-7421(08)60536-8. URL https://www.sciencedirect.com/scie… [↗](https://doi.org/10.1016/S0079-7421(08)60536-8)
38. Eric Mitchell, Rafael Rafailov, Archit Sharma, Chelsea Finn, and Christopher D Manning. [An emulator for fine-tuning large language models using small language models](https://md.research.elamur.ai/free/en/papers/mitchell-2023-emulator-fine-tuning-large-language/). In The Twelfth International Conference on Learning Representations, 2024. URL [https://openreview.net/forum?id=Eo7kv0sllr](https://openreview.net/forum?id=Eo7kv0sllr). [↗](https://openreview.net/forum?id=Eo7kv0sllr)
39. Yasumasa Onoe, Michael Zhang, Eunsol Choi, and Greg Durrett. [Entity cloze by date: What LMs know about unseen entities](https://md.research.elamur.ai/free/en/papers/onoe-2022-entity-cloze-date-what-lms/). In Marine Carpuat, Marie-Catherine de Marneffe, and Ivan Vladimir Meza Ruiz (eds.), Findings of the Association for Computational Linguistics: NAACL 2022, pp. 693–702, Seattle, Uni…
40. OpenAI. [GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/), 2023. URL https://arxiv.org/abs/2303.08774. [↗](https://arxiv.org/abs/2303.08774)
41. Aitor Ormazabal, Mikel Artetxe, and Eneko Agirre. CombLM: Adapting black-box language models through small fine-tuned models. In Houda Bouamor, Juan Pino, and Kalika Bali (eds.), Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pp. 2961–2974, Singapore, Decembe… [find](https://scholar.google.com/scholar?q=Aitor+Ormazabal%2C+Mikel+Artetxe%2C+and+Eneko+Agirre%2E+CombLM%3A+Adapting+black%2Dbox+language+models+through+small+fine%2Dtuned+models%2E+In+Houda+Bouamor%2C+Juan+Pino%2C+and+Kalika+Bali+%28eds%2E%29%2C+Proceedings+of+the+20)
42. Long Ouyang, Jeffrey Wu, Xu Jiang, Diogo Almeida, Carroll Wainwright, Pamela Mishkin, Chong Zhang, Sandhini Agarwal, Katarina Slama, Alex Gray, John Schulman, Jacob Hilton, Fraser Kelton, Luke Miller, Maddie Simens, Amanda Askell, Peter Welinder, Paul Christiano, Jan Leike, and Ryan Lowe. Training l… — [Training language models to follow instructions with human feedback](https://md.research.elamur.ai/free/en/papers/ouyang-2022-training-language-models-follow-instructions/)
43. Jonathan Pei, Kevin Yang, and Dan Klein. PREADD: Prefix-adaptive decoding for controlled text generation. In Anna Rogers, Jordan Boyd-Graber, and Naoaki Okazaki (eds.), Findings of the Association for Computational Linguistics: ACL 2023, pp. 10018–10037, Toronto, Canada, July 2023. Association for C… [find](https://scholar.google.com/scholar?q=Jonathan+Pei%2C+Kevin+Yang%2C+and+Dan+Klein%2E+PREADD%3A+Prefix%2Dadaptive+decoding+for+controlled+text+generation%2E+In+Anna+Rogers%2C+Jordan+Boyd%2DGraber%2C+and+Naoaki+Okazaki+%28eds%2E%29%2C+Findings+of+the+Association+for)
44. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of Machine Learning Research, 21(140):1–67, 2020. URL [http://jmlr.org/papers/v21/20…](http://jmlr.org/papers/v21/20…) [↗](http://jmlr.org/papers/v21/20…)
45. Baptiste Rozière, Jonas Gehring, Fabian Gloeckle, Sten Sootla, Itai Gat, Xiaoqing Ellen Tan, Yossi Adi, Jingyu Liu, Tal Remez, Jérémy Rapin, Artyom Kozhevnikov, Ivan Evtimov, Joanna Bitton, Manish Bhatt, Cristian Canton Ferrer, Aaron Grattafiori, Wenhan Xiong, Alexandre Défossez, Jade Copet, Fai… — [Code Llama: Open Foundation Models for Code](https://md.research.elamur.ai/free/en/papers/roziere-2024-code-llama-open-foundation-models/)
46. Rico Sennrich, Jannis Vamvas, and Alireza Mohammadshahi. [Mitigating hallucinations and off-target machine translation with source-contrastive and language-contrastive decoding](https://md.research.elamur.ai/free/en/papers/sennrich-2024-mitigating-hallucinations-off-target-machine/), 2023. URL https://arxiv.org/abs/2309.07098. [↗](https://arxiv.org/abs/2309.07098)
47. Weijia Shi, Xiaochuang Han, Mike Lewis, Yulia Tsvetkov, Luke Zettlemoyer, and Scott Wen tau Yih. [Trusting your evidence: Hallucinate less with context-aware decoding](https://md.research.elamur.ai/free/en/papers/shi-2024-trusting-your-evidence-hallucinate-less/), 2023. URL https://arxiv.org/abs/2305.14739. [↗](https://arxiv.org/abs/2305.14739)
48. Nisan Stiennon, Long Ouyang, Jeffrey Wu, Daniel Ziegler, Ryan Lowe, Chelsea Voss, Alec Radford, Dario Amodei, and Paul F Christiano. Learning to summarize with human feedback. In H. Larochelle, M. Ranzato, R. Hadsell, M.F. Balcan, and H. Lin (eds.), Advances in Neural Information Processing Systems,… [find](https://scholar.google.com/scholar?q=Nisan+Stiennon%2C+Long+Ouyang%2C+Jeffrey+Wu%2C+Daniel+Ziegler%2C+Ryan+Lowe%2C+Chelsea+Voss%2C+Alec+Radford%2C+Dario+Amodei%2C+and+Paul+F+Christiano%2E+Learning+to+summarize+with+human+feedback%2E+In+H%2E+Larochelle%2C+M%2E+Ran)
49. Hugo Touvron, Louis Martin, Kevin Stone, Peter Albert, Amjad Almahairi, Yasmine Babaei, Nikolay Bashlykov, Soumya Batra, Prajjwal Bhargava, Shruti Bhosale, Dan Bikel, Lukas Blecher, Cristian Canton Ferrer, Moya Chen, Guillem Cucurull, David Esiobu, Jude Fernandes, Jeremy Fu, Wenyin Fu, Brian Fuller,… — [Llama 2: Open Foundation and Fine-Tuned Chat Models](https://md.research.elamur.ai/free/en/papers/touvron-2023-llama-2-open-foundation-fine/)
50. Yizhong Wang, Hamish Ivison, Pradeep Dasigi, Jack Hessel, Tushar Khot, Khyathi Chandu, David Wadden, Kelsey MacMillan, Noah A. Smith, Iz Beltagy, and Hannaneh Hajishirzi. [How far can camels go? Exploring the state of instruction tuning on open resources](https://md.research.elamur.ai/free/en/papers/wang-2023-far-camels-go-exploring-state/). In Thirty-seventh Conference on Neural Inform…
51. Chaoyi Wu, Weixiong Lin, Xiaoman Zhang, Ya Zhang, Yanfeng Wang, and Weidi Xie. [Pmc-llama: Towards building open-source language models for medicine](https://md.research.elamur.ai/free/en/papers/wu-2023-pmc-llama-towards-building-open/), 2023. URL https://arxiv.org/abs/2304.14454. [↗](https://arxiv.org/abs/2304.14454)
52. Kevin Yang and Dan Klein. [FUDGE: Controlled text generation with future discriminators](https://md.research.elamur.ai/free/en/papers/yang-2021-fudge-controlled-text-generation-future/). In Kristina Toutanova, Anna Rumshisky, Luke Zettlemoyer, Dilek Hakkani-Tur, Iz Beltagy, Steven Bethard, Ryan Cotterell, Tanmoy Chakraborty, and Yichao Zhou (eds.), Proceedings of the 2021 Conference of the North…

*Updated 2 November 2026*