---
title: "Hard Prompts Made Easy — paper by Wen et al. (2023)"
description: "The authors introduce PEZ, a gradient-based procedure that learns discrete token… 70.93 accuracy. 37 references, cited by 81 papers here."
authors: ["Yuxin Wen", "Neel Jain", "John Kirchenbauer", "Micah Goldblum", "Jonas Geiping", "Tom Goldstein"]
year: 2023
venue: "arXiv:2302.03668"
source: "https://arxiv.org/abs/2302.03668"
doi: "10.48550/arxiv.2302.03668"
arxiv: "2302.03668"
licence: "arXiv — non-exclusive license to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/wen-2023-hard-prompts-made-easy-gradient/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/wen-2023-hard-prompts-made-easy-gradient/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/6/) / Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery

# Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery

**Authors:** [Yuxin Wen](https://md.research.elamur.ai/free/en/authors/yuxin-wen/), [Neel Jain](https://md.research.elamur.ai/free/en/authors/neel-jain/), [John Kirchenbauer](https://md.research.elamur.ai/free/en/authors/john-kirchenbauer/), [Micah Goldblum](https://md.research.elamur.ai/free/en/authors/micah-goldblum/), [Jonas Geiping](https://md.research.elamur.ai/free/en/authors/jonas-geiping/) and [Tom Goldstein](https://md.research.elamur.ai/free/en/authors/tom-goldstein/) · **Year:** 2023 · **Venue:** [arXiv:2302.03668](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/) ([NeurIPS 2023](https://md.research.elamur.ai/free/en/venues/neural-information-processing-systems/2023/)) · **Licence:** [arXiv — non-exclusive license to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/2302.03668) · [PDF](https://arxiv.org/pdf/2302.03668) · [DOI 10.48550/arxiv.2302.03668](https://doi.org/10.48550/arxiv.2302.03668) · [arXiv 2302.03668](https://arxiv.org/abs/2302.03668) · [OpenAlex](https://openalex.org/W7133225282)

**Citations:** 60 citations (OpenAlex, 2026-10-14) · [81](https://md.research.elamur.ai/free/en/papers/wen-2023-hard-prompts-made-easy-gradient/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Prompts are the main way to steer large generative models. Readable ("hard") prompts are usually written by hand. Learned continuous ("soft") prompts are opaque, tied to a single model, and unusable through text-only APIs. The authors introduce PEZ, a gradient-based procedure that learns discrete token prompts. It borrows an idea from training quantized networks: keep a continuous copy of the prompt embeddings, snap them to the nearest vocabulary embeddings for each forward pass, compute the gradient at the snapped point, and apply it to the continuous copy. The final prompt is the projected result. For images, PEZ optimizes CLIP's image–text cosine similarity. This finds prompts describing a target image without backpropagating through a diffusion model, and those prompts can then drive Stable Diffusion. For text, PEZ learns classification prompts for language models such as GPT-2 Large. The learned prompts match or beat existing discrete optimizers and transfer across models. They also show that word-level content filters can be evaded, which matters for both prompt engineering and API safety.

## Key points
- **Baselines:** PEZ is compared with AutoPrompt (SGD variant) and FluentPrompt. Both project embeddings to the nearest tokens after each update, which can stall optimization and demands careful learning-rate tuning.
- **Text classification:** On AGNEWS, PEZ outperforms the other methods by roughly 2%, and it is comparable on sentiment analysis. Prompts trained on GPT-2 Large with a fluency constraint transfer best, giving about a 14% gain over the bare template on OPT-6.7B.
- **Few-shot AGNEWS (GPT-2 Large, 100 seeds, three appended tokens, λ = 0.03):**
  - k=2: the fluent PEZ prompt reaches 70.93 accuracy, versus 58.34 for the empty template and 74.92 for a soft prompt.
  - k=4: the fluent PEZ prompt reaches 74.15, versus 79.93 for a soft prompt.
  - Each run takes about 5 minutes. Some discovered prompts are readable, such as ones mentioning "BBC" or blogs.
- **Safety:** An optimized prompt reproduced the banned "Afghan girl" image on Midjourney without using the blocked word. The authors argue that only feature-based detectors are likely to stop such workarounds.
- **Code:** Available at github.com/YuxinWenRick/hard-prompts-made-easy.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 17,450 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 17,296 works

## Cite

Cite as: Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Goldblum, Jonas Geiping and Tom Goldstein. 2023. Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery. arXiv:2302.03668.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{wen2023hard,
  title = {{Hard Prompts Made Easy: Gradient-Based Discrete Optimization for Prompt Tuning and Discovery}},
  author = {Yuxin Wen, Neel Jain, John Kirchenbauer, Micah Goldblum, Jonas Geiping and Tom Goldstein},
  year = {2023},
  doi = {10.48550/arxiv.2302.03668},
  eprint = {2302.03668},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.48550/arxiv.2302.03668}
}
```

- [← previous Exploring the Limits of ChatGPT for Query or Aspect-based Text Summarization](https://md.research.elamur.ai/free/en/papers/yang-2023-exploring-limits-chatgpt-query-aspect/)
- [next → MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback](https://md.research.elamur.ai/free/en/papers/wang-2023-mint-evaluating-llms-multi-turn/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/6/)

## References

37 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Brown, T. B., Mann, B., Ryder, N., Subbiah, M., Kaplan, J., Dhariwal, P., Neelakantan, A., Shyam, P., Sastry, G., Askell, A., Agarwal, S., Herbert-Voss, A., Krueger, G., Henighan, T., Child, R., Ramesh, A., Ziegler, D. M., Wu, J., Winter, C., Hesse, C., Chen, M., Sigler, E., Litwin, M., Gray, S., Ch… — [Language Models are Few-Shot Learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/)
2. Cherti, M., Beaumont, R., Wightman, R., Wortsman, M., Ilharco, G., Gordon, C., Schuhmann, C., Schmidt, L., and Jitsev, J. [Reproducible scaling laws for contrastive language-image learning](https://md.research.elamur.ai/free/en/papers/cherti-2022-reproducible-scaling-laws-contrastive-language/). arXiv preprint arXiv:2212.07143, 2022.
3. Chung, H. W., Hou, L., Longpre, S., Zoph, B., Tay, Y., Fedus, W., Li, E., Wang, X., Dehghani, M., Brahma, S., et al. Scaling instruction-finetuned language models. arXiv preprint arXiv:2210.11416, 2022. [find](https://scholar.google.com/scholar?q=Chung%2C+H%2E+W%2E%2C+Hou%2C+L%2E%2C+Longpre%2C+S%2E%2C+Zoph%2C+B%2E%2C+Tay%2C+Y%2E%2C+Fedus%2C+W%2E%2C+Li%2C+E%2E%2C+Wang%2C+X%2E%2C+Dehghani%2C+M%2E%2C+Brahma%2C+S%2E%2C+et+al%2E+Scaling+instruction%2Dfinetuned+language+models%2E+arXiv+preprint+arXiv%3A2210%2E11416%2C+202)
4. Courbariaux, M., Bengio, Y., and David, J.-P. [Binaryconnect: Training deep neural networks with binary weights during propagations](https://md.research.elamur.ai/free/en/papers/courbariaux-2015-binaryconnect-training-deep-neural-networks/). Advances in neural information processing systems, 28, 2015.
5. Courbariaux, M., Hubara, I., Soudry, D., El-Yaniv, R., and Bengio, Y. Binarized neural networks: Training deep neural networks with weights and activations constrained to+ 1 or-1. arXiv preprint arXiv:1602.[02830, 2016](https://md.research.elamur.ai/free/en/papers/courbariaux-2016-binarized-neural-networks-training-deep/).
6. Deng, M., Wang, J., Hsieh, C.-P., Wang, Y., Guo, H., Shu, T., Song, M., Xing, E. P., and Hu, Z. [Rlprompt: Optimizing discrete text prompts with reinforcement learning](https://md.research.elamur.ai/free/en/papers/deng-2022-rlprompt-optimizing-discrete-text-prompts/). ArXiv, abs/2205.12548, 2022.
7. Ding, N., Hu, S., Zhao, W., Chen, Y., Liu, Z., Zheng, H., and Sun, M. [OpenPrompt: An open-source framework for prompt-learning](https://md.research.elamur.ai/free/en/papers/ding-2022-openprompt-open-source-framework-prompt/). In Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics: System Demonstrations, pp. 105–113, Dublin, Ireland, May 2022. Association for C…
8. Ebrahimi, J., Rao, A., Lowd, D., and Dou, D. Hot-Flip: White-box adversarial examples for text classification. In Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pp. 31–36, Melbourne, Australia, July 2018. [Association for Computationa…](https://md.research.elamur.ai/free/en/papers/ebrahimi-2018-hotflip-white-box-adversarial-examples/)
9. Gal, R., Alaluf, Y., Atzmon, Y., Patashnik, O., Bermano, A. H., Chechik, G., and Cohen-Or, D. An image is worth one word: Personalizing text-to-image generation using textual inversion. arXiv preprint arXiv:2208.01618, 2022. [find](https://scholar.google.com/scholar?q=Gal%2C+R%2E%2C+Alaluf%2C+Y%2E%2C+Atzmon%2C+Y%2E%2C+Patashnik%2C+O%2E%2C+Bermano%2C+A%2E+H%2E%2C+Chechik%2C+G%2E%2C+and+Cohen%2DOr%2C+D%2E+An+image+is+worth+one+word%3A+Personalizing+text%2Dto%2Dimage+generation+using+textual+inversion%2E+arXiv+preprint)
10. Hu, X., Gan, Z., Wang, J., Yang, Z., Liu, Z., Lu, Y., and Wang, L. Scaling up vision-language pre-training for image captioning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 17980–17989, 2022. [find](https://scholar.google.com/scholar?q=Hu%2C+X%2E%2C+Gan%2C+Z%2E%2C+Wang%2C+J%2E%2C+Yang%2C+Z%2E%2C+Liu%2C+Z%2E%2C+Lu%2C+Y%2E%2C+and+Wang%2C+L%2E+Scaling+up+vision%2Dlanguage+pre%2Dtraining+for+image+captioning%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Computer+Vision+and+Patter)
11. Khashabi, D., Lyu, X., Min, S., Qin, L., Richardson, K., Welleck, S., Hajishirzi, H., Khot, T., Sabharwal, A., Singh, S., and Choi, Y. Prompt waywardness: The curious case of discretized interpretation of continuous prompts. In Proceedings of the 2022 Conference of the North American Chapter of the… [find](https://scholar.google.com/scholar?q=Khashabi%2C+D%2E%2C+Lyu%2C+X%2E%2C+Min%2C+S%2E%2C+Qin%2C+L%2E%2C+Richardson%2C+K%2E%2C+Welleck%2C+S%2E%2C+Hajishirzi%2C+H%2E%2C+Khot%2C+T%2E%2C+Sabharwal%2C+A%2E%2C+Singh%2C+S%2E%2C+and+Choi%2C+Y%2E+Prompt+waywardness%3A+The+curious+case+of+discretized+interpretatio)
12. Kumar, S., Paria, B., and Tsvetkov, Y. Gradient-based constrained sampling from language models. arXiv preprint arXiv:2205.12558, 2022. [find](https://scholar.google.com/scholar?q=Kumar%2C+S%2E%2C+Paria%2C+B%2E%2C+and+Tsvetkov%2C+Y%2E+Gradient%2Dbased+constrained+sampling+from+language+models%2E+arXiv+preprint+arXiv%3A2205%2E12558%2C+2022%2E)
13. Lester, B., Al-Rfou, R., and Constant, N. [The power of scale for parameter-efficient prompt tuning](https://md.research.elamur.ai/free/en/papers/lester-2021-power-scale-parameter-efficient-prompt/). In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, pp. 3045–3059, Online and Punta Cana, Dominican Republic, November 2021a. Association for Computational Lingu…
14. Lester, B., Al-Rfou, R., and Constant, N. [The Power of Scale for Parameter-Efficient Prompt Tuning](https://md.research.elamur.ai/free/en/papers/lester-2021-power-scale-parameter-efficient-prompt/). arXiv:2104.08691 [cs], September 2021b. URL http://arxiv.org/abs/2104.08691. [↗](http://arxiv.org/abs/2104.08691)
15. Li, H., De, S., Xu, Z., Studer, C., Samet, H., and Goldstein, T. Training quantized nets: A deeper understanding. Advances in Neural Information Processing Systems, 30, 2017. [find](https://scholar.google.com/scholar?q=Li%2C+H%2E%2C+De%2C+S%2E%2C+Xu%2C+Z%2E%2C+Studer%2C+C%2E%2C+Samet%2C+H%2E%2C+and+Goldstein%2C+T%2E+Training+quantized+nets%3A+A+deeper+understanding%2E+Advances+in+Neural+Information+Processing+Systems%2C+30%2C+2017%2E)
16. Li, J., Li, D., Xiong, C., and Hoi, S. [Blip: Bootstrapping language-image pre-training for unified vision-language understanding and generation](https://md.research.elamur.ai/free/en/papers/li-2022-blip-bootstrapping-language-image-pre/). arXiv preprint arXiv:2201.12086, 2022.
17. Li, X. L. and Liang, P. Prefix-tuning: Optimizing continuous prompts for generation. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pp. 4582–4597, 2021. [find](https://scholar.google.com/scholar?q=Li%2C+X%2E+L%2E+and+Liang%2C+P%2E+Prefix%2Dtuning%3A+Optimizing+continuous+prompts+for+generation%2E+In+Proceedings+of+the+59th+Annual+Meeting+of+the+Association+for+Computational+Linguistics+and+the+11th+Internation)
18. Lin, T.-Y., Maire, M., Belongie, S., Hays, J., Perona, P., Ramanan, D., Dollár, P., and Zitnick, C. L. [Microsoft coco: Common objects in context](https://md.research.elamur.ai/free/en/papers/lin-2014-microsoft-coco-common-objects-context/). In European conference on computer vision, pp. 740–755. Springer, 2014.
19. Liu, Z., Luo, P., Wang, X., and Tang, X. Deep learning face attributes in the wild. In Proceedings of the IEEE international conference on computer vision, pp. 3730– 3738, 2015. [find](https://scholar.google.com/scholar?q=Liu%2C+Z%2E%2C+Luo%2C+P%2E%2C+Wang%2C+X%2E%2C+and+Tang%2C+X%2E+Deep+learning+face+attributes+in+the+wild%2E+In+Proceedings+of+the+IEEE+international+conference+on+computer+vision%2C+pp%2E+3730%E2%80%93+3738%2C+2015%2E)
20. Loshchilov, I. and Hutter, F. Decoupled weight decay regularization. arXiv preprint arXiv:1711.05101, 2017. [find](https://scholar.google.com/scholar?q=Loshchilov%2C+I%2E+and+Hutter%2C+F%2E+Decoupled+weight+decay+regularization%2E+arXiv+preprint+arXiv%3A1711%2E05101%2C+2017%2E)
21. McAuley, J. and Leskovec, J. Hidden factors and hidden topics: Understanding rating dimensions with review text. In Proceedings of the 7th ACM Conference on Recommender Systems, RecSys ’13, pp. 165–172, New York, NY, USA, 2013. Association for Computing Machinery. ISBN 9781450324090. doi: 10.1145/25… [find](https://scholar.google.com/scholar?q=McAuley%2C+J%2E+and+Leskovec%2C+J%2E+Hidden+factors+and+hidden+topics%3A+Understanding+rating+dimensions+with+review+text%2E+In+Proceedings+of+the+7th+ACM+Conference+on+Recommender+Systems%2C+RecSys+%E2%80%9913%2C+pp%2E+165%E2%80%9317)
22. Prasad, A., Hase, P., Zhou, X., and Bansal, M. [Grips: Gradient-free, edit-based instruction search for prompting large language models](https://md.research.elamur.ai/free/en/papers/prasad-2023-grips-gradient-free-edit-based/). arXiv preprint arXiv:2203.07281, 2022.
23. Radford, A., Wu, J., Child, R., Luan, D., Amodei, D., and Sutskever, I. Language Models are Unsupervised Multi-task Learners. OpenAI, pp. 24, 2019. [find](https://scholar.google.com/scholar?q=Radford%2C+A%2E%2C+Wu%2C+J%2E%2C+Child%2C+R%2E%2C+Luan%2C+D%2E%2C+Amodei%2C+D%2E%2C+and+Sutskever%2C+I%2E+Language+Models+are+Unsupervised+Multi%2Dtask+Learners%2E+OpenAI%2C+pp%2E+24%2C+2019%2E)
24. Radford, A., Kim, J. W., Hallacy, C., Ramesh, A., Goh, G., Agarwal, S., Sastry, G., Askell, A., Mishkin, P., Clark, J., et al. [Learning transferable visual models from natural language supervision](https://md.research.elamur.ai/free/en/papers/radford-2021-learning-transferable-visual-models-natural/). In International conference on machine learning, pp. 8748–8763. PMLR, 2021.
25. Rando, J., Paleka, D., Lindner, D., Heim, L., and Tramèr, F. Red-teaming the stable diffusion safety filter. ArXiv, abs/2210.04610, 2022. — [arXiv:2210.04610](https://arxiv.org/abs/2210.04610) [find](https://scholar.google.com/scholar?q=Rando%2C+J%2E%2C+Paleka%2C+D%2E%2C+Lindner%2C+D%2E%2C+Heim%2C+L%2E%2C+and+Trame%CC%80r%2C+F%2E+Red%2Dteaming+the+stable+diffusion+safety+filter%2E+ArXiv%2C+abs%2F2210%2E04610%2C+2022%2E+%E2%80%94+%5BarXiv%3A2210%2E04610%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2210%2E04610%29)
26. Rastegari, M., Ordonez, V., Redmon, J., and Farhadi, A. [Xnor-net: Imagenet classification using binary convolutional neural networks](https://md.research.elamur.ai/free/en/papers/rastegari-2016-xnor-net-imagenet-classification-using/). In European conference on computer vision, pp. 525–542. Springer, 2016.
27. Rombach, R., Blattmann, A., Lorenz, D., Esser, P., and Ommer, B. [High-resolution image synthesis with latent diffusion models](https://md.research.elamur.ai/free/en/papers/rombach-2021-high-resolution-image-synthesis-latent/). In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 10684–10695, 2022.
28. Sanh, V., Webson, A., Raffel, C., Bach, S., Sutawika, L., Alyafeai, Z., Chaffin, A., Stiegler, A., Raja, A., Dey, M., Bari, M. S., Xu, C., Thakker, U., Sharma, S. S., Szczechla, E., Kim, T., Chhablani, G., Nayak, N., Datta, D., Chang, J., Jiang, M. T.-J., Wang, H., Manica, M., Shen, S., Yong, Z. X.,… — [Multitask Prompted Training Enables Zero-Shot Task Generalization](https://md.research.elamur.ai/free/en/papers/sanh-2021-multitask-prompted-training-enables-zero/)
29. Santana, G. Gustavosta/Stable-Diffusion-Prompts · Datasets at Hugging Face, December 2022. URL [https://huggingface.co/datasets/Gustavosta/Stable-Diffusion-Prompts](https://huggingface.co/datasets/Gustavosta/Stable-Diffusion-Prompts). [↗](https://huggingface.co/datasets/Gustavosta/Stable-Diffusion-Prompts)
30. Schuhmann, C., Beaumont, R., Vencu, R., Gordon, C., Wightman, R., Cherti, M., Coombes, T., Katta, A., Mullis, C., Wortsman, M., et al. [Laion-5b: An open large-scale dataset for training next generation image-text models](https://md.research.elamur.ai/free/en/papers/schuhmann-2022-laion-5b-open-large-scale/). arXiv preprint arXiv:2210.08402, 2022.
31. Shazeer, N. and Stern, M. Adafactor: Adaptive learning rates with sublinear memory cost. In International Conference on Machine Learning, pp. 4596–4604. PMLR, 2018. [find](https://scholar.google.com/scholar?q=Shazeer%2C+N%2E+and+Stern%2C+M%2E+Adafactor%3A+Adaptive+learning+rates+with+sublinear+memory+cost%2E+In+International+Conference+on+Machine+Learning%2C+pp%2E+4596%E2%80%934604%2E+PMLR%2C+2018%2E)
32. Shi, W., Han, X., Gonen, H., Holtzman, A., Tsvetkov, Y., and Zettlemoyer, L. Toward human readable prompt tuning: Kubrick’s the shining is a good movie, and a good prompt too? arXiv preprint arXiv:2212.10539, 2022. [find](https://scholar.google.com/scholar?q=Shi%2C+W%2E%2C+Han%2C+X%2E%2C+Gonen%2C+H%2E%2C+Holtzman%2C+A%2E%2C+Tsvetkov%2C+Y%2E%2C+and+Zettlemoyer%2C+L%2E+Toward+human+readable+prompt+tuning%3A+Kubrick%E2%80%99s+the+shining+is+a+good+movie%2C+and+a+good+prompt+too%3F+arXiv+preprint+arXiv%3A221)
33. Shin, T., Razeghi, Y., Logan IV, R. L., Wallace, E., and Singh, S. [AutoPrompt: Eliciting Knowledge from Language Models with Automatically Generated Prompts](https://md.research.elamur.ai/free/en/papers/shin-2020-autoprompt-eliciting-knowledge-language-models/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 4222– 4235, Online, November 2020. Asso…
34. Socher, R., Perelygin, A., Wu, J., Chuang, J., Manning, C. D., Ng, A., and Potts, C. [Recursive deep models for semantic compositionality over a sentiment treebank](https://md.research.elamur.ai/free/en/papers/socher-2013-recursive-deep-models-semantic-compositionality/). In Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631– 1642, Seattle, Washington, USA, Oct…
35. Zhang, P., Li, X., Hu, X., Yang, J., Zhang, L., Wang, L., Choi, Y., and Gao, J. Vinvl: Revisiting visual representations in vision-language models. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5579–5588, 2021. [find](https://scholar.google.com/scholar?q=Zhang%2C+P%2E%2C+Li%2C+X%2E%2C+Hu%2C+X%2E%2C+Yang%2C+J%2E%2C+Zhang%2C+L%2E%2C+Wang%2C+L%2E%2C+Choi%2C+Y%2E%2C+and+Gao%2C+J%2E+Vinvl%3A+Revisiting+visual+representations+in+vision%2Dlanguage+models%2E+In+Proceedings+of+the+IEEE%2FCVF+Conference+on+Compute)
36. Zhang, S., Roller, S., Goyal, N., Artetxe, M., Chen, M., Chen, S., Dewan, C., Diab, M., Li, X., Lin, X. V., et al. [Opt: Open pre-trained transformer language models](https://md.research.elamur.ai/free/en/papers/zhang-2022-opt-open-pre-trained-transformer/). arXiv preprint arXiv:2205.01068, 2022.
37. Zhang, X., Zhao, J., and LeCun, Y. [Character-level convolutional networks for text classification](https://md.research.elamur.ai/free/en/papers/zhang-2015-character-level-convolutional-networks-text/). Advances in neural information processing systems, 28, 2015.

*Updated 2 November 2026*