---
title: "AMTA Best Thesis Award Abstract — Salesky 2026"
description: "This award-winning thesis abstract addresses a part of NLP pipelines that has changed little since neural MT emerged: building the model vocabulary."
authors: ["Elizabeth Salesky"]
year: 2026
venue: "Proceedings of the 17th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)"
source: "https://aclanthology.org/2026.amta-research.1/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/salesky-2026-amta-best-thesis-award-abstract/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/salesky-2026-amta-best-thesis-award-abstract/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/) / AMTA Best Thesis Award Abstract: Overcoming Vocabulary Challenges in Natural Language Processing

# AMTA Best Thesis Award Abstract: Overcoming Vocabulary Challenges in Natural Language Processing

**Authors:** Elizabeth Salesky · **Year:** 2026 · **Venue:** Proceedings of the 17th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.amta-research.1/) · [PDF](https://aclanthology.org/2026.amta-research.1.pdf)

## Summary
This award-winning thesis abstract addresses a part of NLP pipelines that has changed little since neural MT emerged: building the model vocabulary. Subword tokenizers are organized around bytes and Unicode storage, not the structure of languages and scripts. As a result, they cover morphologically rich or non-standardized languages poorly and leave models fragile to typos and unusual codepoints. In multilingual systems, they force a trade-off between per-language capacity and parameter count, a "vocabulary bottleneck."

Working within machine translation, Salesky first shows that subword vocabulary size matters. She proposes growing the vocabulary incrementally during training instead of running separate costly trainings for each size. Next, she replaces the embedding matrix with representations of text rendered as images (pixels). This vocabulary-free input makes translation more robust to noise and strengthens cross-lingual transfer within and across scripts, especially for low-resource languages, without the overhead of byte-level or very large subword vocabularies. Finally, a new benchmark for translating text in natural images shows that direct multimodal models propagate fewer errors than OCR+MT cascades. The work matters for extending MT to the world's many languages and scripts.

## Key points
- The thesis targets three vocabulary problems in MT: choosing vocabulary size, robustness to noisy input, and the multilingual vocabulary bottleneck. It is set against the backdrop of roughly 7,000 languages and 150 scripts.
- Online incremental vocabulary expansion (Salesky et al., 2018) is presented as the first method for enlarging neural model vocabularies during training, making size tuning cheaper.
- Pixel-based text representations (Salesky et al., 2021; 2023) remove the discrete vocabulary altogether. They yield more compositional representations and greater positive transfer across scripts, particularly for under-resourced languages.
- To motivate the approach, the abstract cites the cost of conventional vocabularies. Expanding the LLaMA-2 vocabulary from 32k to 260k (534 languages) added about 30% more parameters. In another study, a 4× larger vocabulary made masked LM pretraining about 2.5× slower.
- A benchmark for visually-situated translation, together with a taxonomy of OCR errors (Salesky et al., 2024), shows that end-to-end pixel models are more robust than cascaded OCR→MT pipelines.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/en/shelves/linguistics/) · [Machine translation](https://md.research.elamur.ai/free/en/shelves/machine-translation/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{salesky2026amta,
  title = {{AMTA Best Thesis Award Abstract: Overcoming Vocabulary Challenges in Natural Language Processing}},
  author = {Salesky, Elizabeth},
  booktitle = {Proceedings of the 17th Conference of the Association for Machine Translation in the Americas (Volume 1: Research Track)},
  year = {2026},
  url = {https://aclanthology.org/2026.amta-research.1/}
}
```

- [← previous AMIYA Shared Task: Arabic Modeling In Your Accent at VarDial 2026](https://md.research.elamur.ai/free/en/papers/robinson-2026-amiya-shared-task-arabic-modeling/)
- [next → Amulwe Kimün: A Community-Grounded Demo, Resource, and ASR Baseline for Mapuzugun](https://md.research.elamur.ai/free/en/papers/oliva-2026-amulwe-kimun-community-grounded-demo/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/)