---
title: "Controllable Lexical Simplification for English — Sheang 2022"
description: "The authors build ConLS by fine-tuning T5 on the small TSAR-EN dataset. 22 references, cited by 12 papers here."
authors: ["Kim Cheng Sheang", "Daniel Ferrés", "Horacio Saggion"]
year: 2022
venue: "Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022)"
source: "https://aclanthology.org/2022.tsar-1.19/"
doi: "10.18653/v1/2022.tsar-1.19"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/sheang-2022-controllable-lexical-simplification-english/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/sheang-2022-controllable-lexical-simplification-english/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2022](https://md.research.elamur.ai/free/en/papers/2022/page/5/) / Controllable Lexical Simplification for English

# Controllable Lexical Simplification for English

**Authors:** Kim Cheng Sheang, [Daniel Ferrés](https://md.research.elamur.ai/free/en/authors/daniel-ferres/) and [Horacio Saggion](https://md.research.elamur.ai/free/en/authors/horacio-saggion/) · **Year:** 2022 · **Venue:** [Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022)](https://md.research.elamur.ai/free/en/venues/tsar/) ([Workshop on Text Simplification, Accessibility and Readability 2022](https://md.research.elamur.ai/free/en/venues/tsar/2022/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2022.tsar-1.19/) · [PDF](https://aclanthology.org/2022.tsar-1.19.pdf) · [DOI 10.18653/v1/2022.tsar-1.19](https://doi.org/10.18653/v1/2022.tsar-1.19) · [OpenAlex](https://openalex.org/W4386339106)

**Citations:** 4 citations (OpenAlex, 2026-10-14) · [12](https://md.research.elamur.ai/free/en/papers/sheang-2022-controllable-lexical-simplification-english/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
The paper tackles lexical simplification (LS), the task of replacing a hard word in context with an easier one. Earlier neural LS systems, notably LSBert, generate candidates with a masked language model. This work asks whether fine-tuning a sequence-to-sequence Transformer with explicit control signals can do as well. The authors build ConLS by fine-tuning T5 on the small TSAR-EN dataset. They add three control tokens to the front of each source sentence:
- a character-length ratio between substitute and complex word,
- a frequency-rank ratio based on FastText vocabulary order,
- a coarse encoding of how annotators ranked the candidate.

Special markers enclose the target word in both input and output. At inference, the length and rank values are tuned once on validation data and then frozen. On LexMTurk, BenchLS and NNSeval, ConLS performs on par with a modified LSBert that returns ten candidates, and beats it on several metrics. An ablation shows that using all tokens together works best. This is presented as the first controllable approach to LS.

## Key points
- Training and validation use TSAR-EN, which has 386 instances, each with 25 crowd-sourced substitutes, split 90/10. Testing uses LexMTurk (500 Wikipedia sentences), BenchLS and NNSeval.
- The control tokens are WL (word length ratio), WR (inverse-frequency rank ratio from FastText) and CR (candidate rank, mapped to 1.00/0.75/0.50/0.25/0.00).
- Every gold candidate becomes its own parallel training pair.
- Model size matters a great deal. On LexMTurk, ACC@1 is 23.40 for T5-Small, 60.00 for T5-Base and 80.60 for T5-Large.
- In the ablation on LexMTurk, all tokens together reach ACC@1 80.60 and ACC@1@Top1 43.80. Without tokens the scores are 79.20 and 41.80.
- WL and WR are searched with Optuna in the range 0.5–1.25 in steps of 0.05, with CR fixed at 1.00.
- The LSBert baseline was adapted to output 10 candidates (15 mask selections), and candidates identical to the complex word were removed.

## Interesting ideas
- Control values are computed from the data as ratios between source and target properties, such as length and frequency. They are attached as prefix tokens, so at generation time one can request outputs that are "shorter" or "more common" without any post-filtering.
- Bracket markers around the target span, in both input and output, let the system reliably extract the substituted word from free-form generated text.
- One annotated item with many ranked answers is expanded into many training pairs, each carrying its rank as a coarse token. The model thus learns the ordering itself rather than relying on a separate ranker.
- Control values are not predicted per instance. Instead, they are tuned once on held-out data to maximize a chosen metric and then frozen. This offers a cheap baseline before building a dedicated predictor.
- The ablation over token subsets shows that most single tokens add little and that the gain from all tokens together is only one to two points on a 500-item test. This is a reminder to report small-margin effects carefully.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (back): When generating text from the meaning graph, lexical choice could be steered by explicit, logged ratio knobs for frequency and length. This would make each simplification decision traceable to a stated target value.
- **I2** (learn): When LLM teacher tiers rewrite or relabel sentences, wrapping the focus span in markers would let us align the teacher's answer to a specific tree node instead of guessing by string matching.
- **I3** (learn): Silver data agreed by several witness parsers could be expanded the same way. Each alternative analysis would be tagged with its agreement level, so the learner sees both the consensus and the weaker variants.
- **I4** (learn): Thresholds in self-induced repair rules or confidence cutoffs could first be set by a validation sweep and frozen. A per-sentence predictor would only be adopted if it beats this baseline.
- **I5** (learn): Evaluations of new features should include a "no feature" negative control and subset ablations, so that small gains are not over-interpreted as real improvements.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Universitat Pompeu Fabra](https://md.research.elamur.ai/free/en/universities/universitat-pompeu-fabra/) — University, Barcelona, Spain · 36 works · 11,433 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [English](https://md.research.elamur.ai/free/en/topics/english/) · 1,373 works

## Cite

Cite as: Kim Cheng Sheang, Daniel Ferrés and Horacio Saggion. 2022. Controllable Lexical Simplification for English. Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{sheang2022controllable,
  title = {{Controllable Lexical Simplification for English}},
  author = {Kim Cheng Sheang, Daniel Ferrés and Horacio Saggion},
  booktitle = {Proceedings of the Workshop on Text Simplification, Accessibility, and Readability (TSAR-2022)},
  year = {2022},
  doi = {10.18653/v1/2022.tsar-1.19},
  url = {https://doi.org/10.18653/v1/2022.tsar-1.19}
}
```

- [← previous A Simple and Strong Baseline for End-to-End Neural RST-style Discourse Parsing](https://md.research.elamur.ai/free/en/papers/kobayashi-2022-simple-strong-baseline-end-end/)
- [next → Improving Automatic Speech Recognition for Non-Native English with Transfer Learning and Language Model Decoding](https://md.research.elamur.ai/free/en/papers/sullivan-2022-improving-automatic-speech-recognition-non/)
- [↑ back to papers of 2022](https://md.research.elamur.ai/free/en/papers/2022/page/5/)

## References

22 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Takuya Akiba, Shotaro Sano, Toshihiko Yanase, T. Ohta, and Masanori Koyama. 2019. [Optuna: A next-generation hyperparameter optimization framework](https://md.research.elamur.ai/free/en/papers/akiba-2019-optuna-next-generation-hyperparameter-optimization/).
2. Colby Horn, Cathryn Manduca, and David Kauchak. 2014. [Learning a Lexical Simplifier Using Wikipedia](https://md.research.elamur.ai/free/en/papers/horn-2014-learning-lexical-simplifier-wikipedia/). In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 458–463, Baltimore, Maryland. Association for Computational Linguistics.
3. Mike Lewis, Yinhan Liu, Naman Goyal, Marjan Ghazvininejad, Abdelrahman Mohamed, Omer Levy, Veselin Stoyanov, and Luke Zettlemoyer. 2019. [BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension](https://md.research.elamur.ai/free/en/papers/lewis-2020-bart-denoising-sequence-sequence-pre/). CoRR, abs/1910.13461. [↗](https://arxiv.org/abs/1910.13461)
4. Ilya Loshchilov and Frank Hutter. 2019. Decoupled weight decay regularization. In International Conference on Learning Representations. [find](https://scholar.google.com/scholar?q=Ilya+Loshchilov+and+Frank+Hutter%2E+2019%2E+Decoupled+weight+decay+regularization%2E+In+International+Conference+on+Learning+Representations%2E)
5. Louis Martin, Éric de la Clergerie, Benoît Sagot, and Antoine Bordes. 2020. Controllable sentence simplification. In Proceedings of the 12th Language Resources and Evaluation Conference, pages 4689– 4698. [↗](https://doi.org/10.48550/arxiv.1910.02677)
6. Louis Martin, Angela Fan, Éric de la Clergerie, Antoine Bordes, and Benoît Sagot. 2022. [MUSS: Multilingual unsupervised sentence simplification by mining paraphrases](https://md.research.elamur.ai/free/en/papers/martin-2022-muss-multilingual-unsupervised-sentence-simplification/). In Proceedings of the Thirteenth Language Resources and Evaluation Conference, pages 1651–1664, Marseille, France. European Language Resources Association.
7. Gustavo Paetzold and Lucia Specia. 2016a. Benchmarking Lexical Simplification Systems. In Proceedings of LREC-2016. [find](https://scholar.google.com/scholar?q=Gustavo+Paetzold+and+Lucia+Specia%2E+2016a%2E+Benchmarking+Lexical+Simplification+Systems%2E+In+Proceedings+of+LREC%2D2016%2E)
8. Gustavo Paetzold and Lucia Specia. 2016b. Unsupervised Lexical Simplification for Non-Native Speakers. Proceedings of the AAAI Conference on Artificial Intelligence, 30(1). [find](https://scholar.google.com/scholar?q=Gustavo+Paetzold+and+Lucia+Specia%2E+2016b%2E+Unsupervised+Lexical+Simplification+for+Non%2DNative+Speakers%2E+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+30%281%29%2E)
9. Gustavo Paetzold and Lucia Specia. 2017a. A Survey on Lexical Simplification. Journal of Artificial Intelligence Research, 60:549–593. [↗](https://doi.org/10.1613/jair.5526)
10. Gustavo Paetzold and Lucia Specia. 2017b. Lexical simplification with neural ranking. In Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics: Volume 2, Short Papers, pages 34–40, Valencia, Spain. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Gustavo+Paetzold+and+Lucia+Specia%2E+2017b%2E+Lexical+simplification+with+neural+ranking%2E+In+Proceedings+of+the+15th+Conference+of+the+European+Chapter+of+the+Association+for+Computational+Linguistics%3A+Vo)
11. Jipeng Qiang, Yun Li, Zhu Yi, Yunhao Yuan, and Xindong Wu. 2020. Lexical simplification with pre-trained encoders. Thirty-Fourth AAAI Conference on Artificial Intelligence, pages 8649—-8656. [find](https://scholar.google.com/scholar?q=Jipeng+Qiang%2C+Yun+Li%2C+Zhu+Yi%2C+Yunhao+Yuan%2C+and+Xindong+Wu%2E+2020%2E+Lexical+simplification+with+pre%2Dtrained+encoders%2E+Thirty%2DFourth+AAAI+Conference+on+Artificial+Intelligence%2C+pages+8649%E2%80%94%2D8656%2E)
12. Jipeng Qiang, Yun Li, Yi Zhu, Yunhao Yuan, Yang Shi, and Xindong Wu. 2021. Lsbert: Lexical simplification based on bert. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29:3064– 3076. [↗](https://doi.org/10.1109/taslp.2021.3111589)
13. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). Journal of Machine Learning Research, 21(140):1–67. [↗](https://doi.org/10.48550/arxiv.1910.10683)
14. Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander Rush. 2020. [Transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations, pages 38–45.
15. Mark Yatskar, Bo Pang, Cristian Danescu-Niculescu-Mizil, and Lillian Lee. 2010. [For the sake of simplicity: Unsupervised extraction of lexical simplifications from Wikipedia](https://md.research.elamur.ai/free/en/papers/yatskar-2010-sake-simplicity-unsupervised-extraction-lexical/). In Human Language Technologies: The 2010 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 365–368, Los Angeles, California. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1008.1986)
16. Seid Muhie Yimam, Chris Biemann, Shervin Malmasi, Gustavo H. Paetzold, Lucia Specia, Sanja Stajner, Anaïs Tack, and Marcos Zampieri. 2018. A Report on the Complex Word Identification Shared Task 2018. CoRR, abs/1804.09132. [↗](https://arxiv.org/abs/1804.09132)
17. Sanja Štajner, Richard Evans, Constantin Orasan, and Ruslan Mitkov. 2012. What can readability measures really tell us about text complexity. In Proceedings of workshop on natural language processing for improving textual accessibility, pages 14–22. Citeseer. [find](https://scholar.google.com/scholar?q=Sanja+%C5%A0tajner%2C+Richard+Evans%2C+Constantin+Orasan%2C+and+Ruslan+Mitkov%2E+2012%2E+What+can+readability+measures+really+tell+us+about+text+complexity%2E+In+Proceedings+of+workshop+on+natural+language+processing+)
18. Sanja Štajner, Daniel Ferrés, Matthew Shardlow, Kai North, Marcos Zampieri, and Horacio Saggion. 2022a. Lexical Simplification Benchmarks for English, Portuguese, and Spanish. Frontiers in Artificial Intelligence, 5. [↗](https://doi.org/10.3389/frai.2022.991242)
19. Sanja Štajner, Kim Cheng Sheang, and Horacio Saggion. 2022b. Sentence simplification capabilities of transfer-based models. Proceedings of the AAAI Conference on Artificial Intelligence, 36(11):12172– 12180. [find](https://scholar.google.com/scholar?q=Sanja+%C5%A0tajner%2C+Kim+Cheng+Sheang%2C+and+Horacio+Saggion%2E+2022b%2E+Sentence+simplification+capabilities+of+transfer%2Dbased+models%2E+Proceedings+of+the+AAAI+Conference+on+Artificial+Intelligence%2C+36%2811%29%3A12172%E2%80%93)
20. Horacio Saggion. 2017. Automatic Text Simplification. Synthesis Lectures on Human Language Technologies. Morgan & Claypool Publishers. [↗](https://doi.org/10.1007/978-3-031-02166-4)
21. Matthew Shardlow. 2014. A Survey of Automated Text Simplification. International Journal of Advanced Computer Science and Applications, 4. [↗](https://doi.org/10.14569/specialissue.2014.040109)
22. Kim Cheng Sheang and Horacio Saggion. 2021. Controllable sentence simplification with a unified text-to-text transfer transformer. In Proceedings of the 14th International Conference on Natural Language Generation, pages 341–352, Aberdeen, Scotland, UK. Association for Computational Linguistics. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Remi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, [find](https://scholar.google.com/scholar?q=Kim+Cheng+Sheang+and+Horacio+Saggion%2E+2021%2E+Controllable+sentence+simplification+with+a+unified+text%2Dto%2Dtext+transfer+transformer%2E+In+Proceedings+of+the+14th+International+Conference+on+Natural+Langua)

*Updated 24 October 2026*