---
title: "Fill-in-the-Blanks — Szawerna 2026"
description: "Most privacy work on language data has targeted spotting personal information, while the step that follows, producing realistic replacement names and…"
authors: ["Maria Irena Szawerna", "Jacob Lee Suchardt"]
year: 2026
venue: "Proceedings of the Fifteenth Language Resources and Evaluation Conference"
source: "https://aclanthology.org/2026.lrec-1.90/"
doi: "10.63317/42gh3j2hfmca"
licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)"
licence_url: "https://creativecommons.org/licenses/by-nc/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/papers/szawerna-2026-fill-blanks-automatic-generation-evaluation/"
---
[Elamur Research](https://md.research.elamur.ai/) · [this page in HTML](https://research.elamur.ai/free/papers/szawerna-2026-fill-blanks-automatic-generation-evaluation/)

[home](https://md.research.elamur.ai/) / [papers](https://md.research.elamur.ai/free/papers/) / [2026](https://md.research.elamur.ai/free/papers/2026/page/2/) / Fill-in-the-Blanks: Automatic Generation and Evaluation of Language Models' Pseudonyms for English and Swedish Texts

# Fill-in-the-Blanks: Automatic Generation and Evaluation of Language Models' Pseudonyms for English and Swedish Texts

**Authors:** [Maria Irena Szawerna](https://md.research.elamur.ai/free/authors/maria-irena-szawerna/), Jacob Lee Suchardt · **Year:** 2026 · **Venue:** Proceedings of the Fifteenth Language Resources and Evaluation Conference · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.lrec-1.90/) · [PDF](https://aclanthology.org/2026.lrec-1.90.pdf) · [DOI 10.63317/42gh3j2hfmca](https://doi.org/10.63317/42gh3j2hfmca) · [OpenAlex](https://openalex.org/W7160145496)

**Citations:** 0 citations (openalex-snapshot · 2026-10-14)

## Summary
Most privacy work on language data has targeted spotting personal information, while the step that follows, producing realistic replacement names and phrases and judging their quality, remains underexplored. The authors treat this step as a gap-filling task. Once personally identifiable spans are blanked out, small, locally runnable models propose substitutes. They compare masked language models such as RoBERTa with a compact generative model (Qwen) and simple baselines. The test data are English legal documents from the TAB corpus and a parallel English–Swedish news sample drawn from the Parallel Global Voices corpus. The team annotated the news sample manually. They also test whether Universal Part-of-Speech constraints help and look for automatic quality measures. A fine-grained manual error analysis shows that models frequently produce replacements that are grammatical yet semantically wrong, inconsistent, or clichéd and even stereotypical. Automatic large-scale evaluation remains difficult. The work matters for anyone building shareable corpora that must stay readable after pseudonymization.

## Key points
- The study is organized around three research questions:
  - how well small MLMs and GenLLMs fill PII gaps compared with naïve baselines
  - whether UPOS constraints improve results
  - how pseudonym quality can be scored automatically
- The released resources are the PGV-PII minicorpus, which contains English and Swedish PII annotations and is available via Språkbanken Text, and the code repository "fill-in-the-blanks" on GitHub.
- In one setup (repL), PII in the left-hand context is replaced by the model's own earlier predictions, so later guesses build on earlier ones.
- Generated pseudonyms were labelled with a nine-category taxonomy that includes NO_ERROR, OVERLAP, GRAMMAR, POS, STRUCTURAL and three semantic error types (category, contradiction, merged entities).
- Error-free replacements made up only about 15–33% of cases across the compared systems and languages. Wrong semantic category was frequently the dominant error.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

Fields: [Linguistics & languages](https://md.research.elamur.ai/free/shelves/linguistics/)

## Cite

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{szawerna2026fill,
  title = {{Fill-in-the-Blanks: Automatic Generation and Evaluation of Language Models' Pseudonyms for English and Swedish Texts}},
  author = {Szawerna, Maria Irena and Suchardt, Jacob Lee},
  booktitle = {Proceedings of the Fifteenth Language Resources and Evaluation Conference},
  year = {2026},
  doi = {10.63317/42gh3j2hfmca},
  url = {https://doi.org/10.63317/42gh3j2hfmca}
}
```

- [← previous FIBER: A Multilingual Evaluation Resource for Factual Inference Bias](https://md.research.elamur.ai/free/papers/munis-2026-fiber-multilingual-evaluation-resource-factual/)
- [next → Findings of the Fifth Shared Task on Multilingual Coreference Resolution: Expanding Datasets for Long-Range Entities](https://md.research.elamur.ai/free/papers/novak-2026-findings-fifth-shared-task-multilingual/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/papers/2026/page/2/)

## References

18 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Alyaa Alfalahi, Sara Brissman, and Hercules Dalianis. 2012. Pseudonymisation of Personal Names and other PHIs in an Annotated Clinical Swedish Corpus. In Proceedings of the Third Workshop on Building and Evaluating Resources for Biomedical Text Mining (BioTxtM 2012) held in conjunction with LREC 2012. [find](https://scholar.google.com/scholar?q=Alyaa+Alfalahi%2C+Sara+Brissman%2C+and+Hercules+Dalianis%2E+2012%2E+Pseudonymisation+of+Personal+Names+and+other+PHIs+in+an+Annotated+Clinical+Swedish+Corpus%2E+In+Proceedings+of+the+Third+Workshop+on+Building+)
2. Nikolay Arefyev, Boris Sheludko, Alexander Podolskiy, and Alexander Panchenko. 2020. A comparative study of lexical substitution approaches based on neural language models. [↗](https://doi.org/10.48550/arxiv.2006.00031)
3. Emily M. Bender, Timnit Gebru, Angelina McMillan-Major, and Shmargaret Shmitchell. 2021. On the dangers of stochastic parrots: Can language models be too big? In Proceedings of the 2021 ACM Conference on Fairness, Accountability, and Transparency, FAccT ’21, page 610–623, New York, NY, USA. Association for Computing Machinery. [↗](https://doi.org/10.1145/3442188.3445922)
4. Rogier Blokland, Niko Partanen, and Michael Rießler. 2020. A pseudonymisation method for language documentation corpora: An experiment with spoken Komi. In Proceedings of the Sixth International Workshop on Computational Linguistics of Uralic Languages, pages 1– 8, Wien, Austria. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Rogier+Blokland%2C+Niko+Partanen%2C+and+Michael+Rie%C3%9Fler%2E+2020%2E+A+pseudonymisation+method+for+language+documentation+corpora%3A+An+experiment+with+spoken+Komi%2E+In+Proceedings+of+the+Sixth+International+Works)
5. Hercules Dalianis. 2019. Pseudonymisation of Swedish electronic patient records using a rule-based approach. In Proceedings of the Workshop on NLP and Pseudonymisation, pages 16– 23, Turku, Finland. Linköping Electronic Press. [find](https://scholar.google.com/scholar?q=Hercules+Dalianis%2E+2019%2E+Pseudonymisation+of+Swedish+electronic+patient+records+using+a+rule%2Dbased+approach%2E+In+Proceedings+of+the+Workshop+on+NLP+and+Pseudonymisation%2C+pages+16%E2%80%93+23%2C+Turku%2C+Finland%2E+L)
6. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of deep bidirectional transformers for language understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n19-1423)
7. Benjamin Dutilleul, Mathis Debaillon, and Sandeep Mathias. 2024. ISEP_Presidency_University at MLSP 2024 shared task: Using GPT-3.5 to generate substitutes for lexical simplification. In Proceedings of the 19th Workshop on Innovative Use of NLP for Building Educational Applications (BEA 2024), pages 605–609, Mexico City, Mexico. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Benjamin+Dutilleul%2C+Mathis+Debaillon%2C+and+Sandeep+Mathias%2E+2024%2E+ISEP%5FPresidency%5FUniversity+at+MLSP+2024+shared+task%3A+Using+GPT%2D3%2E5+to+generate+substitutes+for+lexical+simplification%2E+In+Proceedings+o)
8. Elisabeth Eder, Ulrike Krieg-Holz, and Udo Hahn. 2019. De-identification of emails: Pseudonymizing privacy-sensitive data in a German email corpus. In Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019), pages 259– 269, Varna, Bulgaria. INCOMA Ltd. [find](https://scholar.google.com/scholar?q=Elisabeth+Eder%2C+Ulrike+Krieg%2DHolz%2C+and+Udo+Hahn%2E+2019%2E+De%2Didentification+of+emails%3A+Pseudonymizing+privacy%2Dsensitive+data+in+a+German+email+corpus%2E+In+Proceedings+of+the+International+Conference+on+Re)
9. Shayna Gardiner, Tania Habib, Kevin Humphreys, Masha Azizi, Frederic Mailhot, Anne Paling, Preston Thomas, and Nathan Zhang. 2024. Data anonymization for privacy-preserving large language model fine-tuning on call transcripts. In Proceedings of the Workshop on Computational Approaches to Language Data Pseudonymization (CALD-pseudo 2024), pages 64–75, St. Julian’s, Malta. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Shayna+Gardiner%2C+Tania+Habib%2C+Kevin+Humphreys%2C+Masha+Azizi%2C+Frederic+Mailhot%2C+Anne+Paling%2C+Preston+Thomas%2C+and+Nathan+Zhang%2E+2024%2E+Data+anonymization+for+privacy%2Dpreserving+large+language+model+fine%2Dt)
10. Shilong Hou, Ruilin Shang, Zi Long, Xianghua Fu, and Yin Chen. 2025. A general pseudonymization framework for cloud-based llms: Replacing privacy information in controlled text generation. [↗](https://doi.org/10.48550/arxiv.2502.15233)
11. ISO/IEC 29100:2024(en). 2024. Information technology — Security techniques — Privacy framework. Standard, International Organization for Standardization, Geneva, Switzerland. [find](https://scholar.google.com/scholar?q=ISO%2FIEC+29100%3A2024%28en%29%2E+2024%2E+Information+technology+%E2%80%94+Security+techniques+%E2%80%94+Privacy+framework%2E+Standard%2C+International+Organization+for+Standardization%2C+Geneva%2C+Switzerland%2E)
12. Zilyu Ji, Yuntian Shen, Kenneth R. Koedinger, and Jionghao Lin. 2025. Enhancing the de-identification of personally identifiable information in educational data. Journal of Educational Data Mining, 17(2). [↗](https://doi.org/10.5281/zenodo.17114270)
13. Szivia Lestyán, William Letrone, Ludovica Robustelli, and Gergely Biczók. 2025. Anonymity-washing. ArXiv:2505.18627 [cs]. [↗](https://arxiv.org/abs/2505.18627)
14. Chin-Yew Lin. 2004. ROUGE: A package for automatic evaluation of summaries. In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Chin%2DYew+Lin%2E+2004%2E+ROUGE%3A+A+package+for+automatic+evaluation+of+summaries%2E+In+Text+Summarization+Branches+Out%2C+pages+74%E2%80%9381%2C+Barcelona%2C+Spain%2E+Association+for+Computational+Linguistics%2E)
15. Pierre Lison, Ildikó Pilán, David Sanchez, Montserrat Batet, and Lilja Øvrelid. 2021. Anonymisation models for text data: State of the art, challenges and future directions. In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), pages 4188–4203, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Pierre+Lison%2C+Ildik%C3%B3+Pil%C3%A1n%2C+David+Sanchez%2C+Montserrat+Batet%2C+and+Lilja+%C3%98vrelid%2E+2021%2E+Anonymisation+models+for+text+data%3A+State+of+the+art%2C+challenges+and+future+directions%2E+In+Proceedings+of+the+59th)
16. Pulkit Madaan, Krithika Ramesh, Lisa Bauer, Charith Peris, and Anjalie Field. 2026. Multi-token completion for text anonymization. [find](https://scholar.google.com/scholar?q=Pulkit+Madaan%2C+Krithika+Ramesh%2C+Lisa+Bauer%2C+Charith+Peris%2C+and+Anjalie+Field%2E+2026%2E+Multi%2Dtoken+completion+for+text+anonymization%2E)
17. Hiroki Nakayama, Takahiro Kubo, Junya Kamura, Yasufumi Taniguchi, and Xu Liang. 2018. doccano: Text annotation tool for human. Software available from [https://github.com/doccano/doccano](https://github.com/doccano/doccano). [↗](https://github.com/doccano/doccano)
18. Anastasia Nikiforova, Sergey Pletenev, Daria Sinitsyna, Semen Sorokin, Anastasia Lopukhina, and Nick Howell. 2020. Language models for cloze task answer generation in Russian. In Proceedings of the Second Workshop on Linguistic and Neurocognitive Resources, pages 28–37, Marseille, France. European Language Resources Association. [find](https://scholar.google.com/scholar?q=Anastasia+Nikiforova%2C+Sergey+Pletenev%2C+Daria+Sinitsyna%2C+Semen+Sorokin%2C+Anastasia+Lopukhina%2C+and+Nick+Howell%2E+2020%2E+Language+models+for+cloze+task+answer+generation+in+Russian%2E+In+Proceedings+of+the+Se)