--- title: "Evaluating Encoder- and LLM-Based Approaches for Robust" description: "Stripping explicit protected health information (PHI) from clinical notes does not remove all re-identification… 1 reference, cited by 1 paper here." authors: ["Christoph Otto", "Ibrahim Baroud", "Akiko Aizawa", "Sebastian Möller", "Roland Roller", "Lisa Raithel"] year: 2026 venue: "Proceedings of the Joint Workshop on Legal and Ethical Issues in Human Language Technologies and Computational Approaches to Language Data Pseudonymization, Anonymization, De-identification, and Data Privacy (LEGAL2026 and CALD-pseudo 2026) @ LREC 2026" source: "https://aclanthology.org/2026.legal-1.11/" doi: "10.63317/3noxgknxv96m" licence: "CC BY-NC 4.0 (ELRA, LREC proceedings)" licence_url: "https://creativecommons.org/licenses/by-nc/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/otto-2026-evaluating-encoder-llm-based-approaches/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/otto-2026-evaluating-encoder-llm-based-approaches/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/12/) / Evaluating Encoder- and LLM-Based Approaches for Robust Indirect Personal Identifier Detection # Evaluating Encoder- and LLM-Based Approaches for Robust Indirect Personal Identifier Detection **Authors:** Christoph Otto, [Ibrahim Baroud](https://md.research.elamur.ai/free/en/authors/ibrahim-baroud/), [Akiko Aizawa](https://md.research.elamur.ai/free/en/authors/akiko-aizawa/), [Sebastian Möller](https://md.research.elamur.ai/free/en/authors/sebastian-moller/), [Roland Roller](https://md.research.elamur.ai/free/en/authors/roland-roller/) and Lisa Raithel · **Year:** 2026 · **Venue:** [Proceedings of the Joint Workshop on Legal and Ethical Issues in Human Language Technologies and Computational Approaches to Language Data Pseudonymization, Anonymization, De-identification, and Data Privacy (LEGAL2026 and CALD-pseudo 2026) @ LREC 2026](https://md.research.elamur.ai/free/en/venues/legal/) ([Joint Workshop on Legal and Ethical Issues in Human Language Technologies and Computational Approaches to Language Data Pseudonymization, Anonymization, De-identification, and Data Privacy @ LREC 2026](https://md.research.elamur.ai/free/en/venues/legal/2026/)) · **Publisher:** [ELRA](https://md.research.elamur.ai/free/en/publishers/elra/) · **Licence:** [CC BY-NC 4.0 (ELRA, LREC proceedings)](https://creativecommons.org/licenses/by-nc/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.legal-1.11/) · [PDF](https://aclanthology.org/2026.legal-1.11.pdf) · [DOI 10.63317/3noxgknxv96m](https://doi.org/10.63317/3noxgknxv96m) · [OpenAlex](https://openalex.org/W7166594769) **Citations:** 0 citations (OpenAlex, 2026-10-14) · 1 in our library, of the 149,439 papers whose bibliographies we read ## Summary Stripping explicit protected health information (PHI) from clinical notes does not remove all re-identification risk. Contextual details such as occupation, family situation, lifestyle, or named institutions can still single out a patient when linked with outside sources. The authors address the automatic detection of these Indirect Personal Identifiers (IPIs) at span level in English discharge summaries. Using the IPI-annotated MIMIC-III corpus of Baroud et al. (2025), they compare three paradigms: - fine-tuned encoder-only models; - prompt-based frontier LLMs, including a two-stage variant that separates span finding from labelling; - hybrid pipelines, where an LLM proposes candidate spans and a RoBERTa-large classifier assigns labels. A fine-tuned RoBERTa-large performs best overall. It beats the earlier BERT-base baseline and far exceeds ChatGPT-5.2, and the hybrid setups also fall short of it. The authors argue that IPI detection is a distinct problem, marked by heavy class imbalance and very diverse expressions within each class. In this setting, larger models do not automatically give balanced per-class performance. Supervised encoders are therefore currently the most dependable basis for anonymization that goes beyond fixed PHI lists. ## Key points - **Data:** 100 de-identified MIMIC-III discharge summaries with 6199 annotated spans (inter-annotator agreement 0.87) across nine labels (BODY_DESC, SOCIO, DETAILS, DIRECT_ID, FAMILY, HEALTH_FCLT, RELATIVE_TIME, LFSTL, OTHER). The labels are dominated by RELATIVE_TIME and HEALTH_FCLT. - **Main result:** RoBERTa-large reaches 0.906 micro-F1 and 0.724 macro-F1. ChatGPT-5.2 reaches 0.509 and 0.487, and the prior BERT-base baseline 0.78 and 0.50. - **Encoder choice:** BioBERT, ClinicalBERT and ModernBERT brought no macro-level gains. QLoRA tuning of Qwen-14B did not consistently beat few-shot prompting of frontier LLMs. - **Hybrid bottleneck:** The ChatGPT-5.2 filtering stage recovered only 0.671 micro and 0.743 macro span recall, for example 638 of 1003 RELATIVE_TIME spans. This caps what the hybrid pipeline can achieve. - **Evaluation setup:** Relaxed span-level precision, recall and F1, with all documents split into chunks of up to 512 tokens. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* ## Institutions - [German Research Centre for Artificial Intelligence](https://md.research.elamur.ai/free/en/labs/german-research-centre-for-artificial-intelligence/) — Research institute, Kaiserslautern, Germany · 77 works · 29,479 citations - [University of Potsdam](https://md.research.elamur.ai/free/en/universities/university-of-potsdam/) — University, Potsdam, Germany · 44 works · 5,305 citations - [National Institute of Informatics](https://md.research.elamur.ai/free/en/labs/national-institute-of-informatics/) — Research institute, Tokyo, Japan · 37 works · 3,247 citations - [Technische Universität Berlin](https://md.research.elamur.ai/free/en/universities/technische-universitat-berlin/) — University, Berlin, Germany · 22 works · 957 citations 📚 Shelves (6): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works · [💭 Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/) · 1,552 works · [💭 Social Psychology](https://md.research.elamur.ai/free/en/shelves/psychology/social-psychology/) · 306 works 🧭 Topics (1): [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,220 works ## Cite Cite as: Christoph Otto, Ibrahim Baroud, Akiko Aizawa, Sebastian Möller, Roland Roller and Lisa Raithel. 2026. Evaluating Encoder- and LLM-Based Approaches for Robust Indirect Personal Identifier Detection. Proceedings of the Joint Workshop on Legal and Ethical Issues in Human Language Technologies and Computational Approaches to Language Data Pseudonymization, Anonymization, De-identification, and Data Privacy (LEGAL2026 and CALD-pseudo 2026) @ LREC 2026. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{otto2026evaluating, title = {{Evaluating Encoder- and LLM-Based Approaches for Robust Indirect Personal Identifier Detection}}, author = {Christoph Otto, Ibrahim Baroud, Akiko Aizawa, Sebastian Möller, Roland Roller and Lisa Raithel}, booktitle = {Proceedings of the Joint Workshop on Legal and Ethical Issues in Human Language Technologies and Computational Approaches to Language Data Pseudonymization, Anonymization, De-identification, and Data Privacy (LEGAL2026 and CALD-pseudo 2026) @ LREC 2026}, year = {2026}, doi = {10.63317/3noxgknxv96m}, url = {https://doi.org/10.63317/3noxgknxv96m} } ``` - [← previous Evaluating Document-Tuned Transformer Representations for Person-level Mental Health Assessment](https://md.research.elamur.ai/free/en/papers/marker-2026-evaluating-document-tuned-transformer-representations/) - [next → Evaluating Large Language Models for Medical Named Entity Recognition in Urdu: A Benchmark Study](https://md.research.elamur.ai/free/en/papers/nasim-2026-evaluating-large-language-models-medical/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/12/) ## References 1 references, 0 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Ibrahim Baroud, Lisa Raithel, Sebastian Möller, and Roland Roller. 2025. Beyond de-identification: A structured approach for defining and detecting indirect identifiers in medical texts. In Proceedings of the Sixth Workshop on Privacy in Natural Language Processing, pages 75–85, Albuquerque, New Mex… [find](https://scholar.google.com/scholar?q=Ibrahim+Baroud%2C+Lisa+Raithel%2C+Sebastian+M%C3%B6ller%2C+and+Roland+Roller%2E+2025%2E+Beyond+de%2Didentification%3A+A+structured+approach+for+defining+and+detecting+indirect+identifiers+in+medical+texts%2E+In+Proceeding) *Updated 31 October 2026*