--- title: "Prompt Engineering for Named Entity Extraction from Portuguese" description: "The authors test whether LLMs guided only by prompts, through In-Context Learning, can do this work instead of fine-tuned… 13 references." authors: ["Giovanni Maffeo", "Catarina Silva", "Hugo Gonçalo Oliveira"] year: 2026 venue: "Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1" source: "https://aclanthology.org/2026.propor-1.116/" licence: "CC BY 4.0 (ACL Anthology)" licence_url: "https://creativecommons.org/licenses/by/4.0/" summary_licence: "Elamur Cosmic License" canonical: "https://research.elamur.ai/free/en/papers/maffeo-2026-prompt-engineering-named-entity-extraction/" --- [Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/maffeo-2026-prompt-engineering-named-entity-extraction/) [home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/45/) / Prompt Engineering for Named Entity Extraction from Portuguese Legal Documents # Prompt Engineering for Named Entity Extraction from Portuguese Legal Documents **Authors:** Giovanni Maffeo, [Catarina Silva](https://md.research.elamur.ai/free/en/authors/catarina-silva/) and [Hugo Gonçalo Oliveira](https://md.research.elamur.ai/free/en/authors/hugo-goncalo-oliveira/) · **Year:** 2026 · **Venue:** [Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1](https://md.research.elamur.ai/free/en/venues/propor/) ([International Conference on Computational Processing of Portuguese 2026](https://md.research.elamur.ai/free/en/venues/propor/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/) **Links:** [Publisher's page](https://aclanthology.org/2026.propor-1.116/) · [PDF](https://aclanthology.org/2026.propor-1.116.pdf) ## Summary Supervised NER for Portuguese legal texts is limited because annotated corpora are scarce and expensive to build. The authors test whether LLMs guided only by prompts, through In-Context Learning, can do this work instead of fine-tuned models. They adapt a prompting framework from clinical NER (Zhang et al., 2023) to the LeNER-Br corpus. Models mark entities with HTML span tags. The study varies three factors: - one prompt per entity type versus a single prompt, - chunk length (300, 600 or 900 characters, or single sentences), - prompt content, adding layers step by step: guidelines, context-free examples, rules drawn from error analysis, and one- or five-shot examples. Performance is measured with entity-level Exact Match Micro F1. Prompt content had a larger effect than splitting the task or shortening the input. The best setup also worked on the unseen test split, where GPT-5.2 reached 71.9% and a 4-bit Qwen-2.5:32B reached 57.9%. These scores remain well below a supervised LSTM-CRF (92.53%). Still, the results show that prompting is a workable option when labelled legal data is lacking. ## Key points - Baseline setup: Qwen 3 (8.2B, Q4_K_M quantisation, temperature 0) run on 900-character chunks. RQ1–RQ3 used the LeNER-Br validation split and RQ4 used the test split. The authors dropped CDJUR-BR because its annotations were inconsistent; for example, "Ministério Público" was sometimes labelled Person and sometimes left unlabelled. - Adding instructions based on common errors (H3d) gave the largest single gain, raising Micro F1 from 41.8% to 48.0%. One-shot prompting (H3e) was the best configuration overall, and five examples did not improve on it. - Giving every entity type its own dedicated prompt outperformed the baseline. Among chunking options, single sentences worked best, followed by 300, 600 and 900 characters. - Among the small 4-bit models, Qwen-3:8B scored highest. An ensemble of the three small models beat the weaker individual ones but not the larger Qwen-2.5:32B or GPT-5.2. - A manual check of 5 GPT-5.2 output chunks found that most errors involved entity boundaries (50%, n=7). Missing entities accounted for 35.7% and spurious extractions for 14.3%, with no wrong-type errors. Code and prompts are published at github.com/giovannimaffeo/prompt-engineering-portuguese-legal-ner. *Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.* 📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [⚖️ Law](https://md.research.elamur.ai/free/en/shelves/social-sciences/law/) · 84 works 🧭 Topics (2): [🏷️ Named entity recognition](https://md.research.elamur.ai/free/en/topics/named-entity-recognition/) · 214 works · [⚖️ Legal AI](https://md.research.elamur.ai/free/en/topics/legal-ai/) · 150 works ## Cite Cite as: Giovanni Maffeo, Catarina Silva and Hugo Gonçalo Oliveira. 2026. Prompt Engineering for Named Entity Extraction from Portuguese Legal Documents. Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1. The paper itself, not this page. BibTeX from the library's record of the work. ```bibtex @inproceedings{maffeo2026prompt, title = {{Prompt Engineering for Named Entity Extraction from Portuguese Legal Documents}}, author = {Giovanni Maffeo, Catarina Silva and Hugo Gonçalo Oliveira}, booktitle = {Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1}, year = {2026}, url = {https://aclanthology.org/2026.propor-1.116/} } ``` - [← previous Progressive Visual Refinement for Multi-modal Summarization](https://md.research.elamur.ai/free/en/papers/xiong-2026-progressive-visual-refinement-multi-modal/) - [next → Prompt Optimization for Relation Extraction using Reinforcement Learning](https://md.research.elamur.ai/free/en/papers/liu-2026-prompt-optimization-relation-extraction-reinforcement/) - [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/45/) ## References 13 references, 2 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet. 1. Hidelberg O. Albuquerque, Rosimeire Costa, Gabriel Silvestre, Ellen Souza, Nádia F. F. Silva, Diego Vitório, and André C. P. L. F. de Carvalho. 2022. Ulyssesner-br: A corpus of brazilian legislative documents for named entity recognition. In Computational Processing of the Portuguese Language (PRO-POR). Springer. [find](https://scholar.google.com/scholar?q=Hidelberg+O%2E+Albuquerque%2C+Rosimeire+Costa%2C+Gabriel+Silvestre%2C+Ellen+Souza%2C+N%C3%A1dia+F%2E+F%2E+Silva%2C+Diego+Vit%C3%B3rio%2C+and+Andr%C3%A9+C%2E+P%2E+L%2E+F%2E+de+Carvalho%2E+2022%2E+Ulyssesner%2Dbr%3A+A+corpus+of+brazilian+legislative+d) 2. Maurício Brito, Vládia Pinheiro, Vasco Furtado, João Araújo Monteiro Neto, Francisco das Chagas Jucá Bomfim, André C. F. Costa, Raquel Silveira, and Nilsiton Aragão. 2023. Cdjur-br: Uma coleção dourada do judiciário brasileiro com entidades nomeadas refinadas. In Simpósio Brasileiro de Tecnologia da Informação e da Linguagem Humana (STIL). Sociedade Brasileira de Computação. [find](https://scholar.google.com/scholar?q=Maur%C3%ADcio+Brito%2C+Vl%C3%A1dia+Pinheiro%2C+Vasco+Furtado%2C+Jo%C3%A3o+Ara%C3%BAjo+Monteiro+Neto%2C+Francisco+das+Chagas+Juc%C3%A1+Bomfim%2C+Andr%C3%A9+C%2E+F%2E+Costa%2C+Raquel+Silveira%2C+and+Nilsiton+Arag%C3%A3o%2E+2023%2E+Cdjur%2Dbr%3A+Uma+cole%C3%A7%C3%A3o+dourad) 3. Cristina Mota, Diana Santos, Luís Miguel Cabral, and Nuno Silva. 2010. Segundo harem: Avaliação de reconhecimento de entidades mencionadas em português. In Actas do Encontro Nacional da Associação Portuguesa de Linguística. [find](https://scholar.google.com/scholar?q=Cristina+Mota%2C+Diana+Santos%2C+Lu%C3%ADs+Miguel+Cabral%2C+and+Nuno+Silva%2E+2010%2E+Segundo+harem%3A+Avalia%C3%A7%C3%A3o+de+reconhecimento+de+entidades+mencionadas+em+portugu%C3%AAs%2E+In+Actas+do+Encontro+Nacional+da+Associa%C3%A7%C3%A3o+Por) 4. Fei Yu, Linda Quartey, and Frank Schilder. 2023. Exploring the effectiveness of prompt engineering for legal reasoning tasks. In Findings of the Association for Computational Linguistics: ACL 2023. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Fei+Yu%2C+Linda+Quartey%2C+and+Frank+Schilder%2E+2023%2E+Exploring+the+effectiveness+of+prompt+engineering+for+legal+reasoning+tasks%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+ACL+2023%2E+As) 5. Hao Zhang, Jingyan Wang, Shuai Yu, and 1 others. 2023. Improving large language models for clinical named entity recognition via prompt engineering. arXiv preprint. [find](https://scholar.google.com/scholar?q=Hao+Zhang%2C+Jingyan+Wang%2C+Shuai+Yu%2C+and+1+others%2E+2023%2E+Improving+large+language+models+for+clinical+named+entity+recognition+via+prompt+engineering%2E+arXiv+preprint%2E) 6. Lucy Zheng, Neel Guha, Patrick Ivy, Daniel E. Ho, and Peter Henderson. 2021. When does pretraining help? assessing self-supervised learning for law and the case of casehold. In Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing (EMNLP). [find](https://scholar.google.com/scholar?q=Lucy+Zheng%2C+Neel+Guha%2C+Patrick+Ivy%2C+Daniel+E%2E+Ho%2C+and+Peter+Henderson%2E+2021%2E+When+does+pretraining+help%3F+assessing+self%2Dsupervised+learning+for+law+and+the+case+of+casehold%2E+In+Proceedings+of+the+2021) 7. Ilias Chalkidis, Manos Fergadiotis, Prodromos Malakasiotis, and Nikos Aletras. 2020. [Legal-bert: The muppets straight out of law school](https://md.research.elamur.ai/free/en/papers/chalkidis-2020-legal-bert-muppets-straight-out/). In Findings of the Association for Computational Linguistics: EMNLP 2020. 8. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [Bert: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of NAACL-HLT. 9. Daniel Jurafsky and James H. Martin. 2023. Speech and Language Processing. Draft Edition, available online. [find](https://scholar.google.com/scholar?q=Daniel+Jurafsky+and+James+H%2E+Martin%2E+2023%2E+Speech+and+Language+Processing%2E+Draft+Edition%2C+available+online%2E) 10. Daniel Martin Katz, Michael James Bommarito, and Josh Blackman. 2020. Complex societies and the growth of the law. arXiv preprint arXiv:2005.07646. [↗](https://arxiv.org/abs/2005.07646) 11. Junjie Li, Aixin Sun, Jie Han, and Chenliang Li. 2022. A survey on deep learning for named entity recognition. IEEE Transactions on Knowledge and Data Engineering. [find](https://scholar.google.com/scholar?q=Junjie+Li%2C+Aixin+Sun%2C+Jie+Han%2C+and+Chenliang+Li%2E+2022%2E+A+survey+on+deep+learning+for+named+entity+recognition%2E+IEEE+Transactions+on+Knowledge+and+Data+Engineering%2E) 12. Pedro H. Luz de Araujo, Teófilo E. de Campos, Renato R. R. de Oliveira, Matheus Stauffer, Samuel Couto, and Paulo Bermejo. 2018. LeNER-Br: A dataset for named entity recognition in brazilian legal text. In Proceedings of the International Conference on the Computational Processing of Portuguese (PROPOR). [find](https://scholar.google.com/scholar?q=Pedro+H%2E+Luz+de+Araujo%2C+Te%C3%B3filo+E%2E+de+Campos%2C+Renato+R%2E+R%2E+de+Oliveira%2C+Matheus+Stauffer%2C+Samuel+Couto%2C+and+Paulo+Bermejo%2E+2018%2E+LeNER%2DBr%3A+A+dataset+for+named+entity+recognition+in+brazilian+legal+tex) 13. Chao Ma. 2023. Prompt engineering and calibration for zero-shot commonsense reasoning. arXiv preprint, arXiv:2304.06962. [↗](https://arxiv.org/abs/2304.06962) *Updated 31 October 2026*