---
title: "Neuro-symbolic Approaches for Rubric-Based Automatic Essay"
description: "The authors draw on the official ENEM Graders Guideline, which breaks each of the five traits into subcriteria, and propose… 26 references."
authors: ["Igor Cataneo Silveira", "Denis Deratani Mauá"]
year: 2026
venue: "Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1"
source: "https://aclanthology.org/2026.propor-1.78/"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/silveira-2026-neuro-symbolic-approaches-rubric-based/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/silveira-2026-neuro-symbolic-approaches-rubric-based/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/43/) / Neuro-symbolic Approaches for Rubric-Based Automatic Essay Evaluation of ENEM Essays

# Neuro-symbolic Approaches for Rubric-Based Automatic Essay Evaluation of ENEM Essays

**Authors:** [Igor Cataneo Silveira](https://md.research.elamur.ai/free/en/authors/igor-cataneo-silveira/) and [Denis Deratani Mauá](https://md.research.elamur.ai/free/en/authors/denis-deratani-maua/) · **Year:** 2026 · **Venue:** [Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1](https://md.research.elamur.ai/free/en/venues/propor/) ([International Conference on Computational Processing of Portuguese 2026](https://md.research.elamur.ai/free/en/venues/propor/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.propor-1.78/) · [PDF](https://aclanthology.org/2026.propor-1.78.pdf)

## Summary
Automated scoring of ENEM argumentative essays has mostly been treated as plain score prediction. That approach ignores the official grading rubrics and gives students little insight into why they received a grade. The authors draw on the official ENEM Graders Guideline, which breaks each of the five traits into subcriteria, and propose two neuro-symbolic scoring methods.

In the first, GPT4o writes a rubric-grounded justification of an essay's score. A BERTimbau-base model then predicts the trait score from that explanation rather than from the essay itself. The second method encodes the rubrics as logical rules over subcriteria, whose truth values are predicted by neural subnetworks trained with the Semantic Loss.

To support and evaluate this, two professional graders annotated a new set of 63 mock-ENEM essays for subcriteria. Both methods score about as well as purely statistical state-of-the-art models. They also offer finer-grained feedback, which is relevant for formative assessment and transparent high-stakes grading.

## Key points
- **New dataset.** 63 essays from AES-ENEM Source B were labeled by two expert graders against 71 subcriteria grouped into 20 dimensions. It is released on Hugging Face (`igorcs/enem-subcriteria-dataset`).
- **Rubric structure.** Trait C1 has 9 subcriteria in two dimensions: syntax (5) and mistakes (4). Exactly one subcriterion holds per dimension, and the score follows logically from them.
- **Inter-grader agreement (QWK) is lower on the subcriteria dataset than on AES-ENEM.** For example, C4 drops from .45 to .18 and C2 from .54 to .30.
- **Explanation-based scoring.** Training on GPT4o explanations performs comparably to training on essay text. The authors take this as evidence that the explanations are reliable.
- **Transfer and limitations.** For C1, subnetworks trained only on the small dataset transferred zero-shot to AES-ENEM with .58 QWK, and further distant supervision improved results. The authors note limitations: limited data coverage (especially for low-scoring essays), use of GPT4o as the only LLM, and a model that was not fully optimized.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

📚 Shelves (2): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works

🧭 Topics (1): [📏 Evaluation & benchmarks](https://md.research.elamur.ai/free/en/topics/evaluation-benchmarks/) · 1,341 works

## Cite

Cite as: Igor Cataneo Silveira and Denis Deratani Mauá. 2026. Neuro-symbolic Approaches for Rubric-Based Automatic Essay Evaluation of ENEM Essays. Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{silveira2026neuro,
  title = {{Neuro-symbolic Approaches for Rubric-Based Automatic Essay Evaluation of ENEM Essays}},
  author = {Igor Cataneo Silveira and Denis Deratani Mauá},
  booktitle = {Proceedings of the 17th International Conference on Computational Processing of Portuguese (PROPOR 2026) - Vol. 1},
  year = {2026},
  url = {https://aclanthology.org/2026.propor-1.78/}
}
```

- [← previous NeuralFSM: Adaptive Multi-Agent Coordination via Learning Finite-State Execution Policy](https://md.research.elamur.ai/free/en/papers/wang-2026-neuralfsm-adaptive-multi-agent-coordination/)
- [next → Neuronal Insights into LLM Attacks: Targeted Neuron Tuning for Precise and Robust Vulnerability Patching](https://md.research.elamur.ai/free/en/papers/shi-2026-neuronal-insights-into-llm-attacks/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/43/)

## References

26 references, 1 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Evelin Amorim and Adriano Veloso. 2017. A multi-aspect analysis of automatic essay scoring for Brazilian Portuguese. In Proceedings of the Student Research Workshop at the 15th Conference of the European Chapter of the Association for Computational Linguistics, pages 94–102. Association for Computat… [find](https://scholar.google.com/scholar?q=Evelin+Amorim+and+Adriano+Veloso%2E+2017%2E+A+multi%2Daspect+analysis+of+automatic+essay+scoring+for+Brazilian+Portuguese%2E+In+Proceedings+of+the+Student+Research+Workshop+at+the+15th+Conference+of+the+Europ)
2. Rafael Anchiêta, Anthony Luz, Shara Lopes, and Raimundo Moura. 2025. A zero-shot prompting approach for automated feedback generation on enem Renato Lui Geh, Jonas Gonçalves, Igor C. Silveira, Denis D. Mauá, and Fabio G. Cozman. 2024. dPASP: A Probabilistic Logic Programming Environment For Neurosym… [find](https://scholar.google.com/scholar?q=Rafael+Anchi%C3%AAta%2C+Anthony+Luz%2C+Shara+Lopes%2C+and+Raimundo+Moura%2E+2025%2E+A+zero%2Dshot+prompting+approach+for+automated+feedback+generation+on+enem+Renato+Lui+Geh%2C+Jonas+Gon%C3%A7alves%2C+Igor+C%2E+Silveira%2C+Denis+D)
3. Renato Lui Geh, Jonas Gonçalves, Igor Cataneo Silveira, Denis Deratani Mauá, and Fabio Gagliardi Cozman. 2023. dpasp: A comprehensive differentiable probabilistic answer set programming environment for neurosymbolic learning and reasoning. Preprint, arXiv:2308.02944. — [arXiv:2308.02944](https://arxiv.org/abs/2308.02944) [find](https://scholar.google.com/scholar?q=Renato+Lui+Geh%2C+Jonas+Gon%C3%A7alves%2C+Igor+Cataneo+Silveira%2C+Denis+Deratani+Mau%C3%A1%2C+and+Fabio+Gagliardi+Cozman%2E+2023%2E+dpasp%3A+A+comprehensive+differentiable+probabilistic+answer+set+programming+environment+fo)
4. Jiani Huang, Ziyang Li, Binghong Chen, Karan Samel, Mayur Naik, Le Song, and Xujie Si. 2021. Scallop: From probabilistic deductive databases to scalable differentiable reasoning. In Advances in Neural Information Processing Systems, volume 34, page 25134–25145. Curran Associates, Inc. [find](https://scholar.google.com/scholar?q=Jiani+Huang%2C+Ziyang+Li%2C+Binghong+Chen%2C+Karan+Samel%2C+Mayur+Naik%2C+Le+Song%2C+and+Xujie+Si%2E+2021%2E+Scallop%3A+From+probabilistic+deductive+databases+to+scalable+differentiable+reasoning%2E+In+Advances+in+Neural)
5. Shiyu Ji, Farnoosh Hashemi, Joice Chen, Juanwen Pan, Weicheng Ma, Hefan Zhang, Sophia Pan, Ming Cheng, Shubham Mohole, Saeed Hassanpour, Soroush Vosoughi, and Michael Macy. 2025. A generalizable rhetorical strategy annotation model using LLM-based debate simulation and labelling. In Findings of the… [find](https://scholar.google.com/scholar?q=Shiyu+Ji%2C+Farnoosh+Hashemi%2C+Joice+Chen%2C+Juanwen+Pan%2C+Weicheng+Ma%2C+Hefan+Zhang%2C+Sophia+Pan%2C+Ming+Cheng%2C+Shubham+Mohole%2C+Saeed+Hassanpour%2C+Soroush+Vosoughi%2C+and+Michael+Macy%2E+2025%2E+A+generalizable+rheto)
6. Sidney Evaldo Leal, Magali Sanches Duran, Carolina Evaristo Scarton, Nathan Siegle Hartmann, and Sandra Maria Aluísio. 2024. NILC-Metrix: Assessing the Complexity of Written and Spoken Language in Brazilian Portuguese. Language Resources and Evaluation, 58(1):73–110. [find](https://scholar.google.com/scholar?q=Sidney+Evaldo+Leal%2C+Magali+Sanches+Duran%2C+Carolina+Evaristo+Scarton%2C+Nathan+Siegle+Hartmann%2C+and+Sandra+Maria+Alu%C3%ADsio%2E+2024%2E+NILC%2DMetrix%3A+Assessing+the+Complexity+of+Written+and+Spoken+Language+in+Bra)
7. Ziyang Li, Jiani Huang, and Mayur Naik. 2023. Scallop: A language for neurosymbolic programming. Proc. ACM Program. Lang., 7(PLDI). [find](https://scholar.google.com/scholar?q=Ziyang+Li%2C+Jiani+Huang%2C+and+Mayur+Naik%2E+2023%2E+Scallop%3A+A+language+for+neurosymbolic+programming%2E+Proc%2E+ACM+Program%2E+Lang%2E%2C+7%28PLDI%29%2E)
8. Jiawei Liu, Yang Xu, and Yaguang Zhu. 2019. Automated essay scoring based on two-stage learning. Preprint, arXiv:1901.07744. — [arXiv:1901.07744](https://arxiv.org/abs/1901.07744) [find](https://scholar.google.com/scholar?q=Jiawei+Liu%2C+Yang+Xu%2C+and+Yaguang+Zhu%2E+2019%2E+Automated+essay+scoring+based+on+two%2Dstage+learning%2E+Preprint%2C+arXiv%3A1901%2E07744%2E+%E2%80%94+%5BarXiv%3A1901%2E07744%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1901%2E07744%29)
9. Robin Manhaeve, Sebastijan Dumancic, Angelika Kimmig, Thomas Demeester, and Luc De Raedt. 2018. [Deepproblog: Neural probabilistic logic programming](https://md.research.elamur.ai/free/en/papers/manhaeve-2018-deepproblog-neural-probabilistic-logic/). In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc.
10. Robin Manhaeve, Sebastijan Dumančić, Angelika Kimmig, Thomas Demeester, and Luc De Raedt. 2021. Neural probabilistic logic programming in deepproblog. Artificial Intelligence, 298:103504. [find](https://scholar.google.com/scholar?q=Robin+Manhaeve%2C+Sebastijan+Dumanc%CC%8Cic%CC%81%2C+Angelika+Kimmig%2C+Thomas+Demeester%2C+and+Luc+De+Raedt%2E+2021%2E+Neural+probabilistic+logic+programming+in+deepproblog%2E+Artificial+Intelligence%2C+298%3A103504%2E)
11. Jeziel Marinho, Rafael Anchiêta, and Raimundo Moura. 2021. Essay-br: a brazilian corpus of essays. In Anais do III Dataset Showcase Workshop, pages 53– 64. Sociedade Brasileira de Computação. [find](https://scholar.google.com/scholar?q=Jeziel+Marinho%2C+Rafael+Anchi%C3%AAta%2C+and+Raimundo+Moura%2E+2021%2E+Essay%2Dbr%3A+a+brazilian+corpus+of+essays%2E+In+Anais+do+III+Dataset+Showcase+Workshop%2C+pages+53%E2%80%93+64%2E+Sociedade+Brasileira+de+Computa%C3%A7%C3%A3o%2E)
12. Jeziel Marinho, Rafael Anchiêta, and Raimundo Moura. 2022a. Essay-br: a brazilian corpus to automatic essay scoring task. Journal of Information and Data Management, 13(1):65–76. [find](https://scholar.google.com/scholar?q=Jeziel+Marinho%2C+Rafael+Anchi%C3%AAta%2C+and+Raimundo+Moura%2E+2022a%2E+Essay%2Dbr%3A+a+brazilian+corpus+to+automatic+essay+scoring+task%2E+Journal+of+Information+and+Data+Management%2C+13%281%29%3A65%E2%80%9376%2E)
13. Jeziel Marinho, Fábio Cordeiro, Rafael Anchiêta, and Raimundo Moura. 2022b. Automated essay scoring: An approach based on enem competencies. In Anais do XIX Encontro Nacional de Inteligência Artificial e Computacional, pages 49–60. [find](https://scholar.google.com/scholar?q=Jeziel+Marinho%2C+F%C3%A1bio+Cordeiro%2C+Rafael+Anchi%C3%AAta%2C+and+Raimundo+Moura%2E+2022b%2E+Automated+essay+scoring%3A+An+approach+based+on+enem+competencies%2E+In+Anais+do+XIX+Encontro+Nacional+de+Intelig%C3%AAncia+Artificia)
14. Rafael Ferreira Mello, Hilário Oliveira, Moésio Wenceslau, Hyan Batista, Thiago Cordeiro, Ig Ibert Bittencourt, and Seiji Isotanif. 2024. PROPOR’24 Competition on Automatic Essay Scoring of Portuguese Narrative Essays. In Proceedings of the 16th International Conference on Computational Processing o… [find](https://scholar.google.com/scholar?q=Rafael+Ferreira+Mello%2C+Hil%C3%A1rio+Oliveira%2C+Mo%C3%A9sio+Wenceslau%2C+Hyan+Batista%2C+Thiago+Cordeiro%2C+Ig+Ibert+Bittencourt%2C+and+Seiji+Isotanif%2E+2024%2E+PROPOR%E2%80%9924+Competition+on+Automatic+Essay+Scoring+of+Portuguese)
15. Hilário Oliveira, Rafael Ferreira Mello, Péricles Miranda, Hyan Batista, Moésio Wenceslau Silva da Filho, Thiago Cordeiro, Ig Ibert Bittencourt, and Seiji Isotani. 2025. A benchmark dataset of narrative student essays with multi-competency grades for automatic essay scoring in brazilian portuguese.… [find](https://scholar.google.com/scholar?q=Hil%C3%A1rio+Oliveira%2C+Rafael+Ferreira+Mello%2C+P%C3%A9ricles+Miranda%2C+Hyan+Batista%2C+Mo%C3%A9sio+Wenceslau+Silva+da+Filho%2C+Thiago+Cordeiro%2C+Ig+Ibert+Bittencourt%2C+and+Seiji+Isotani%2E+2025%2E+A+benchmark+dataset+of+narrati)
16. Ellis B. Page. 1966. The imminence of... grading essays by computer. The Phi Delta Kappan, pages 238–243. [find](https://scholar.google.com/scholar?q=Ellis+B%2E+Page%2E+1966%2E+The+imminence+of%2E%2E%2E+grading+essays+by+computer%2E+The+Phi+Delta+Kappan%2C+pages+238%E2%80%93243%2E)
17. Eugénio Ribeiro, Nuno Mamede, and Jorge Baptista. 2024a. Automatic text readability assessment in European Portuguese. In Proceedings of the 16th International Conference on Computational Processing of Portuguese - Vol. 1, pages 97–107, Santiago de Compostela, Galicia/Spain. Association for Computat… [find](https://scholar.google.com/scholar?q=Eug%C3%A9nio+Ribeiro%2C+Nuno+Mamede%2C+and+Jorge+Baptista%2E+2024a%2E+Automatic+text+readability+assessment+in+European+Portuguese%2E+In+Proceedings+of+the+16th+International+Conference+on+Computational+Processing+o)
18. Eugénio Ribeiro, Nuno Mamede, and Jorge Baptista. 2024b. Exploring the Automated Scoring of Narrative Essays in Brazilian Portuguese using Transformer Models. In Proceedings of the 16th International Conference on Computational Processing of Portuguese-Vol. 2, pages 14–17. [find](https://scholar.google.com/scholar?q=Eug%C3%A9nio+Ribeiro%2C+Nuno+Mamede%2C+and+Jorge+Baptista%2E+2024b%2E+Exploring+the+Automated+Scoring+of+Narrative+Essays+in+Brazilian+Portuguese+using+Transformer+Models%2E+In+Proceedings+of+the+16th+International+)
19. Victor Hugo Nascimento Rocha, Igor Cataneo Silveira, Paulo Pirozelli, Denis Deratani Mauá, and Fabio Gagliardi Cozman. 2023. Assessing good, bad and ugly arguments generated by chatgpt: a new dataset, its methodology and associated tasks. In Progress in Artificial Intelligence: 22nd EPIA Conference… [find](https://scholar.google.com/scholar?q=Victor+Hugo+Nascimento+Rocha%2C+Igor+Cataneo+Silveira%2C+Paulo+Pirozelli%2C+Denis+Deratani+Mau%C3%A1%2C+and+Fabio+Gagliardi+Cozman%2E+2023%2E+Assessing+good%2C+bad+and+ugly+arguments+generated+by+chatgpt%3A+a+new+dataset%2C)
20. Mark D Shermis and Jill Burstein. 2013. Handbook of automated essay evaluation. NY: Routledge. [find](https://scholar.google.com/scholar?q=Mark+D+Shermis+and+Jill+Burstein%2E+2013%2E+Handbook+of+automated+essay+evaluation%2E+NY%3A+Routledge%2E)
21. Igor Cataneo Silveira, André Barbosa, Daniel Silva Lopes da Costa, and Denis Deratani Mauá. 2025a. Investigating Universal Adversarial Attacks Against Transformers-Based Automatic Essay Scoring Systems. In Intelligent Systems, pages 169–183, Cham. Springer Nature Switzerland. [find](https://scholar.google.com/scholar?q=Igor+Cataneo+Silveira%2C+Andr%C3%A9+Barbosa%2C+Daniel+Silva+Lopes+da+Costa%2C+and+Denis+Deratani+Mau%C3%A1%2E+2025a%2E+Investigating+Universal+Adversarial+Attacks+Against+Transformers%2DBased+Automatic+Essay+Scoring+System)
22. Igor Cataneo Silveira, André Barbosa, and Denis Deratani Mauá. 2024. A new benchmark for automatic essay scoring in Portuguese. In Proceedings of the 16th International Conference on Computational Processing of Portuguese - Vol. 1, pages 228–237. [find](https://scholar.google.com/scholar?q=Igor+Cataneo+Silveira%2C+Andr%C3%A9+Barbosa%2C+and+Denis+Deratani+Mau%C3%A1%2E+2024%2E+A+new+benchmark+for+automatic+essay+scoring+in+Portuguese%2E+In+Proceedings+of+the+16th+International+Conference+on+Computational+Pro)
23. Igor Cataneo Silveira, Eugénio Ribeiro, Nuno Mamede, and Jorge Baptista. 2025b. Aprendizado por transferência para correçao automática de redaçao. Linguamática, 17(2):99–116. [find](https://scholar.google.com/scholar?q=Igor+Cataneo+Silveira%2C+Eug%C3%A9nio+Ribeiro%2C+Nuno+Mamede%2C+and+Jorge+Baptista%2E+2025b%2E+Aprendizado+por+transfer%C3%AAncia+para+corre%C3%A7ao+autom%C3%A1tica+de+reda%C3%A7ao%2E+Linguam%C3%A1tica%2C+17%282%29%3A99%E2%80%93116%2E)
24. Fábio Souza, Rodrigo Nogueira, and Roberto Lotufo. 2020. BERTimbau: pretrained BERT models for Brazilian Portuguese. In 9th Brazilian Conference on Intelligent Systems, BRACIS, Rio Grande do Sul, Brazil, October 20-23 (to appear). [find](https://scholar.google.com/scholar?q=F%C3%A1bio+Souza%2C+Rodrigo+Nogueira%2C+and+Roberto+Lotufo%2E+2020%2E+BERTimbau%3A+pretrained+BERT+models+for+Brazilian+Portuguese%2E+In+9th+Brazilian+Conference+on+Intelligent+Systems%2C+BRACIS%2C+Rio+Grande+do+Sul%2C+Braz)
25. Jingyi Xu, Zilu Zhang, Tal Friedman, Yitao Liang, and Guy Van den Broeck. 2018. A semantic loss function for deep learning with symbolic knowledge. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pages 5502–5511. PMLR. [find](https://scholar.google.com/scholar?q=Jingyi+Xu%2C+Zilu+Zhang%2C+Tal+Friedman%2C+Yitao+Liang%2C+and+Guy+Van+den+Broeck%2E+2018%2E+A+semantic+loss+function+for+deep+learning+with+symbolic+knowledge%2E+In+Proceedings+of+the+35th+International+Conference+)
26. Zhun Yang, Adam Ishay, and Joohyung Lee. 2020. Neurasp: Embracing neural networks into answer set programming. In Proceedings of the Twenty-Ninth International Joint Conference on Artificial Intelligence, IJCAI-20, pages 1755–1762. International Joint Conferences on Artificial Intelligence Organizat… [find](https://scholar.google.com/scholar?q=Zhun+Yang%2C+Adam+Ishay%2C+and+Joohyung+Lee%2E+2020%2E+Neurasp%3A+Embracing+neural+networks+into+answer+set+programming%2E+In+Proceedings+of+the+Twenty%2DNinth+International+Joint+Conference+on+Artificial+Intellige)

*Updated 1 November 2026*