---
title: "LiTEx — paper by Hong et al. (2025) — summary and ideas"
description: "The authors propose LiTEx, a taxonomy for English free-text NLI explanations. 70.2 accuracy. 35 references, cited by 7 papers here."
authors: ["Pingjun Hong", "Beiduo Chen", "Siyao Peng", "Marie-Catherine de Marneffe", "Barbara Plank"]
year: 2025
venue: "Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing"
source: "https://aclanthology.org/2025.emnlp-main.1728/"
doi: "10.18653/v1/2025.emnlp-main.1728"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/hong-2025-litex-linguistic-taxonomy-explanations-understanding/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/hong-2025-litex-linguistic-taxonomy-explanations-understanding/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2025](https://md.research.elamur.ai/free/en/papers/2025/page/9/) / LiTEx: A Linguistic Taxonomy of Explanations for Understanding Within-Label Variation in Natural Language Inference

# LiTEx: A Linguistic Taxonomy of Explanations for Understanding Within-Label Variation in Natural Language Inference

**Authors:** [Pingjun Hong](https://md.research.elamur.ai/free/en/authors/pingjun-hong/), [Beiduo Chen](https://md.research.elamur.ai/free/en/authors/beiduo-chen/), [Siyao Peng](https://md.research.elamur.ai/free/en/authors/siyao-peng/), [Marie-Catherine de Marneffe](https://md.research.elamur.ai/free/en/authors/marie-catherine-de-marneffe/) and [Barbara Plank](https://md.research.elamur.ai/free/en/authors/barbara-plank/) · **Year:** 2025 · **Venue:** [Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2025](https://md.research.elamur.ai/free/en/venues/emnlp/2025/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2025.emnlp-main.1728/) · [PDF](https://aclanthology.org/2025.emnlp-main.1728.pdf) · [DOI 10.18653/v1/2025.emnlp-main.1728](https://doi.org/10.18653/v1/2025.emnlp-main.1728) · [OpenAlex](https://openalex.org/W4416035048)

**Citations:** 1 citations (OpenAlex, 2026-10-14) · [7](https://md.research.elamur.ai/free/en/papers/hong-2025-litex-linguistic-taxonomy-explanations-understanding/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Research on NLI disagreement usually studies annotators who choose different labels. This paper targets a less visible case: annotators who pick the same label but justify it in different ways. Highlighted spans, the usual structured form of explanation, cannot show this. Identical spans can hide different arguments, and different spans can express one argument. The authors propose LiTEx, a taxonomy for English free-text NLI explanations. It has two families: text-based reasoning with six subtypes and world-knowledge reasoning with two. Each category comes with a guiding question and decision criteria. They label a subset of e-SNLI and test reliability through inter-annotator agreement and automatic classification. They also examine how the categories relate to labels, highlights and explanation similarity. Finally, they prompt LLMs to write explanations given only the label, the label plus highlights, or the label plus a LiTEx category. Category-conditioned outputs sit closer to human explanations in embedding space and cover more reasoning types. The work makes reasoning diversity measurable and controllable, and it releases e-SNLI enriched with category labels.

## Key points
- The eight categories are split into two families. The Text-Based family contains six of them: Syntactic, Coreference, Logic Conflict, Semantic, Absence of Mention, and Pragmatic. World-Knowledge has two: Factual Knowledge and Inferential Knowledge.
- Explanations were classified into the taxonomy automatically.
  - Fine-tuned BERT-base reached 70.2 accuracy and 57.8 F1, and RoBERTa-base reached 68.9 accuracy and 50.4 F1.
  - Random and majority baselines scored 12.5 and 31.3 accuracy.
  - LLMs such as Llama-3.2-3B-Instruct and gpt-3.5-turbo were also tested with few-shot prompting.
- Generation experiments compared GPT4o, DeepSeek and Llama, each asked to produce all plausible explanations.
  - GPT4o averaged 3.59 explanations per item, against 5.90 for DeepSeek and 6.14 for Llama.
  - This partly explains GPT4o's lower coverage.
- Highlight-guided outputs occupied compact regions of embedding space but missed core human ideas. Generated explanations were generally longer than human ones.
- The authors list three limitations: categories were assigned by a single annotator, only explicit verbalized reasoning is covered, and experiments use e-SNLI only. Data and code are at github.com/mainlp/LiTEx.

## Interesting ideas
- Each category is defined by a yes/no guiding question plus a concrete check, which makes borderline labelling repeatable for humans and models alike.
- The taxonomy cleanly separates reasoning grounded only in the premise/hypothesis text from reasoning that needs outside facts or cultural norms.
- Conditioning an LLM on a reasoning category, rather than on highlighted spans, yields more varied and more human-like explanations.
- Generated explanation sets are compared to human sets by coverage in embedding space, with the caveat that producing more outputs inflates coverage.
- The paper draws an explicit boundary between a logical-exclusivity argument ("cannot be both tall and short") and a plain lexical-contrast argument ("tall and short differ").

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (answer): Each answer justification could be tagged with such a question-based type, so a reader can see what kind of step produced the answer.
- **I2** (world): Every inferred state or relation could be marked as derived from the text graph or from compiled knowledge (WordNet, Wikidata, norms), keeping provenance explicit.
- **I3** (learn): LLM teacher tiers could be prompted once per reasoning category to collect diverse silver rationales instead of one generic justification.
- **I4** (learn): Coverage-style metrics should be normalized for output count, with a control system that simply over-generates.
- **I5** (frames): Contradiction detection could distinguish antonymy found in lexical relations from incompatibility of two states on the same entity, and report which one fired.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Ludwig-Maximilians-Universität München](https://md.research.elamur.ai/free/en/universities/ludwig-maximilians-universitat-munchen/) — University, Munich, Germany · 107 works · 56,478 citations
- [UCLouvain](https://md.research.elamur.ai/free/en/universities/uclouvain/) — University, Louvain-la-Neuve, Belgium · 20 works · 3,741 citations
- [Munich Center for Machine Learning](https://md.research.elamur.ai/free/en/universities/munich-center-for-machine-learning/) — University, Munich, Germany · 57 works · 1,094 citations
- [University of Vienna](https://md.research.elamur.ai/free/en/universities/university-of-vienna/) — University, Vienna, Austria · 16 works · 38,729 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works

## Cite

Cite as: Pingjun Hong, Beiduo Chen, Siyao Peng, Marie-Catherine de Marneffe and Barbara Plank. 2025. LiTEx: A Linguistic Taxonomy of Explanations for Understanding Within-Label Variation in Natural Language Inference. Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{hong2025litex,
  title = {{LiTEx: A Linguistic Taxonomy of Explanations for Understanding Within-Label Variation in Natural Language Inference}},
  author = {Pingjun Hong, Beiduo Chen, Siyao Peng, Marie-Catherine {de Marneffe} and Barbara Plank},
  booktitle = {Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing},
  year = {2025},
  doi = {10.18653/v1/2025.emnlp-main.1728},
  url = {https://doi.org/10.18653/v1/2025.emnlp-main.1728}
}
```

- [← previous Leveraging Allophony in Self-Supervised Speech Models for Atypical Pronunciation Assessment](https://md.research.elamur.ai/free/en/papers/choi-2025-leveraging-allophony-self-supervised-speech/)
- [next → LogicPro: Improving Complex Logical Reasoning via Program-Guided Learning](https://md.research.elamur.ai/free/en/papers/jiang-2025-logicpro-improving-complex-logical-reasoning/)
- [↑ back to papers of 2025](https://md.research.elamur.ai/free/en/papers/2025/page/9/)

## References

35 references, 23 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel Ziegler, Jeffrey Wu, Clemens Winter, and 12 others. 2020. [Language models are few-shot learners](https://md.research.elamur.ai/free/en/papers/brown-2020-language-models-few-shot-learners/). In Advances in Neural Information Processing Systems, volume 33, pages 1877–1901. Curran Associates, Inc.
2. Federico Cabitza, Andrea Campagner, and Valerio Basile. 2023. [Toward a perspectivist turn in ground truthing for predictive computing](https://md.research.elamur.ai/free/en/papers/basile-2021-toward-perspectivist-turn-ground-truthing/). Proceedings of the AAAI Conference on Artificial Intelligence, 37(6):6860–6868.
3. Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. [e-SNLI: Natural language inference with natural language explanations](https://md.research.elamur.ai/free/en/papers/camburu-2018-e-snli-natural-language-inference/). In Advances in Neural Information Processing Systems, volume 31. Curran Associates, Inc. [↗](https://doi.org/10.48550/arxiv.1812.01193)
4. Beiduo Chen, Yang Janet Liu, Anna Korhonen, and Barbara Plank. 2025a. Threading the needle: Reweaving chain-of-thought reasoning to explain human label variation. CoRR, abs/2505.23368. [↗](https://arxiv.org/abs/2505.23368)
5. Beiduo Chen, Siyao Peng, Anna Korhonen, and Barbara Plank. 2025b. [A rose by any other name: LLM-generated explanations are good proxies for human explanations to collect label distributions on NLI](https://md.research.elamur.ai/free/en/papers/chen-2025-rose-any-other-name-llm/). In Findings of the Association for Computational Linguistics: ACL 2025, pages 10777–10802, Vienna, Austria. Association for Computational Linguistics.
6. Beiduo Chen, Xinpeng Wang, Siyao Peng, Robert Litschko, Anna Korhonen, and Barbara Plank. 2024. [“Seeing the big through the small”: Can LLMs approximate human judgment distributions on NLI from a few explanations?](https://md.research.elamur.ai/free/en/papers/chen-2024-seeing-big-through-small-can/) In Findings of the Association for Computational Linguistics: EMNLP 2024, pages 14396–14419, Miami, Florida, USA. Association for Computational Linguistics.
7. Ernest Davis. 2017. Logical formalizations of commonsense reasoning: A survey. J. Artif. Int. Res., 59(1):651–723. [↗](https://doi.org/10.1613/jair.5339)
8. Marie-Catherine de Marneffe, Christopher D. Manning, and Christopher Potts. 2012. [Did it happen? The pragmatic complexity of veridicality assessment](https://md.research.elamur.ai/free/en/papers/marneffe-2012-did-it-happen-pragmatic-complexity/). Computational Linguistics, 38(2):301–333.
9. DeepSeek-AI, Aixin Liu, Bei Feng, Bing Xue, Bingxuan Wang, Bochao Wu, Chengda Lu, Chenggang Zhao, Chengqi Deng, Chenyu Zhang, Chong Ruan, Damai Dai, Daya Guo, Dejian Yang, Deli Chen, Dongjie Ji, Erhang Li, Fangyun Lin, Fucong Dai, and 80 others. 2024. Deepseek-v3 technical report. CoRR, abs/2412.19437. [↗](https://arxiv.org/abs/2412.19437)
10. Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. [BERT: Pre-training of deep bidirectional transformers for language understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers), pages 4171–4186. Association for Computational Linguistics.
11. Mario Giulianelli, Joris Baan, Wilker Aziz, Raquel Fernández, and Barbara Plank. 2023. What comes next? evaluating uncertainty in neural text generators against human production variability. In Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing, pages 14349–14371, Singapore. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Mario+Giulianelli%2C+Joris+Baan%2C+Wilker+Aziz%2C+Raquel+Fern%C3%A1ndez%2C+and+Barbara+Plank%2E+2023%2E+What+comes+next%3F+evaluating+uncertainty+in+neural+text+generators+against+human+production+variability%2E+In+Procee)
12. Ari Holtzman, Jan Buys, Li Du, Maxwell Forbes, and Yejin Choi. 2020. [The curious case of neural text degeneration](https://md.research.elamur.ai/free/en/papers/holtzman-2019-curious-case-neural-text-degeneration/). In International Conference on Learning Representations. [↗](https://doi.org/10.48550/arxiv.1904.09751)
13. Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. 2022. [LoRA: Low-rank adaptation of large language models](https://md.research.elamur.ai/free/en/papers/hu-2021-lora-low-rank-adaptation-large/). In International Conference on Learning Representations.
14. Shiyuan Huang, Siddarth Mamidanna, Shreedhar Jangam, Yilun Zhou, and Leilani H. Gilpin. 2023. Can large language models explain themselves? A study of LLM-generated self-explanations. CoRR, abs/2310.11207. [↗](https://arxiv.org/abs/2310.11207)
15. Filip Ilievski, Alessandro Oltramari, Kaixin Ma, Bin Zhang, Deborah L. McGuinness, and Pedro Szekely. 2021. Dimensions of commonsense knowledge. Knowledge-Based Systems, 229:107347. [↗](https://doi.org/10.1016/j.knosys.2021.107347)
16. Chathuri Jayaweera and Bonnie Dorr. 2025. [From disagreement to understanding: The case for ambiguity detection in NLI](https://md.research.elamur.ai/free/en/papers/jayaweera-2025-disagreement-understanding-case-ambiguity-detection/). CoRR, abs/2507.15114. [↗](https://arxiv.org/abs/2507.15114)
17. Nan-Jiang Jiang and Marie-Catherine de Marneffe. 2022. [Investigating reasons for disagreement in natural language inference](https://md.research.elamur.ai/free/en/papers/jiang-2022-investigating-reasons-disagreement-natural-language/). Transactions of the Association for Computational Linguistics, 10:1357–1374. [↗](https://doi.org/10.1162/tacl_a_00523)
18. Nan-Jiang Jiang, Chenhao Tan, and Marie-Catherine de Marneffe. 2023. [Ecologically valid explanations for label variation in NLI](https://md.research.elamur.ai/free/en/papers/jiang-2023-ecologically-valid-explanations-label-variation/). In Findings of the Association for Computational Linguistics: EMNLP 2023, pages 10622–10633. Association for Computational Linguistics.
19. Shiyang Li, Jianshu Chen, yelong shen, Zhiyu Chen, Xinlu Zhang, Zekun Li, Hong Wang, Jing Qian, Baolin Peng, Yi Mao, Wenhu Chen, and Xifeng Yan. 2024. Explanations from large language models make small reasoners better. In 2nd Workshop on Sustainable AI. [find](https://scholar.google.com/scholar?q=Shiyang+Li%2C+Jianshu+Chen%2C+yelong+shen%2C+Zhiyu+Chen%2C+Xinlu+Zhang%2C+Zekun+Li%2C+Hong+Wang%2C+Jing+Qian%2C+Baolin+Peng%2C+Yi+Mao%2C+Wenhu+Chen%2C+and+Xifeng+Yan%2E+2024%2E+Explanations+from+large+language+models+make+smal)
20. Chin-Yew Lin. 2004. [ROUGE: A package for automatic evaluation of summaries](https://md.research.elamur.ai/free/en/papers/lin-2004-rouge-package-automatic-evaluation-summaries/). In Text Summarization Branches Out, pages 74–81, Barcelona, Spain. Association for Computational Linguistics.
21. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [RoBERTa: A robustly optimized BERT pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). CoRR, abs/1907.11692. [↗](https://arxiv.org/abs/1907.11692)
22. Peter LoBue and Alexander Yates. 2011. [Types of common-sense knowledge needed for recognizing textual entailment](https://md.research.elamur.ai/free/en/papers/lobue-2011-types-common-sense-knowledge-needed/). In Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics: Human Language Technologies, pages 329–334, Portland, Oregon, USA. Association for Computational Linguistics.
23. Meta. 2024. Introducing Meta Llama 3: The most capable openly available LLM to date. [find](https://scholar.google.com/scholar?q=Meta%2E+2024%2E+Introducing+Meta+Llama+3%3A+The+most+capable+openly+available+LLM+to+date%2E)
24. Yixin Nie, Adina Williams, Emily Dinan, Mohit Bansal, Jason Weston, and Douwe Kiela. 2020a. [Adversarial NLI: A new benchmark for natural language understanding](https://md.research.elamur.ai/free/en/papers/nie-2020-adversarial-nli-new-benchmark-natural/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4885–4901, Online. Association for Computational Linguistics.
25. Yixin Nie, Xiang Zhou, and Mohit Bansal. 2020b. [What can we learn from collective human opinions on natural language inference data?](https://md.research.elamur.ai/free/en/papers/nie-2020-what-can-we-learn-collective/) In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9131–9143, Online. Association for Computational Linguistics.
26. OpenAI. 2023. [GPT-4 technical report](https://md.research.elamur.ai/free/en/papers/openai-2023-gpt-4-technical-report/). abs/2303.08774. [↗](https://arxiv.org/abs/2303.08774)
27. CoRR, Kishore Papineni, Salim Roukos, Todd Ward, and Wei-Jing Zhu. 2002. [Bleu: A method for automatic evaluation of machine translation](https://md.research.elamur.ai/free/en/papers/papineni-2002-bleu-method-automatic-evaluation-machine/). In Proceedings of the 40th Annual Meeting of the Association for Computational Linguistics, pages 311–318, Philadelphia, Pennsylvania, USA. Association for Computational Linguistics.
28. Barbara Plank. 2022. [The “problem” of human label variation: On ground truth in data, modeling and evaluation](https://md.research.elamur.ai/free/en/papers/plank-2022-problem-human-label-variation-ground/). In Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing, pages 10671–10682, Abu Dhabi, United Arab Emirates. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/2022.emnlp-main.731)
29. Nils Reimers and Iryna Gurevych. 2019. [Sentence-BERT: Sentence embeddings using Siamese BERT-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 3982–3992, Hong Kong, China. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/d19-1410)
30. Mark Sammons, V.G.Vinod Vydiswaran, and Dan Roth. 2010. [“Ask not what textual entailment can do for you...”](https://md.research.elamur.ai/free/en/papers/sammons-2010-ask-not-what-textual-entailment/). In Proceedings of the 48th Annual Meeting of the Association for Computational Linguistics, pages 1199–1208, Uppsala, Sweden. Association for Computational Linguistics.
31. Mandy Simons, Judith Tonhauser, David Beaver, and Craige Roberts. 2011. What projects and why. Proceedings of SALT; Vol 20 (2010); 309-327, 20. [↗](https://doi.org/10.3765/salt.v20i0.2584)
32. Chenhao Tan. 2022. On the diversity and limits of human explanations. In Proceedings of the 2022 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pages 2173–2188, Seattle, United States. Association for Computational Linguistics. [↗](https://doi.org/10.48448/aphg-ya28)
33. Alexandra N. Uma, Tommaso Fornaciari, Dirk Hovy, Silviu Paun, Barbara Plank, and Massimo Poesio. 2022. Learning from disagreement: A survey. J. Artif. Int. Res., 72:1385–1470. [↗](https://doi.org/10.1613/jair.1.12752)
34. Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of Machine Learning Research, 9(86):2579–2605. [find](https://scholar.google.com/scholar?q=Laurens+van+der+Maaten+and+Geoffrey+Hinton%2E+2008%2E+Visualizing+data+using+t%2DSNE%2E+Journal+of+Machine+Learning+Research%2C+9%2886%29%3A2579%E2%80%932605%2E)
35. Leon Weber-Genzel, Siyao Peng, Marie-Catherine De Marneffe, and Barbara Plank. 2024. [VariErr NLI: Separating annotation error from human label variation](https://md.research.elamur.ai/free/en/papers/genzel-2024-varierr-nli-separating-annotation-error/). In Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 2256–2269, Bangkok, Thailand. Association for Computational Linguistics.

*Updated 1 November 2026*