---
title: "LIBERTy — paper by Toker et al. (2026) — summary and ideas"
description: "The authors propose LIBERTy, a framework that defines an explicit structural causal model (SCM) of text generation. 71 references."
authors: ["Gilat Toker", "Nitay Calderon", "Ohad Amosy", "Roi Reichart"]
year: 2026
venue: "Findings of the Association for Computational Linguistics: ACL 2026"
source: "https://aclanthology.org/2026.findings-acl.399/"
doi: "10.18653/v1/2026.findings-acl.399"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/toker-2026-liberty-causal-framework-benchmarking-concept/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/toker-2026-liberty-causal-framework-benchmarking-concept/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2026](https://md.research.elamur.ai/free/en/papers/2026/page/39/) / LIBERTy: A Causal Framework for Benchmarking Concept-Based Explanations of LLMs with Structural Counterfactuals

# LIBERTy: A Causal Framework for Benchmarking Concept-Based Explanations of LLMs with Structural Counterfactuals

**Authors:** Gilat Toker, [Nitay Calderon](https://md.research.elamur.ai/free/en/authors/nitay-calderon/), Ohad Amosy and [Roi Reichart](https://md.research.elamur.ai/free/en/authors/roi-reichart/) · **Year:** 2026 · **Venue:** [Findings of the Association for Computational Linguistics: ACL 2026](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2026](https://md.research.elamur.ai/free/en/venues/findings/2026/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2026.findings-acl.399/) · [PDF](https://aclanthology.org/2026.findings-acl.399.pdf) · [DOI 10.18653/v1/2026.findings-acl.399](https://doi.org/10.18653/v1/2026.findings-acl.399) · [OpenAlex](https://openalex.org/W7166845935)

**Citations:** 0 citations (OpenAlex, 2026-10-14)

## Summary
Judging whether a concept-based explanation of an LLM is faithful requires the true causal effect of a concept, such as gender or experience, on the model's output. That effect is normally unknown. Earlier benchmarks such as CEBaB approximate it with human-written counterfactuals of short restaurant reviews, using a small, flat causal graph. The authors propose LIBERTy, a framework that defines an explicit structural causal model (SCM) of text generation. In it, an LLM turns concept values into text, and templated real-world text and author personas act as exogenous noise. A concept is intervened on and the change is propagated through the SCM, with noise held fixed and decoding at temperature zero. This produces structural counterfactuals and reference causal effects without human annotation. The authors release three datasets and a new metric, then benchmark many explanation methods on five models. Matching on representations from a dataset-specific fine-tuned model performs best, but large room for improvement remains. The resource offers a scalable, causally grounded testbed for building more faithful explainability methods.

## Key points
- **Datasets:** three domains (disease detection, CV screening, workplace violence prediction), each built from a hand-specified causal graph grounded in domain knowledge.
- **New metric:** order-faithfulness checks whether a method ranks concept-intervention effects correctly. This makes it usable for methods whose importance scores are on arbitrary scales.
- **Methods compared:** approaches that remove concepts with linear projections, produce counterfactual texts, pair similar examples, or assign importance scores to concepts, all evaluated against individual-level (ICaCE) and population-level (CaCE) effects.
- **Model sensitivity:** zero-shot LLMs (Llama-3.1-8B, GPT-4o) barely react to interventions on Race, Gender and Age, plausibly due to post-training alignment. Among fine-tuned models, Qwen2.5-1.5B tracks the gold effects most closely, though a gap remains.
- **Data quality and code:** a human evaluation found the synthetic texts coherent and fluent, with counterfactuals differing in only one concept. Code is available at github.com/GilatToker/Liberty-benchmark.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Technion – Israel Institute of Technology](https://md.research.elamur.ai/free/en/universities/technion-israel-institute-of-technology/) — University, Haifa, Israel · 54 works · 7,237 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 18,614 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 17,965 works · [🤝 Social Sciences](https://md.research.elamur.ai/free/en/shelves/social-sciences/) · 1,954 works · [🤝 Safety Research](https://md.research.elamur.ai/free/en/shelves/social-sciences/safety-research/) · 626 works

## Cite

Cite as: Gilat Toker, Nitay Calderon, Ohad Amosy and Roi Reichart. 2026. LIBERTy: A Causal Framework for Benchmarking Concept-Based Explanations of LLMs with Structural Counterfactuals. Findings of the Association for Computational Linguistics: ACL 2026.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{toker2026liberty,
  title = {{LIBERTy: A Causal Framework for Benchmarking Concept-Based Explanations of LLMs with Structural Counterfactuals}},
  author = {Gilat Toker, Nitay Calderon, Ohad Amosy and Roi Reichart},
  booktitle = {Findings of the Association for Computational Linguistics: ACL 2026},
  year = {2026},
  doi = {10.18653/v1/2026.findings-acl.399},
  url = {https://doi.org/10.18653/v1/2026.findings-acl.399}
}
```

- [← previous LGSE: Lexically Grounded Subword Embedding Initialization for Low-Resource Language Adaptation](https://md.research.elamur.ai/free/en/papers/teklehaymanot-2026-lgse-lexically-grounded-subword-embedding/)
- [next → Libras-UFPel Corpus: A Parallel Dataset of Brazilian Sign Language and Portuguese for Multimodal Research and Processing](https://md.research.elamur.ai/free/en/papers/martins-2026-libras-ufpel-corpus-parallel-dataset/)
- [↑ back to papers of 2026](https://md.research.elamur.ai/free/en/papers/2026/page/39/)

## References

71 references, 13 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Eldar David Abraham, Karel D’Oosterlinck, Amir Feder, Yair Ori Gat, Atticus Geiger, Christopher Potts, Roi Reichart, and Zhengxuan Wu. 2022. Cebab: Estimating the causal effects of real-world concepts on NLP model behavior. CoRR, abs/2205.14140. [find](https://scholar.google.com/scholar?q=Eldar+David+Abraham%2C+Karel+D%E2%80%99Oosterlinck%2C+Amir+Feder%2C+Yair+Ori+Gat%2C+Atticus+Geiger%2C+Christopher+Potts%2C+Roi+Reichart%2C+and+Zhengxuan+Wu%2E+2022%2E+Cebab%3A+Estimating+the+causal+effects+of+real%2Dworld+concepts)
2. Ahmed Alqaraawi, Martin Schuessler, Philipp Weiß, Enrico Costanza, and Nadia Berthouze. 2020. Evaluating saliency map explanations for convolutional neural networks: A user study. CoRR, abs/2002.00772. — [arXiv:2002.00772](https://arxiv.org/abs/2002.00772) [find](https://scholar.google.com/scholar?q=Ahmed+Alqaraawi%2C+Martin+Schuessler%2C+Philipp+Wei%C3%9F%2C+Enrico+Costanza%2C+and+Nadia+Berthouze%2E+2020%2E+Evaluating+saliency+map+explanations+for+convolutional+neural+networks%3A+A+user+study%2E+CoRR%2C+abs%2F2002%2E00772)
3. Esma Balkir, Svetlana Kiritchenko, Isar Nejadgholi, and Kathleen C. Fraser. 2022. Challenges in applying explainability methods to improve the fairness of NLP models. CoRR, abs/2206.03945. [find](https://scholar.google.com/scholar?q=Esma+Balkir%2C+Svetlana+Kiritchenko%2C+Isar+Nejadgholi%2C+and+Kathleen+C%2E+Fraser%2E+2022%2E+Challenges+in+applying+explainability+methods+to+improve+the+fairness+of+NLP+models%2E+CoRR%2C+abs%2F2206%2E03945%2E)
4. Nora Belrose, David Schneider-Joseph, Shauli Ravfogel, Ryan Cotterell, Edward Raff, and Stella Biderman. 2023. LEACE: perfect linear concept erasure in closed form. CoRR, abs/2306.03819. — [arXiv:2306.03819](https://arxiv.org/abs/2306.03819) [find](https://scholar.google.com/scholar?q=Nora+Belrose%2C+David+Schneider%2DJoseph%2C+Shauli+Ravfogel%2C+Ryan+Cotterell%2C+Edward+Raff%2C+and+Stella+Biderman%2E+2023%2E+LEACE%3A+perfect+linear+concept+erasure+in+closed+form%2E+CoRR%2C+abs%2F2306%2E03819%2E+%E2%80%94+%5BarXiv%3A2306)
5. Jin Du, Li Chen, Xun Xian, An Luo, Fangqiao Tian, Ganghua Wang, Charles Doss, Xiaotong Shen, and Jie Ding. 2025. Ice cream doesn’t cause drowning: Benchmarking llms against statistical pitfalls in causal inference. CoRR, abs/2505.13770. — [arXiv:2505.13770](https://arxiv.org/abs/2505.13770) [find](https://scholar.google.com/scholar?q=Jin+Du%2C+Li+Chen%2C+Xun+Xian%2C+An+Luo%2C+Fangqiao+Tian%2C+Ganghua+Wang%2C+Charles+Doss%2C+Xiaotong+Shen%2C+and+Jie+Ding%2E+2025%2E+Ice+cream+doesn%E2%80%99t+cause+drowning%3A+Benchmarking+llms+against+statistical+pitfalls+in+cau)
6. Kenza Benkirane, Jackie Kay, and María Pérez-Ortiz. 2024. How can we diagnose and treat bias in large language models for clinical decision-making? CoRR, abs/2410.16574. [find](https://scholar.google.com/scholar?q=Kenza+Benkirane%2C+Jackie+Kay%2C+and+Mar%C3%ADa+P%C3%A9rez%2DOrtiz%2E+2024%2E+How+can+we+diagnose+and+treat+bias+in+large+language+models+for+clinical+decision%2Dmaking%3F+CoRR%2C+abs%2F2410%2E16574%2E)
7. Abhimanyu Dubey, Abhinav Jauhri, Abhinav Pandey, Abhishek Kadian, Ahmad Al-Dahle, Aiesha Letman, Akhil Mathur, Alan Schelten, Amy Yang, Angela Fan, Anirudh Goyal, Anthony Hartshorn, Aobo Yang, Archi Mitra, Archie Sravankumar, Artem Korenev, Arthur Hinsvark, Arun Rao, Aston Zhang, and 82 others. 2024… — [The Llama 3 Herd of Models](https://md.research.elamur.ai/free/en/papers/grattafiori-2024-llama-3-herd-models/)
8. Diane Bouchacourt and Ludovic Denoyer. 2019. EDUCE: explaining model decisions through unsupervised concepts extraction. CoRR, abs/1905.11852. — [arXiv:1905.11852](https://arxiv.org/abs/1905.11852) [find](https://scholar.google.com/scholar?q=Diane+Bouchacourt+and+Ludovic+Denoyer%2E+2019%2E+EDUCE%3A+explaining+model+decisions+through+unsupervised+concepts+extraction%2E+CoRR%2C+abs%2F1905%2E11852%2E+%E2%80%94+%5BarXiv%3A1905%2E11852%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1905%2E11852%29)
9. Amir Feder, Nadav Oved, Uri Shalit, and Roi Reichart. 2021. [Causalm: Causal model explanation through counterfactual language models](https://md.research.elamur.ai/free/en/papers/feder-2021-causalm-causal-model-explanation-through/). Comput. Linguistics, 47(2):333–386.
10. Roger K Cady and Curtis P Schreiber. 2002. Sinus headache or migraine? considerations in making a differential diagnosis. Neurology, 58(9_suppl_6):S10–S14. [find](https://scholar.google.com/scholar?q=Roger+K+Cady+and+Curtis+P+Schreiber%2E+2002%2E+Sinus+headache+or+migraine%3F+considerations+in+making+a+differential+diagnosis%2E+Neurology%2C+58%289%5Fsuppl%5F6%29%3AS10%E2%80%93S14%2E)
11. Yair Ori Gat, Nitay Calderon, Amir Feder, Alexander Chapanin, Amit Sharma, and Roi Reichart. 2023. Faithful explanations of black-box NLP models using llm-generated counterfactuals. CoRR, abs/2310.00603. — [arXiv:2310.00603](https://arxiv.org/abs/2310.00603) [find](https://scholar.google.com/scholar?q=Yair+Ori+Gat%2C+Nitay+Calderon%2C+Amir+Feder%2C+Alexander+Chapanin%2C+Amit+Sharma%2C+and+Roi+Reichart%2E+2023%2E+Faithful+explanations+of+black%2Dbox+NLP+models+using+llm%2Dgenerated+counterfactuals%2E+CoRR%2C+abs%2F2310%2E006)
12. Nitay Calderon, Liat Ein-Dor, and Roi Reichart. 2025. Multi-domain explainability of preferences. CoRR, abs/2505.20088. [find](https://scholar.google.com/scholar?q=Nitay+Calderon%2C+Liat+Ein%2DDor%2C+and+Roi+Reichart%2E+2025%2E+Multi%2Ddomain+explainability+of+preferences%2E+CoRR%2C+abs%2F2505%2E20088%2E)
13. Nitay Calderon and Roi Reichart. 2024. On behalf of the stakeholders: Trends in NLP model interpretability in the era of llms. CoRR, abs/2407.19200. [find](https://scholar.google.com/scholar?q=Nitay+Calderon+and+Roi+Reichart%2E+2024%2E+On+behalf+of+the+stakeholders%3A+Trends+in+NLP+model+interpretability+in+the+era+of+llms%2E+CoRR%2C+abs%2F2407%2E19200%2E)
14. Fateme Hashemi Chaleshtori, Atreya Ghosal, Alexander Gill, Purbid Bambroo, and Ana Marasovic. 2024. On evaluating explanation utility for human-ai decision making in NLP. CoRR, abs/2407.03545. [find](https://scholar.google.com/scholar?q=Fateme+Hashemi+Chaleshtori%2C+Atreya+Ghosal%2C+Alexander+Gill%2C+Purbid+Bambroo%2C+and+Ana+Marasovic%2E+2024%2E+On+evaluating+explanation+utility+for+human%2Dai+decision+making+in+NLP%2E+CoRR%2C+abs%2F2407%2E03545%2E)
15. Ivi Chatzi, Nina L. Corvelo Benz, Eleni Straitouri, Stratis Tsirtsis, and Manuel Gomez-Rodriguez. 2025. Counterfactual token generation in large language models. In Causal Learning and Reasoning, Lausanne, Switzerland, 7-9 May 2025, volume 275 of Proceedings of Machine Learning Research, pages 1291–… [find](https://scholar.google.com/scholar?q=Ivi+Chatzi%2C+Nina+L%2E+Corvelo+Benz%2C+Eleni+Straitouri%2C+Stratis+Tsirtsis%2C+and+Manuel+Gomez%2DRodriguez%2E+2025%2E+Counterfactual+token+generation+in+large+language+models%2E+In+Causal+Learning+and+Reasoning%2C+Laus)
16. Giandomenico Cornacchia, Vito Walter Anelli, Fedelucio Narducci, Azzurra Ragone, and Eugenio Di Sciascio. 2023. Counterfactual reasoning for bias evaluation and detection in a fairness under unawareness setting. CoRR, abs/2302.08204. — [arXiv:2302.08204](https://arxiv.org/abs/2302.08204) [find](https://scholar.google.com/scholar?q=Giandomenico+Cornacchia%2C+Vito+Walter+Anelli%2C+Fedelucio+Narducci%2C+Azzurra+Ragone%2C+and+Eugenio+Di+Sciascio%2E+2023%2E+Counterfactual+reasoning+for+bias+evaluation+and+detection+in+a+fairness+under+unawarene)
17. Bo Cowgill, Fabrizio Dell’Acqua, Samuel Deng, Daniel Hsu, Nakul Verma, and Augustin Chaintreau. 2020. Biased programmers? or biased data? A field experiment in operationalizing AI ethics. CoRR, abs/2012.02394. — [arXiv:2012.02394](https://arxiv.org/abs/2012.02394) [find](https://scholar.google.com/scholar?q=Bo+Cowgill%2C+Fabrizio+Dell%E2%80%99Acqua%2C+Samuel+Deng%2C+Daniel+Hsu%2C+Nakul+Verma%2C+and+Augustin+Chaintreau%2E+2020%2E+Biased+programmers%3F+or+biased+data%3F+A+field+experiment+in+operationalizing+AI+ethics%2E+CoRR%2C+abs%2F20)
18. Fahim Dalvi, Abdul Rafae Khan, Firoj Alam, Nadir Durrani, Jia Xu, and Hassan Sajjad. 2022. Discovering latent concepts learned in BERT. CoRR, abs/2205.07237. — [arXiv:2205.07237](https://arxiv.org/abs/2205.07237) [find](https://scholar.google.com/scholar?q=Fahim+Dalvi%2C+Abdul+Rafae+Khan%2C+Firoj+Alam%2C+Nadir+Durrani%2C+Jia+Xu%2C+and+Hassan+Sajjad%2E+2022%2E+Discovering+latent+concepts+learned+in+BERT%2E+CoRR%2C+abs%2F2205%2E07237%2E+%E2%80%94+%5BarXiv%3A2205%2E07237%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs)
19. Jeffrey Dastin. 2018. Amazon scrapped ‘ai’ recruiting tool that showed bias against women. [find](https://scholar.google.com/scholar?q=Jeffrey+Dastin%2E+2018%2E+Amazon+scrapped+%E2%80%98ai%E2%80%99+recruiting+tool+that+showed+bias+against+women%2E)
20. S Gerberich, T Church, P McGovern, and et al. 2004. An epidemiological study of the magnitude and consequences of work related violence: the minnesota nurses’ study. Occupational and Environmental Medicine, 61(6):495–503. [find](https://scholar.google.com/scholar?q=S+Gerberich%2C+T+Church%2C+P+McGovern%2C+and+et+al%2E+2004%2E+An+epidemiological+study+of+the+magnitude+and+consequences+of+work+related+violence%3A+the+minnesota+nurses%E2%80%99+study%2E+Occupational+and+Environmental+Med)
21. Amirata Ghorbani, James Wexler, James Y. Zou, and Been Kim. 2019. Towards automatic concept-based explanations. In Advances in Neural Information Processing Systems 32: Annual Conference on Neural Information Processing Systems 2019, NeurIPS 2019, December 8-14, 2019, Vancouver, BC, Canada, pages 92… [find](https://scholar.google.com/scholar?q=Amirata+Ghorbani%2C+James+Wexler%2C+James+Y%2E+Zou%2C+and+Been+Kim%2E+2019%2E+Towards+automatic+concept%2Dbased+explanations%2E+In+Advances+in+Neural+Information+Processing+Systems+32%3A+Annual+Conference+on+Neural+Inf)
22. Yash Goyal, Uri Shalit, and Been Kim. 2019. Explaining classifiers with causal concept effect (cace). CoRR, abs/1907.07165. — [arXiv:1907.07165](https://arxiv.org/abs/1907.07165) [find](https://scholar.google.com/scholar?q=Yash+Goyal%2C+Uri+Shalit%2C+and+Been+Kim%2E+2019%2E+Explaining+classifiers+with+causal+concept+effect+%28cace%29%2E+CoRR%2C+abs%2F1907%2E07165%2E+%E2%80%94+%5BarXiv%3A1907%2E07165%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1907%2E07165%29)
23. Riccardo Guidotti, Anna Monreale, Franco Turini, Dino Pedreschi, and Fosca Giannotti. 2018. A survey of methods for explaining black box models. CoRR, abs/1802.01933. — [arXiv:1802.01933](https://arxiv.org/abs/1802.01933) [find](https://scholar.google.com/scholar?q=Riccardo+Guidotti%2C+Anna+Monreale%2C+Franco+Turini%2C+Dino+Pedreschi%2C+and+Fosca+Giannotti%2E+2018%2E+A+survey+of+methods+for+explaining+black+box+models%2E+CoRR%2C+abs%2F1802%2E01933%2E+%E2%80%94+%5BarXiv%3A1802%2E01933%5D%28https%3A%2F%2Farxi)
24. Sai Gurrapu, Ajay Kulkarni, Lifu Huang, Ismini Lourentzou, and Feras A. Batarseh. 2023. Rationalization for explainable NLP: a survey. Frontiers Artif. Intell., 6. [find](https://scholar.google.com/scholar?q=Sai+Gurrapu%2C+Ajay+Kulkarni%2C+Lifu+Huang%2C+Ismini+Lourentzou%2C+and+Feras+A%2E+Batarseh%2E+2023%2E+Rationalization+for+explainable+NLP%3A+a+survey%2E+Frontiers+Artif%2E+Intell%2E%2C+6%2E)
25. Peter Hase and Mohit Bansal. 2020. [Evaluating explainable AI: which algorithmic explanations help users predict model behavior?](https://md.research.elamur.ai/free/en/papers/hase-2020-evaluating-explainable-ai-which-algorithmic/) CoRR, abs/2005.01831.
26. Pengcheng He, Xiaodong Liu, Jianfeng Gao, and Weizhu Chen. 2020. [Deberta: Decoding-enhanced BERT with disentangled attention](https://md.research.elamur.ai/free/en/papers/he-2020-deberta-decoding-enhanced-bert-disentangled/). CoRR, abs/2006.03654.
27. Anna Hedström, Philine Lou Bommer, Kristoffer Knutsen Wickstrøm, Wojciech Samek, Sebastian Lapuschkin, and Marina M.-C. Höhne. 2023. The meta-evaluation problem in explainable AI: identifying reliable estimators with metaquantus. [find](https://scholar.google.com/scholar?q=Anna+Hedstr%C3%B6m%2C+Philine+Lou+Bommer%2C+Kristoffer+Knutsen+Wickstr%C3%B8m%2C+Wojciech+Samek%2C+Sebastian+Lapuschkin%2C+and+Marina+M%2E%2DC%2E+H%C3%B6hne%2E+2023%2E+The+meta%2Devaluation+problem+in+explainable+AI%3A+identifying+reliable)
28. Trans. Mach. Learn. Res., 2023. [find](https://scholar.google.com/scholar?q=Trans%2E+Mach%2E+Learn%2E+Res%2E%2C+2023%2E)
29. XinYue Jiang, Jingsong He, and Li Gu. 2025. MTCR: method for matching texts against causal relationship. Neural Process. Lett., 57(3):58. [find](https://scholar.google.com/scholar?q=XinYue+Jiang%2C+Jingsong+He%2C+and+Li+Gu%2E+2025%2E+MTCR%3A+method+for+matching+texts+against+causal+relationship%2E+Neural+Process%2E+Lett%2E%2C+57%283%29%3A58%2E)
30. Enkelejda Kasneci, Kathrin Sessler, Stefan Küchemann, Maria Bannert, Daryna Dementieva, Frank Fischer, Urs Gasser, Georg Groh, Stephan Günnemann, Eyke Hüllermeier, Stephan Krusche, Gitta Kutyniok, Tilman Michaeli, Claudia Nerdel, Jürgen Pfeffer, Oleksandra Poquet, Michael Sailer, Albrecht Schmidt, T… — [ChatGPT for Good? On Opportunities and Challenges of Large Language Models for Education](https://md.research.elamur.ai/free/en/papers/kasneci-2023-chatgpt-good-opportunities-challenges-large/)
31. Been Kim, Martin Wattenberg, Justin Gilmer, Carrie J. Cai, James Wexler, Fernanda B. Viégas, and Rory Sayres. 2018. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (TCAV). In Proceedings of the 35th International Conference on Machine Learning, ICML… [find](https://scholar.google.com/scholar?q=Been+Kim%2C+Martin+Wattenberg%2C+Justin+Gilmer%2C+Carrie+J%2E+Cai%2C+James+Wexler%2C+Fernanda+B%2E+Vi%C3%A9gas%2C+and+Rory+Sayres%2E+2018%2E+Interpretability+beyond+feature+attribution%3A+Quantitative+testing+with+concept+activ)
32. Sunnie S. Y. Kim, Elizabeth Anne Watkins, Olga Russakovsky, Ruth Fong, and Andrés Monroy-Hernández. 2022. "help me help the ai": Understanding how explainability can support human-ai interaction. CoRR, abs/2210.03735. — [arXiv:2210.03735](https://arxiv.org/abs/2210.03735) [find](https://scholar.google.com/scholar?q=Sunnie+S%2E+Y%2E+Kim%2C+Elizabeth+Anne+Watkins%2C+Olga+Russakovsky%2C+Ruth+Fong%2C+and+Andr%C3%A9s+Monroy%2DHern%C3%A1ndez%2E+2022%2E+%22help+me+help+the+ai%22%3A+Understanding+how+explainability+can+support+human%2Dai+interaction%2E+CoRR)
33. Pang Wei Koh, Thao Nguyen, Yew Siang Tang, Stephen Mussmann, Emma Pierson, Been Kim, and Percy Liang. 2020. Concept bottleneck models. In Proceedings of the 37th International Conference on Machine Learning, ICML 2020, 13-18 July 2020, Virtual Event, volume 119 of Proceedings of Machine Learning Res… [find](https://scholar.google.com/scholar?q=Pang+Wei+Koh%2C+Thao+Nguyen%2C+Yew+Siang+Tang%2C+Stephen+Mussmann%2C+Emma+Pierson%2C+Been+Kim%2C+and+Percy+Liang%2E+2020%2E+Concept+bottleneck+models%2E+In+Proceedings+of+the+37th+International+Conference+on+Machine+Le)
34. Sören R. Künzel, Jasjeet S. Sekhon, Peter J. Bickel, and Bin Yu. 2019. Meta-learners for estimating heterogeneous treatment effects using machine learning. arXiv. [find](https://scholar.google.com/scholar?q=S%C3%B6ren+R%2E+K%C3%BCnzel%2C+Jasjeet+S%2E+Sekhon%2C+Peter+J%2E+Bickel%2C+and+Bin+Yu%2E+2019%2E+Meta%2Dlearners+for+estimating+heterogeneous+treatment+effects+using+machine+learning%2E+arXiv%2E)
35. Jun Rui Lee, Sadegh Emami, Michael David Hollins, Timothy C. H. Wong, Carlos Ignacio Villalobos Sánchez, Francesca Toni, Dekai Zhang, and Adam Dejl. 2025. Xai-units: Benchmarking explainability methods with unit tests. CoRR, abs/2506.01059. — [arXiv:2506.01059](https://arxiv.org/abs/2506.01059) [find](https://scholar.google.com/scholar?q=Jun+Rui+Lee%2C+Sadegh+Emami%2C+Michael+David+Hollins%2C+Timothy+C%2E+H%2E+Wong%2C+Carlos+Ignacio+Villalobos+S%C3%A1nchez%2C+Francesca+Toni%2C+Dekai+Zhang%2C+and+Adam+Dejl%2E+2025%2E+Xai%2Dunits%3A+Benchmarking+explainability+method)
36. Yongqi Li, Mayi Xu, Xin Miao, Shen Zhou, and Tieyun Qian. 2024. Prompting large language models for counterfactual generation: An empirical study. In Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation, LREC/COLING 2024, 20-25 May, 2… [find](https://scholar.google.com/scholar?q=Yongqi+Li%2C+Mayi+Xu%2C+Xin+Miao%2C+Shen+Zhou%2C+and+Tieyun+Qian%2E+2024%2E+Prompting+large+language+models+for+counterfactual+generation%3A+An+empirical+study%2E+In+Proceedings+of+the+2024+Joint+International+Confer)
37. Scott M. Lundberg and Su-In Lee. 2017. [A unified approach to interpreting model predictions](https://md.research.elamur.ai/free/en/papers/lundberg-2017-unified-approach-interpreting-model-predictions/). CoRR, abs/1705.07874.
38. Siwen Luo, Hamish Ivison, Soyeon Caren Han, and Josiah Poon. 2024. Local interpretations for explainable natural language processing: A survey. ACM Comput. Surv., 56(9):232:1–232:36. [find](https://scholar.google.com/scholar?q=Siwen+Luo%2C+Hamish+Ivison%2C+Soyeon+Caren+Han%2C+and+Josiah+Poon%2E+2024%2E+Local+interpretations+for+explainable+natural+language+processing%3A+A+survey%2E+ACM+Comput%2E+Surv%2E%2C+56%289%29%3A232%3A1%E2%80%93232%3A36%2E)
39. Qing Lyu, Marianna Apidianaki, and Chris Callison-Burch. 2022. [Towards faithful model explanation in NLP: A survey](https://md.research.elamur.ai/free/en/papers/lyu-2024-faithful-model-explanation-nlp-survey/). CoRR, abs/2209.11326.
40. Arnold S Monto, Stefan Gravenstein, Michael Elliott, Michael Colopy, and Jo Schweinle. 2000. Clinical signs and symptoms predicting influenza infection. Archives of internal medicine, 160(21):3243–3247. [find](https://scholar.google.com/scholar?q=Arnold+S+Monto%2C+Stefan+Gravenstein%2C+Michael+Elliott%2C+Michael+Colopy%2C+and+Jo+Schweinle%2E+2000%2E+Clinical+signs+and+symptoms+predicting+influenza+infection%2E+Archives+of+internal+medicine%2C+160%2821%29%3A3243%E2%80%93324)
41. Yuqi Nie, Yaxuan Kong, Xiaowen Dong, John M. Mulvey, H. Vincent Poor, Qingsong Wen, and Stefan Zohren. 2024. A survey of large language models for financial applications: Progress, prospects and challenges. CoRR, abs/2406.11903. — [arXiv:2406.11903](https://arxiv.org/abs/2406.11903) [find](https://scholar.google.com/scholar?q=Yuqi+Nie%2C+Yaxuan+Kong%2C+Xiaowen+Dong%2C+John+M%2E+Mulvey%2C+H%2E+Vincent+Poor%2C+Qingsong+Wen%2C+and+Stefan+Zohren%2E+2024%2E+A+survey+of+large+language+models+for+financial+applications%3A+Progress%2C+prospects+and+chall)
42. OpenAI. 2024. Gpt-4o technical report. arXiv preprint. [find](https://scholar.google.com/scholar?q=OpenAI%2E+2024%2E+Gpt%2D4o+technical+report%2E+arXiv+preprint%2E)
43. Judea Pearl. 2009. Causality: Models, Reasoning, and Inference, 2 edition. Cambridge University Press. [find](https://scholar.google.com/scholar?q=Judea+Pearl%2E+2009%2E+Causality%3A+Models%2C+Reasoning%2C+and+Inference%2C+2+edition%2E+Cambridge+University+Press%2E)
44. Judea Pearl. 2013. Structural counterfactuals: A brief introduction. Cognitive science, 37(6):977–985. [find](https://scholar.google.com/scholar?q=Judea+Pearl%2E+2013%2E+Structural+counterfactuals%3A+A+brief+introduction%2E+Cognitive+science%2C+37%286%29%3A977%E2%80%93985%2E)
45. Lotem Peled-Cohen, Maya Zadok, Nitay Calderon, Hila Gonen, and Roi Reichart. 2025. Dementia through different eyes: Explainable modeling of human and LLM perceptions for early awareness. CoRR, abs/2505.13418. [find](https://scholar.google.com/scholar?q=Lotem+Peled%2DCohen%2C+Maya+Zadok%2C+Nitay+Calderon%2C+Hila+Gonen%2C+and+Roi+Reichart%2E+2025%2E+Dementia+through+different+eyes%3A+Explainable+modeling+of+human+and+LLM+perceptions+for+early+awareness%2E+CoRR%2C+abs%2F250)
46. Eleonora Poeta, Gabriele Ciravegna, Eliana Pastor, Tania Cerquitelli, and Elena Baralis. 2023. Concept-based explainable artificial intelligence: A survey. CoRR, abs/2312.12936. — [arXiv:2312.12936](https://arxiv.org/abs/2312.12936) [find](https://scholar.google.com/scholar?q=Eleonora+Poeta%2C+Gabriele+Ciravegna%2C+Eliana+Pastor%2C+Tania+Cerquitelli%2C+and+Elena+Baralis%2E+2023%2E+Concept%2Dbased+explainable+artificial+intelligence%3A+A+survey%2E+CoRR%2C+abs%2F2312%2E12936%2E+%E2%80%94+%5BarXiv%3A2312%2E12936%5D%28h)
47. Colin Raffel, Noam Shazeer, Adam Roberts, Katherine Lee, Sharan Narang, Michael Matena, Yanqi Zhou, Wei Li, and Peter J. Liu. 2020. [Exploring the limits of transfer learning with a unified text-to-text transformer](https://md.research.elamur.ai/free/en/papers/raffel-2019-exploring-limits-transfer-learning-unified/). J. Mach. Learn. Res., 21:140:1–140:67.
48. Manish Raghavan, Solon Barocas, Jon M. Kleinberg, and Karen Levy. 2020. Mitigating bias in algorithmic hiring: evaluating claims and practices. In FAT* ’20: Conference on Fairness, Accountability, and Transparency, Barcelona, Spain, January 27-30, 2020, pages 469–481. ACM. [find](https://scholar.google.com/scholar?q=Manish+Raghavan%2C+Solon+Barocas%2C+Jon+M%2E+Kleinberg%2C+and+Karen+Levy%2E+2020%2E+Mitigating+bias+in+algorithmic+hiring%3A+evaluating+claims+and+practices%2E+In+FAT%2A+%E2%80%9920%3A+Conference+on+Fairness%2C+Accountability%2C+and)
49. Shauli Ravfogel, Anej Svete, Vésteinn Snæbjarnarson, and Ryan Cotterell. 2025. Gumbel counterfactual generation from language models. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net. [find](https://scholar.google.com/scholar?q=Shauli+Ravfogel%2C+Anej+Svete%2C+V%C3%A9steinn+Sn%C3%A6bjarnarson%2C+and+Ryan+Cotterell%2E+2025%2E+Gumbel+counterfactual+generation+from+language+models%2E+In+The+Thirteenth+International+Conference+on+Learning+Representat)
50. Shauli Ravfogel, Michael Twiton, Yoav Goldberg, and Ryan Cotterell. 2022. Linear adversarial concept erasure. CoRR, abs/2201.12091. — [arXiv:2201.12091](https://arxiv.org/abs/2201.12091) [find](https://scholar.google.com/scholar?q=Shauli+Ravfogel%2C+Michael+Twiton%2C+Yoav+Goldberg%2C+and+Ryan+Cotterell%2E+2022%2E+Linear+adversarial+concept+erasure%2E+CoRR%2C+abs%2F2201%2E12091%2E+%E2%80%94+%5BarXiv%3A2201%2E12091%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2201%2E12091%29)
51. Nils Reimers and Iryna Gurevych. 2019. [Sentence-bert: Sentence embeddings using siamese bert-networks](https://md.research.elamur.ai/free/en/papers/reimers-2019-sentence-bert-sentence-embeddings-siamese/). In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, EMNLP-IJCNLP 2019, Hong Kong, Ch…
52. Marco Túlio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. ["why should I trust you?": Explaining the predictions of any classifier](https://md.research.elamur.ai/free/en/papers/ribeiro-2016-why-should-i-trust-explaining/). CoRR, abs/1602.04938.
53. Marcel Robeer, Floris Bex, and Ad Feelders. 2021. Generating realistic natural language counterfactuals. In Findings of the Association for Computational Linguistics: EMNLP 2021, Virtual Event / Punta Cana, Dominican Republic, 16-20 November, 2021, pages 3611–3625. Association for Computational Ling… [find](https://scholar.google.com/scholar?q=Marcel+Robeer%2C+Floris+Bex%2C+and+Ad+Feelders%2E+2021%2E+Generating+realistic+natural+language+counterfactuals%2E+In+Findings+of+the+Association+for+Computational+Linguistics%3A+EMNLP+2021%2C+Virtual+Event+%2F+Punta)
54. Wojciech Samek, Grégoire Montavon, Sebastian Lapuschkin, Christopher J. Anders, and Klaus-Robert Müller. 2021. Explaining deep neural networks and beyond: A review of methods and applications. Proc. IEEE, 109(3):247–278. [find](https://scholar.google.com/scholar?q=Wojciech+Samek%2C+Gr%C3%A9goire+Montavon%2C+Sebastian+Lapuschkin%2C+Christopher+J%2E+Anders%2C+and+Klaus%2DRobert+M%C3%BCller%2E+2021%2E+Explaining+deep+neural+networks+and+beyond%3A+A+review+of+methods+and+applications%2E+Proc%2E+I)
55. Pratinav Seth and Vinay Kumar Sankarapu. 2025. Bridging the gap in xai-why reliable metrics matter for explainability and compliance. CoRR, abs/2502.04695. — [arXiv:2502.04695](https://arxiv.org/abs/2502.04695) [find](https://scholar.google.com/scholar?q=Pratinav+Seth+and+Vinay+Kumar+Sankarapu%2E+2025%2E+Bridging+the+gap+in+xai%2Dwhy+reliable+metrics+matter+for+explainability+and+compliance%2E+CoRR%2C+abs%2F2502%2E04695%2E+%E2%80%94+%5BarXiv%3A2502%2E04695%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F2)
56. Ruihao Shui, Yixin Cao, Xiang Wang, and Tat-Seng Chua. 2023. A comprehensive evaluation of large language models on legal judgment prediction. CoRR, abs/2310.11761. [find](https://scholar.google.com/scholar?q=Ruihao+Shui%2C+Yixin+Cao%2C+Xiang+Wang%2C+and+Tat%2DSeng+Chua%2E+2023%2E+A+comprehensive+evaluation+of+large+language+models+on+legal+judgment+prediction%2E+CoRR%2C+abs%2F2310%2E11761%2E)
57. Qwen Team. 2023. Qwen: The official repo of qwen chat. [find](https://scholar.google.com/scholar?q=Qwen+Team%2E+2023%2E+Qwen%3A+The+official+repo+of+qwen+chat%2E)
58. James Thorne, Andreas Vlachos, Christos Christodoulopoulos, and Arpit Mittal. 2019. Generating token-level explanations for natural language inference. CoRR, abs/1904.10717. [find](https://scholar.google.com/scholar?q=James+Thorne%2C+Andreas+Vlachos%2C+Christos+Christodoulopoulos%2C+and+Arpit+Mittal%2E+2019%2E+Generating+token%2Dlevel+explanations+for+natural+language+inference%2E+CoRR%2C+abs%2F1904%2E10717%2E)
59. Victor Veitch, Dhanya Sridhar, and David M. Blei. 2020. Adapting text embeddings for causal inference. In Proceedings of the Thirty-Sixth Conference on Uncertainty in Artificial Intelligence, UAI 2020, virtual online, August 3-6, 2020, volume 124 of Proceedings of Machine Learning Research, pages 91… [find](https://scholar.google.com/scholar?q=Victor+Veitch%2C+Dhanya+Sridhar%2C+and+David+M%2E+Blei%2E+2020%2E+Adapting+text+embeddings+for+causal+inference%2E+In+Proceedings+of+the+Thirty%2DSixth+Conference+on+Uncertainty+in+Artificial+Intelligence%2C+UAI+2020)
60. Lijie Wang, Yaozong Shen, Shuyuan Peng, Shuai Zhang, Xinyan Xiao, Hao Liu, Hongxuan Tang, Ying Chen, Hua Wu, and Haifeng Wang. 2022. A fine-grained interpretability evaluation benchmark for neural NLP. CoRR, abs/2205.11097. [find](https://scholar.google.com/scholar?q=Lijie+Wang%2C+Yaozong+Shen%2C+Shuyuan+Peng%2C+Shuai+Zhang%2C+Xinyan+Xiao%2C+Hao+Liu%2C+Hongxuan+Tang%2C+Ying+Chen%2C+Hua+Wu%2C+and+Haifeng+Wang%2E+2022%2E+A+fine%2Dgrained+interpretability+evaluation+benchmark+for+neural+NLP)
61. Yongjie Wang, Xiaoqi Qiu, Yu Yue, Xu Guo, Zhiwei Zeng, Yuhong Feng, and Zhiqi Shen. 2024. A survey on natural language counterfactual generation. In Findings of the Association for Computational Linguistics: EMNLP 2024, Miami, Florida, USA, November 12-16, 2024, pages 4798–4818. Association for Comp… [find](https://scholar.google.com/scholar?q=Yongjie+Wang%2C+Xiaoqi+Qiu%2C+Yu+Yue%2C+Xu+Guo%2C+Zhiwei+Zeng%2C+Yuhong+Feng%2C+and+Zhiqi+Shen%2E+2024%2E+A+survey+on+natural+language+counterfactual+generation%2E+In+Findings+of+the+Association+for+Computational+Lingu)
62. Zach Wood-Doughty, Ilya Shpitser, and Mark Dredze. 2018. Challenges of using text classifiers for causal inference. CoRR, abs/1810.00956. [find](https://scholar.google.com/scholar?q=Zach+Wood%2DDoughty%2C+Ilya+Shpitser%2C+and+Mark+Dredze%2E+2018%2E+Challenges+of+using+text+classifiers+for+causal+inference%2E+CoRR%2C+abs%2F1810%2E00956%2E)
63. Tongshuang Wu, Marco Túlio Ribeiro, Jeffrey Heer, and Daniel S. Weld. 2021. [Polyjuice: Generating counterfactuals for explaining, evaluating, and improving models](https://md.research.elamur.ai/free/en/papers/wu-2021-polyjuice-generating-counterfactuals-explaining-evaluating/). In Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference o…
64. Zhengxuan Wu, Karel D’Oosterlinck, Atticus Geiger, Amir Zur, and Christopher Potts. 2022. Causal proxy models for concept-based model explanations. CoRR, abs/2209.14279. [find](https://scholar.google.com/scholar?q=Zhengxuan+Wu%2C+Karel+D%E2%80%99Oosterlinck%2C+Atticus+Geiger%2C+Amir+Zur%2C+and+Christopher+Potts%2E+2022%2E+Causal+proxy+models+for+concept%2Dbased+model+explanations%2E+CoRR%2C+abs%2F2209%2E14279%2E)
65. Zhengxuan Wu, Atticus Geiger, Christopher Potts, and Noah D. Goodman. 2023. [Interpretability at scale: Identifying causal mechanisms in alpaca](https://md.research.elamur.ai/free/en/papers/wu-2023-interpretability-scale-identifying-causal-mechanisms/). CoRR, abs/2305.08809.
66. Fan Yang, Mengnan Du, and Xia Hu. 2019. Evaluating explanation without ground truth in interpretable machine learning. CoRR, abs/1907.06831. — [arXiv:1907.06831](https://arxiv.org/abs/1907.06831) [find](https://scholar.google.com/scholar?q=Fan+Yang%2C+Mengnan+Du%2C+and+Xia+Hu%2E+2019%2E+Evaluating+explanation+without+ground+truth+in+interpretable+machine+learning%2E+CoRR%2C+abs%2F1907%2E06831%2E+%E2%80%94+%5BarXiv%3A1907%2E06831%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1907%2E06831%29)
67. Chih-Kuan Yeh, Been Kim, Sercan Ömer Arik, Chun-Liang Li, Tomas Pfister, and Pradeep Ravikumar. 2020. On completeness-aware concept-based explanations in deep neural networks. In Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, Ne… [find](https://scholar.google.com/scholar?q=Chih%2DKuan+Yeh%2C+Been+Kim%2C+Sercan+%C3%96mer+Arik%2C+Chun%2DLiang+Li%2C+Tomas+Pfister%2C+and+Pradeep+Ravikumar%2E+2020%2E+On+completeness%2Daware+concept%2Dbased+explanations+in+deep+neural+networks%2E+In+Advances+in+Neural+In)
68. Wei Jie Yeo, Ranjan Satapathy, and Erik Cambria. 2024. Towards faithful natural language explanations: A study using activation patching in large language models. CoRR, abs/2410.14155. [find](https://scholar.google.com/scholar?q=Wei+Jie+Yeo%2C+Ranjan+Satapathy%2C+and+Erik+Cambria%2E+2024%2E+Towards+faithful+natural+language+explanations%3A+A+study+using+activation+patching+in+large+language+models%2E+CoRR%2C+abs%2F2410%2E14155%2E)
69. Xuemin Yu, Fahim Dalvi, Nadir Durrani, and Hassan Sajjad. 2024. Latent concept-based explanation of NLP models. CoRR, abs/2404.12545. [find](https://scholar.google.com/scholar?q=Xuemin+Yu%2C+Fahim+Dalvi%2C+Nadir+Durrani%2C+and+Hassan+Sajjad%2E+2024%2E+Latent+concept%2Dbased+explanation+of+NLP+models%2E+CoRR%2C+abs%2F2404%2E12545%2E)
70. Raymond Zhang, Neha Nayak Kennard, Daniel Scott Smith, Daniel A. McFarland, Andrew McCallum, and Katherine Keith. 2023. Causal matching with text embeddings: A case study in estimating the causal effects of peer review policies. In Findings of the Association for Computational Linguistics: ACL 2023,… [find](https://scholar.google.com/scholar?q=Raymond+Zhang%2C+Neha+Nayak+Kennard%2C+Daniel+Scott+Smith%2C+Daniel+A%2E+McFarland%2C+Andrew+McCallum%2C+and+Katherine+Keith%2E+2023%2E+Causal+matching+with+text+embeddings%3A+A+case+study+in+estimating+the+causal+effe)
71. Haiyan Zhao, Hanjie Chen, Fan Yang, Ninghao Liu, Huiqi Deng, Hengyi Cai, Shuaiqiang Wang, Dawei Yin, and Mengnan Du. 2024. [Explainability for large language models: A survey](https://md.research.elamur.ai/free/en/papers/zhao-2023-explainability-large-language-models-survey/). ACM Trans. Intell. Syst. Technol., 15(2):20:1–20:38.

*Updated 1 November 2026*