---
title: "Interventional Probing in High Dimensions: An NLI Case Study"
description: "Probing classifiers can show that a feature is encoded in a model's hidden states, but they cannot… 19 references, cited by 4 papers here."
authors: ["Julia Rozanova", "Marco Valentino", "Lucas Cordeiro", "André Freitas"]
year: 2023
venue: "Findings of the Association for Computational Linguistics: EACL 2023"
source: "https://aclanthology.org/2023.findings-eacl.188/"
doi: "10.18653/v1/2023.findings-eacl.188"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/rozanova-2023-interventional-probing-high-dimensions-nli/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/rozanova-2023-interventional-probing-high-dimensions-nli/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2023](https://md.research.elamur.ai/free/en/papers/2023/page/12/) / Interventional Probing in High Dimensions: An NLI Case Study

# Interventional Probing in High Dimensions: An NLI Case Study

**Authors:** [Julia Rozanova](https://md.research.elamur.ai/free/en/authors/julia-rozanova/), [Marco Valentino](https://md.research.elamur.ai/free/en/authors/marco-valentino/), Lucas Cordeiro and [André Freitas](https://md.research.elamur.ai/free/en/authors/andre-freitas/) · **Year:** 2023 · **Venue:** [Findings of the Association for Computational Linguistics: EACL 2023](https://md.research.elamur.ai/free/en/venues/findings/) ([Findings of the Association for Computational Linguistics 2023](https://md.research.elamur.ai/free/en/venues/findings/2023/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2023.findings-eacl.188/) · [PDF](https://aclanthology.org/2023.findings-eacl.188.pdf) · [DOI 10.18653/v1/2023.findings-eacl.188](https://doi.org/10.18653/v1/2023.findings-eacl.188) · [OpenAlex](https://openalex.org/W4386576819)

**Citations:** 3 citations (OpenAlex, 2026-10-14) · 4 in our library, of the 149,439 papers whose bibliographies we read

## Summary
Probing classifiers can show that a feature is encoded in a model's hidden states, but they cannot show that the model actually relies on it. Interventional methods address this gap. The authors test them on natural logic, a controlled slice of NLI where the entailment label follows deterministically from two intermediate features: context monotonicity and the lexical relation between substituted noun phrases.

They apply amnesic probing to BERT and RoBERTa NLI classifiers. This method uses INLP to erase probe-detected directions. They also introduce a mirror-image variant called mnestic probing, which keeps only the probe-selected directions and discards everything else.

Amnesic probing produces contradictory verdicts. Erasing the two decisive features, their composite, or even the gold entailment label barely changes NLI accuracy. The authors attribute this to the tiny rank of the removed subspace relative to the very high embedding dimension, and to redundancy in the representations. Mnestic probing gives results that match theoretical expectations and separates model behaviours more clearly. The work warns against reading "no accuracy drop" as "feature unused."

## Key points
- **Setting.** The data are single-step natural logic inferences. A noun phrase is replaced by a hyponym, hypernym or unrelated phrase, inside an upward- or downward-monotone context shaped by negation, quantifiers or determiners.
- **Models.** The study covers BERT and RoBERTa classifiers fine-tuned for NLI, including a pair trained on MNLI and SNLI.
- **Features probed.** Four labels are probed: monotonicity, lexical relation, their composite, and the gold entailment label (used as a sanity check).
- **Method.** INLP iteratively trains linear SVM probes. Amnesic probing projects onto the intersection of their nullspaces. Mnestic probing projects onto the union of their rowspaces. Both are compared against removing or keeping randomly chosen directions.
- **Reproducibility.** Code and data are released at github.com/juliarozanova/mnestic_probing. The paper appears in Findings of EACL 2023, pp. 2489–2500.

## Interesting ideas
- Validate an interpretability method on a task whose causal structure is known in advance, so that a wrong verdict is detectable rather than just puzzling.
- Run the intervention on the target label itself as a sanity check: if erasing the answer does not hurt accuracy, the method is broken, not the model.
- In high-dimensional vectors, deleting a handful of random directions almost never matters, so random-removal baselines become uninformative when the probed feature spans few dimensions.
- The reverse intervention (keep only the probe-selected directions, then compare against equally sized random subspaces) turns a needle-in-haystack test into a low-rank one that is easier to interpret.
- Entailment in this fragment factorises cleanly into context monotonicity combined with the lexical relation between substituted terms.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Build small synthetic fragments where the correct answer is fully determined by known frame or world features, and use them to check that our explanation and ablation tools report the right dependencies.
- **I2** (learn): Add this as a standard negative control whenever we ablate features in gparse or the readers.
- **I3** (tree): When analysing gparse embeddings, do not trust "random ablation had no effect" as a baseline; size-match controls and check their alignment with the probe subspace.
- **I4** (tree): Restrict gparse token vectors to directions predicting POS or morphology and see how much attachment accuracy survives, which shows what the parser really leans on.
- **I5** (answer): A reader can compute these two signals explicitly, using polarity from the UD tree and hyponymy from WordNet, and combine them into an explainable entailment verdict instead of a black-box score.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Manchester](https://md.research.elamur.ai/free/en/universities/university-of-manchester/) — University, Manchester, United Kingdom · 87 works · 44,938 citations
- [Idiap Research Institute](https://md.research.elamur.ai/free/en/labs/idiap-research-institute/) — Research institute, Martigny-Combe, Switzerland · 32 works · 1,954 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 16,667 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 16,194 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 16,039 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 15,891 works

🧭 Topics (2): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 1,426 works · [🔬 Interpretability](https://md.research.elamur.ai/free/en/topics/interpretability/) · 130 works

## Cite

Cite as: Julia Rozanova, Marco Valentino, Lucas Cordeiro and André Freitas. 2023. Interventional Probing in High Dimensions: An NLI Case Study. Findings of the Association for Computational Linguistics: EACL 2023.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{rozanova2023interventional,
  title = {{Interventional Probing in High Dimensions: An NLI Case Study}},
  author = {Julia Rozanova, Marco Valentino, Lucas Cordeiro and André Freitas},
  booktitle = {Findings of the Association for Computational Linguistics: EACL 2023},
  year = {2023},
  doi = {10.18653/v1/2023.findings-eacl.188},
  url = {https://doi.org/10.18653/v1/2023.findings-eacl.188}
}
```

- [← previous How About Kind of Generating Hedges using End-to-End Neural Models?](https://md.research.elamur.ai/free/en/papers/abulimiti-2023-how-about-kind-generating-hedges/)
- [next → ivrit.ai: A Comprehensive Dataset of Hebrew Speech for AI Research and Development](https://md.research.elamur.ai/free/en/papers/marmor-2023-ivrit-ai-comprehensive-dataset-hebrew/)
- [↑ back to papers of 2023](https://md.research.elamur.ai/free/en/papers/2023/page/12/)

## References

19 references, 9 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Amir Feder, Nadav Oved, Uri Shalit, and Roi Reichart. 2021. [CausaLM: Causal model explanation through counterfactual language models](https://md.research.elamur.ai/free/en/papers/feder-2021-causalm-causal-model-explanation-through/). Computational Linguistics, 47(2):333–386. [↗](https://doi.org/10.1162/coli_a_00404)
2. Abubakar Abid, Mert Yuksekgonul, and James Zou. 2022. Meaningfully debugging model mistakes using conceptual counterfactual explanations. In Pro-Itai Gat, Guy Lorberbom, Idan Schwartz, and Tamir Hazan. 2021. Latent space explanation by intervention. [↗](https://doi.org/10.48550/arxiv.2106.12723)
3. Atticus Geiger, Hanson Lu, Thomas F Icard, and Christopher Potts. 2021. Causal abstractions of neural networks. In Advances in Neural Information Processing Systems. [↗](https://doi.org/10.48550/arxiv.2106.02997)
4. Atticus Geiger, Kyle Richardson, and Christopher Potts. 2020. [Neural natural language inference models partially embed theories of lexical entailment and negation](https://md.research.elamur.ai/free/en/papers/geiger-2020-neural-natural-language-inference-models/). In Proceedings of the Third BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pages 163–173, Online. Association for Computational Linguistics.
5. Mario Giulianelli, Jack Harding, Florian Mohnert, Dieuwke Hupkes, and Willem Zuidema. 2018. Under the hood: Using diagnostic classifiers to investigate and improve how language models track agreement information. In Proceedings of the 2018 EMNLP Workshop BlackboxNLP: Analyzing and Interpreting Neural Networks for NLP, pages 240–248, Brussels, Belgium. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/w18-5426)
6. Yash Goyal, Amir Feder, Uri Shalit, and Been Kim. 2019. Explaining classifiers with causal concept effect (cace). arXiv preprint arXiv:1907.07165. [↗](https://arxiv.org/abs/1907.07165)
7. John Hewitt and Percy Liang. 2019. Designing and interpreting probes with control tasks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP), pages 2733–2743, Hong Kong, China. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/d19-1275)
8. Been Kim, Martin Wattenberg, Justin Gilmer, Carrie Cai, James Wexler, Fernanda Viegas, et al. 2018. Interpretability beyond feature attribution: Quantitative testing with concept activation vectors (tcav). In International conference on machine learning, pages 2668–2677. PMLR. [↗](https://doi.org/10.48550/arxiv.1711.11279)
9. Y. Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, M. Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. [Roberta: A robustly optimized bert pretraining approach](https://md.research.elamur.ai/free/en/papers/liu-2019-roberta-robustly-optimized-bert-pretraining/). ArXiv, abs/1907.11692. [↗](https://arxiv.org/abs/1907.11692)
10. Bill MacCartney and Christopher D. Manning. 2007. Natural logic for textul inference. In Proceedings of the ACL-PASCAL Workshop on Textual Entailment and Paraphrasing, pages 193–200, Prague. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Bill+MacCartney+and+Christopher+D%2E+Manning%2E+2007%2E+Natural+logic+for+textul+inference%2E+In+Proceedings+of+the+ACL%2DPASCAL+Workshop+on+Textual+Entailment+and+Paraphrasing%2C+pages+193%E2%80%93200%2C+Prague%2E+Associati)
11. Tiago Pimentel, Josef Valvoda, Rowan Hall Maudslay, Ran Zmigrod, Adina Williams, and Ryan Cotterell. 2020. [Information-theoretic probing for linguistic structure](https://md.research.elamur.ai/free/en/papers/pimentel-2020-information-theoretic-probing-linguistic-structure/). In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pages 4609–4622, Online. Association for Computational Linguistics. the Association for Computational Linguistics, ACL 2020, Online, July 5-10, 2020, pages 7237–7256. Association for Computational Linguistics.
12. Abhilasha Ravichander, Yonatan Belinkov, and Eduard Hovy. 2021. [Probing the probing paradigm: Does probing accuracy entail task relevance?](https://md.research.elamur.ai/free/en/papers/ravichander-2021-probing-probing-paradigm-does-probing/) In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pages 3363–3377, Online. Association for Computational Linguistics.
13. Julia Rozanova, Deborah Ferreira, Mokanarangan Thayaparan, Marco Valentino, and André Freitas. 2021a. [Supporting context monotonicity abstractions in neural nli models](https://md.research.elamur.ai/free/en/papers/rozanova-2021-supporting-context-monotonicity-abstractions-neural/). [↗](https://doi.org/10.48550/arxiv.2105.08008)
14. Julia Rozanova, Deborah Ferreira, Marco Valentino, Mokanarangan Thayaparan, and André Freitas. 2021b. Decomposing natural logic inferences in neural NLI. CoRR, abs/2112.08289. [↗](https://arxiv.org/abs/2112.08289)
15. Elena Voita and Ivan Titov. 2020. [Information-theoretic probing with minimum description length](https://md.research.elamur.ai/free/en/papers/voita-2020-information-theoretic-probing-minimum-description/). In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 183–196, Online. Association for Computational Linguistics.
16. Adina Williams, Nikita Nangia, and Samuel Bowman. 2018. [A broad-coverage challenge corpus for sentence understanding through inference](https://md.research.elamur.ai/free/en/papers/williams-2018-broad-coverage-challenge-corpus-sentence/). In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long Papers), pages 1112–1122. Association for Computational Linguistics. [↗](https://doi.org/10.18653/v1/n18-1101)
17. Hitomi Yanaka, Koji Mineshima, Daisuke Bekki, Kentaro Inui, Satoshi Sekine, Lasha Abzianidze, and Johan Bos. 2019. HELP: A dataset for identifying shortcomings of neural models in monotonicity reasoning. In Proceedings of the Eighth Joint Conference on Lexical and Computational Semantics (*SEM 2019), pages 250–255, Minneapolis, Minnesota. Association for Computational Linguistics. [↗](https://doi.org/10.48550/arxiv.1904.12166)
18. Zining Zhu and Frank Rudzicz. 2020. An information theoretic view on selecting linguistic probes. In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 9251–9262, Online. Association for Computational Linguistics. [find](https://scholar.google.com/scholar?q=Zining+Zhu+and+Frank+Rudzicz%2E+2020%2E+An+information+theoretic+view+on+selecting+linguistic+probes%2E+In+Proceedings+of+the+2020+Conference+on+Empirical+Methods+in+Natural+Language+Processing+%28EMNLP%29%2C+pag)
19. A Shauli Ravfogel, Yanai Elazar, Hila Gonen, Michael Twiton, and Yoav Goldberg. 2020. [Null it out: Guarding protected attributes by iterative nullspace projection](https://md.research.elamur.ai/free/en/papers/ravfogel-2020-null-it-out-guarding-protected/). In Proceedings of the 58th Annual Meeting of Expanded Amnesic Intervention Results *Figure 8: Single Feature Amnesic Probing* *Figure 9: Composite Feature Label Amnesic Probing* *Figure 10: Sanity Check: Entailment Gold Label Amnesic Probing*

*Updated 1 November 2026*