---
title: "How Fast can BERT Learn Simple Natural Language Inference?"
description: "They build synthetic entailment data from templates that describe where two objects sit relative to each… 35 references, cited by 3 papers here."
authors: ["Yi-Chung Lin", "Keh-Yih Su"]
year: 2021
venue: "Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume"
source: "https://aclanthology.org/2021.eacl-main.51/"
doi: "10.18653/v1/2021.eacl-main.51"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/lin-2021-how-fast-can-bert-learn/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/lin-2021-how-fast-can-bert-learn/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2021](https://md.research.elamur.ai/free/en/papers/2021/page/6/) / How Fast can BERT Learn Simple Natural Language Inference?

# How Fast can BERT Learn Simple Natural Language Inference?

**Authors:** Yi-Chung Lin and Keh-Yih Su · **Year:** 2021 · **Venue:** [Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume](https://md.research.elamur.ai/free/en/venues/eacl/) ([EACL 2021](https://md.research.elamur.ai/free/en/venues/eacl/2021/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2021.eacl-main.51/) · [PDF](https://aclanthology.org/2021.eacl-main.51.pdf) · [DOI 10.18653/v1/2021.eacl-main.51](https://doi.org/10.18653/v1/2021.eacl-main.51) · [OpenAlex](https://openalex.org/W3154482395)

**Citations:** 7 citations (OpenAlex, 2026-10-14) · 3 in our library, of the 141,840 papers whose bibliographies we read

## Summary
Lin and Su ask whether BERT can learn natural language inference once every dataset shortcut is removed, and how much data it then needs. They build synthetic entailment data from templates that describe where two objects sit relative to each other, for instance a sentence placing one named person to another's left. Every example has the same wording apart from randomly drawn names, so annotation artifacts cannot leak the label. They fine-tune "BERT-Base, Uncased" on growing training sets and draw learning curves. Test sets use unseen person names and, separately, fruit and vegetable names. BERT learns slowly. Adding the antonym "right" makes the task about 30 times more data-hungry. It also generalises poorly from people to fruit. When the input is extended with a few hand-designed comparison features (do the predicates and arguments of premise and hypothesis match?), the data needed falls sharply, by a factor of about 1,500 in the best setting. The authors conclude that injecting domain knowledge may be essential for efficient neural NLI, especially where labelled data is scarce.

## Key points
- **Name sets:** 1,696 first names that are whole tokens in the BERT vocabulary were split 1,356/170/170 into train, validation and evaluation pools. An extra pool of 30 fruit/vegetable names supports a cross-domain test. The resulting datasets are Name-T, Name-V, Name-E and Fruit-E, with 1,000 examples each for validation and test.
- **EXP-SP (only "left", 6 templates):** about 100 training examples give over 99% on Name-E. Even 3,000 examples do not reach 100%, and Fruit-E stays below 99% at that size.
- **EXP-AP (adds "right", 24 templates: 4 entailment, 4 contradiction, 16 neutral):** BERT needs roughly 3,000 examples to pass 99% on Name-E.
- **EXP-SF (three true/false match indicators appended after a separator):** about 100 examples suffice for over 99% on Name-E. Replacing "true"/"false" with the arbitrary words "fortification"/"mississippi" gave similar results.
- **EXP-DF (a three-valued true/false/fuzzy feature for direct, swapped or other argument alignment):** 20 examples reach 99.9% on Fruit-E. Plain EXP-AP needs 30,000 examples for 99.1%.

## Interesting ideas
- Build probe data from templates where only the slot fillers vary, so no surface cue can predict the label and any success must come from real inference.
- Test on a disjoint vocabulary of a different entity type (people in training, fruit at test) to measure whether a learned relation transfers beyond the fillers it saw.
- Hand the model explicit comparison signals between aligned fields (same predicate? first argument equal? arguments swapped?) instead of hoping it discovers them, which cuts data needs dramatically.
- A three-valued feature that separates "swapped arguments" from "unrelated arguments" resolves the contradiction-versus-neutral confusion better than independent binary flags.
- Antonymous predicates (left/right) multiply the difficulty of a learned model, so the converse mapping is better supplied as explicit knowledge.

## For Mova

How these ideas could serve Mova, our small open language model — its files, stages and tests:

- **I1** (learn): Template-generated evaluation sets where only the arguments change could serve as negative controls, showing whether a reader answers from the frame structure rather than lexical overlap.
- **I2** (answer): Fable and word-problem readers could be checked with swapped entity types (animals ↔ objects) to confirm that role reasoning does not depend on familiar names.
- **I3** (frames): Aligning premise and hypothesis frames role by role and emitting match/swap/mismatch flags would give a transparent, cheap entailment signal over who-did-what-to-whom structures.
- **I4** (world): When checking a claim against the text's world state, record whether a relation holds, holds in reverse, or is simply unknown, so that "reversed" yields contradiction and "unknown" yields neutral.
- **I5** (world): Store converse-relation pairs (left/right, buy/sell, parent/child) in the compiled ontology so the reasoner can normalise relations before comparing them, rather than learning this from examples.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Institute of Information Science, Academia Sinica](https://md.research.elamur.ai/free/en/labs/institute-of-information-science-academia-sinica/) — Research institute, Taipei, Taiwan · 2 works · 46 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,085 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,888 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 510 works

## Cite

Cite as: Yi-Chung Lin and Keh-Yih Su. 2021. How Fast can BERT Learn Simple Natural Language Inference?. Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{lin2021how,
  title = {{How Fast can BERT Learn Simple Natural Language Inference?}},
  author = {Yi-Chung Lin and Keh-Yih Su},
  booktitle = {Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume},
  year = {2021},
  doi = {10.18653/v1/2021.eacl-main.51},
  url = {https://doi.org/10.18653/v1/2021.eacl-main.51}
}
```

- [← previous Enhancing Students’ Understanding Of English Grammar Through Literary Works And Exploring Students’ Voice On Their Use](https://md.research.elamur.ai/free/en/papers/aniuranti-2021-enhancing-students-understanding-english-grammar/)
- [next → Marked Attribute Bias in Natural Language Inference](https://md.research.elamur.ai/free/en/papers/dawkins-2021-marked-attribute-bias-natural-language/)
- [↑ back to papers of 2021](https://md.research.elamur.ai/free/en/papers/2021/page/6/)

## References

35 references, 15 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Betty van Aken, Benjamin Winter, Alexander Löser, and Felix A. Gers. 2019. How Does BERT Answer Questions: A Layer-Wise Analysis of Transformer Representations. Computing Research Repository, arXiv:1909.04925. Visual Object Classification, and Recognising Tectual Entailment, Springer, pages 177–190. [↗](https://arxiv.org/abs/1909.04925)
2. Luisa Bentivogli, Peter Clark, Ido Dagan, and Danilo Giampiccolo. 2011. The Seventh PASCAL Recognizing Textual Entailment Challenge. In Proceedings of the 4th Text Analysis Conference. Gaithersburg: National Institute of Standards and Technology, pages 1–16. [find](https://scholar.google.com/scholar?q=Luisa+Bentivogli%2C+Peter+Clark%2C+Ido+Dagan%2C+and+Danilo+Giampiccolo%2E+2011%2E+The+Seventh+PASCAL+Recognizing+Textual+Entailment+Challenge%2E+In+Proceedings+of+the+4th+Text+Analysis+Conference%2E+Gaithersburg%3A+N)
3. Jacob Devlin, Ming-Wei Chang, Kenton Lee and Kristina Toutanova. 2019. [BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding](https://md.research.elamur.ai/free/en/papers/devlin-2019-bert-pre-training-deep-bidirectional/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, pages 4171–4186.
4. Samuel R. Bowman, Gabor Angeli, Christopher Potts and Christopher D. Manning. 2015. A large annotated corpus for learning natural language inference. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pages 632–642. [↗](https://doi.org/10.48550/arxiv.1508.05326)
5. Dheeru Dua, Yizhong Wang, Pradeep Dasigi, Gabriel Stanovsky, Sameer Singh, and Matt Gardner. 2019. [DROP: A Reading Comprehension Benchmark Requiring Discrete Reasoning Over Paragraphs](https://md.research.elamur.ai/free/en/papers/dua-2019-drop-reading-comprehension-benchmark-requiring/). In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, pages 2368–2378. [↗](https://doi.org/10.48550/arxiv.1903.00161)
6. Samuel R. Bowman and Xiandan Zhu. 2019 Deep Learning for Natural Language Inference. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Tutorials. [find](https://scholar.google.com/scholar?q=Samuel+R%2E+Bowman+and+Xiandan+Zhu%2E+2019+Deep+Learning+for+Natural+Language+Inference%2E+In+Proceedings+of+the+2019+Conference+of+the+North+American+Chapter+of+the+Association+for+Computational+Linguistic)
7. Max Glockner, Vered Shwartz and Yoav Goldberg. 2018. [Breaking NLI Systems with Sentences that Require Simple Lexical Inferences](https://md.research.elamur.ai/free/en/papers/glockner-2018-breaking-nli-systems-sentences-that/). In Proceedings of the 56 th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, pages 650–655.
8. Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. [e-SNLI: Natural Language Inference with Natural Language Explanations](https://md.research.elamur.ai/free/en/papers/camburu-2018-e-snli-natural-language-inference/). In Proceedings of the 32 nd Conference on Neural Information Processing Systems (NeurIPS 2018). [↗](https://doi.org/10.48550/arxiv.1812.01193)
9. Yichen Gong, Heng Luo and Jian Zhang. 2017. Natural Language Inference Over Interaction Space. Computing Research Repository, arXiv:1709.04348. [↗](https://arxiv.org/abs/1709.04348)
10. Danqi Chen, Adam Fisch, Jason Weston, and Antoine Bordes. 2017. [Reading Wikipedia to Answer Open-Domain Questions](https://md.research.elamur.ai/free/en/papers/chen-2017-reading-wikipedia-answer-open-domain/). In Proceedings of the 55 th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, pages 1870–1879.
11. Çağlar Gülçehre and Yoshua Bengio. 2016. Knowledge Matters: Importance of Prior Information for Optimization. Journal of Machine Learning Research, 17(8): 1-32. [find](https://scholar.google.com/scholar?q=%C3%87a%C4%9Flar+G%C3%BCl%C3%A7ehre+and+Yoshua+Bengio%2E+2016%2E+Knowledge+Matters%3A+Importance+of+Prior+Information+for+Optimization%2E+Journal+of+Machine+Learning+Research%2C+17%288%29%3A+1%2D32%2E)
12. Suchin Gururangan, Swabha Swayamdipta, Omer Levy, Roy Schwartz, Samuel Bowman, and Noah A. Smith. 2018. Annotation Artifacts in Natural Language Inference Data. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, pages 107–112. [find](https://scholar.google.com/scholar?q=Suchin+Gururangan%2C+Swabha+Swayamdipta%2C+Omer+Levy%2C+Roy+Schwartz%2C+Samuel+Bowman%2C+and+Noah+A%2E+Smith%2E+2018%2E+Annotation+Artifacts+in+Natural+Language+Inference+Data%2E+In+Proceedings+of+the+2018+Conference+o)
13. Kevin Clark, Urvashi Khandelwal, Omer Levy, and Christopher D. Manning. 2019. What Does BERT Look At? An Analysis of BERT's Attention. Computing Research Repository, arXiv:1906.04341. [↗](https://arxiv.org/abs/1906.04341)
14. Andy Coenen, Emily Reif, Ann Yuan, Been Kim, Adam Pearce, Fernanda Viégas, and Martin Wattenberg. 2019. [Visualizing and Measuring the Geometry of BERT](https://md.research.elamur.ai/free/en/papers/coenen-2019-visualizing-measuring-geometry-bert/). Computing Research Repository, arXiv:1906.02715. [↗](https://arxiv.org/abs/1906.02715)
15. Nanjiang Jiang and Marie-Catherine de Marneffe, 2019. Evaluating BERT for natural language inference: A case study on the Commitment Bank. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9 th International Joint Conference on Natural Language Processing. Association for Computational Linguistics, pages 6086–6091. [find](https://scholar.google.com/scholar?q=Nanjiang+Jiang+and+Marie%2DCatherine+de+Marneffe%2C+2019%2E+Evaluating+BERT+for+natural+language+inference%3A+A+case+study+on+the+Commitment+Bank%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in)
16. Alexis Conneau, Ruty Rinott, Guillaume Lample, Adina Williams, Samuel Bowman, Holger Schwenk, and Veselin Stoyanov. 2018. XNLI: Evaluating Cross-lingual Sentence Representations. In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pages 2475–2485. [↗](https://doi.org/10.48550/arxiv.1809.05053)
17. Tushar Khot, Ashish Sabharwal, and Peter Clark. 2018. SCITAIL: A Textual Entailment Dataset from Science Question Answering. In Proceedings of the 32 nd AAAI Conference on Artificial Intelligence (AAAI-18). Association for the Advancement of Artificial Intelligence, pages 5189–5197. [find](https://scholar.google.com/scholar?q=Tushar+Khot%2C+Ashish+Sabharwal%2C+and+Peter+Clark%2E+2018%2E+SCITAIL%3A+A+Textual+Entailment+Dataset+from+Science+Question+Answering%2E+In+Proceedings+of+the+32+nd+AAAI+Conference+on+Artificial+Intelligence+%28AAA)
18. Ido Dagan, Oren Glickman, and Bernardo Magnini. 2006. The PASCAL Recognising Textual Entailment Challenge. In Machine Learning Challenges. Evaluating Predictive Uncertainty, inference. In Proceedings of the Seventh Joint Conference on Lexical and Computational Semantics. Association for Computational Linguistics, pages 180–191. [find](https://scholar.google.com/scholar?q=Ido+Dagan%2C+Oren+Glickman%2C+and+Bernardo+Magnini%2E+2006%2E+The+PASCAL+Recognising+Textual+Entailment+Challenge%2E+In+Machine+Learning+Challenges%2E+Evaluating+Predictive+Uncertainty%2C+inference%2E+In+Proceedings+)
19. Alice Lai, Yonatan Bisk, and Julia Hockenmaier. 2017. Natural Language Inference from Multiple Premises. In Proceedings of the 8 th International Joint Conference on Natural Language Processing. Asian Federation of Natural Language Processing, pages 100–109. [↗](https://doi.org/10.48550/arxiv.1710.02925)
20. Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin and Sameer Singh. 2020. Beyond Accuracy: Behavioral Testing of NLP Models with CheckList. In Proceedings of the 58 th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, pages 4902–4912. [find](https://scholar.google.com/scholar?q=Marco+Tulio+Ribeiro%2C+Tongshuang+Wu%2C+Carlos+Guestrin+and+Sameer+Singh%2E+2020%2E+Beyond+Accuracy%3A+Behavioral+Testing+of+NLP+Models+with+CheckList%2E+In+Proceedings+of+the+58+th+Annual+Meeting+of+the+Associat)
21. Kevin Lin, Oyvind Tafjord, Peter Clark, and Matt Gardner. 2019. [Reasoning Over Paragraph Effects in Situations](https://md.research.elamur.ai/free/en/papers/lin-2019-reasoning-over-paragraph-effects-situations/). Computing Research Repository, arXiv:1908.05852. [↗](https://arxiv.org/abs/1908.05852)
22. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019. RoBERTa: A Robustly Optimized BERT Pretraining Approach. Computing Research Repository, arXiv:1907.11692. [↗](https://arxiv.org/abs/1907.11692)
23. Alon Talmor, Jonathan Herzig, Nicholas Lourie, and Jonathan Berant. 2019. CoomonSenseQA: A Question Answering Challenge Targeting Commonsense Knowledge. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, pages 4149–4158. [find](https://scholar.google.com/scholar?q=Alon+Talmor%2C+Jonathan+Herzig%2C+Nicholas+Lourie%2C+and+Jonathan+Berant%2E+2019%2E+CoomonSenseQA%3A+A+Question+Answering+Challenge+Targeting+Commonsense+Knowledge%2E+In+Proceedings+of+the+2019+Conference+of+the+No)
24. Marco Marelli, Luisa Bentivogli, Marco Baroni, Raffaella Bernardi, Stefano Menini and Roberto Zamparelli. 2014a. [SemEval-2014 Task 1: Evaluation of Compositional Distributional Semantic Models on Full Sentences through Semantic Relatedness and Textual Entailment](https://md.research.elamur.ai/free/en/papers/marelli-2014-semeval-2014-task-1-evaluation/). In Proceedings of the 8th International Workshop on Semantic Evaluation (SemEval 2014). Association for Computational Linguistics, pages 1–8.
25. Ian Tenney, Dipanjan Das, and Ellie Pavlick. 2019. [BERT Rediscovers the Classical NLP Pipeline](https://md.research.elamur.ai/free/en/papers/tenney-2019-bert-rediscovers-classical-nlp-pipeline/). In Proceedings of the 57 th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, pages 4593–4601. [↗](https://doi.org/10.48550/arxiv.1905.05950)
26. Masatoshi Tsuchiya. 2018. [Performance Impact Caused by Hidden Bias of Training Data for Recognizing Textual Entailment](https://md.research.elamur.ai/free/en/papers/tsuchiya-2018-performance-impact-caused-hidden-bias/). In Proceedings of the 11 th International Conference on Language Resources and Evaluation (LREC 2018). https://www.aclweb.org/anthology/L18-1239 [↗](https://aclanthology.org/L18-1239/)
27. Marco Marelli, Stefano Menini, Marco Baroni, Luisa Bentivogli, Raffaella Bernardi and Roberto Zamparelli. 2014b. [A SICK cure for the evaluation of compositional distributional semantic models](https://md.research.elamur.ai/free/en/papers/marelli-2014-sick-cure-evaluation-compositional-distributional/). In Proceedings of the 9 th International Conference on Language Resources and Evaluation. European Language Resources Association (ELRA), pages 216–223.
28. R. Thomas McCoy, Ellie Pavlick, and Tal Linzen. 2019. [Right for the Wrong Reasons: Diagnosing Syntactic Heuristics in Natural Language Inference](https://md.research.elamur.ai/free/en/papers/mccoy-2019-right-wrong-reasons-diagnosing-syntactic/). In Proceedings of the 57 th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, pages 3428–3448.
29. Paul Michel, Omer Levy, and Graham Neubig. 2019. [Are Sixteen Heads Really Better than One?](https://md.research.elamur.ai/free/en/papers/michel-2019-sixteen-heads-really-better-than/) In Proceedings of the 33 rd Conference on Neural Information Processing Systems (NeurIPS 2019). arXiv:1905.10650. [↗](https://arxiv.org/abs/1905.10650)
30. Aakanksha Naik, Abhilasha Ravichander, Norman Sadeh, Carolyn Rose, and Graham Neubig. 2018. Stress Test Evaluation for Natural Language Inference. In Proceedings of the 27 th International Conference on Computational Linguistics. Association for Compu-tational Linguistics, pages 2340–2353. [↗](https://doi.org/10.48550/arxiv.1806.00692)
31. Adina Willams, Nikita Nangia, and Samuel R Bowman. 2018. A Broad-Coverage Challenge Corpus for Sentence Understanding through Inference. In Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Association for Computational Linguistics, pages 1112–1122. [find](https://scholar.google.com/scholar?q=Adina+Willams%2C+Nikita+Nangia%2C+and+Samuel+R+Bowman%2E+2018%2E+A+Broad%2DCoverage+Challenge+Corpus+for+Sentence+Understanding+through+Inference%2E+In+Proceedings+of+the+2018+Conference+of+the+North+American+Cha)
32. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William W. Cohen, Ruslan Salakhutdinov, and Christopher D. Manning. 2018. [HOTPOTQA: A Dataset for Diverse Explainable Multi-hop Question Answering](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics, pages 2369–2380. [↗](https://doi.org/10.60692/e8xhv-9km80)
33. Zhilin Yang, Zihang Dai, Yiming Yang, Jaime Carbonell, Ruslan Salakhutdinov, and Quoc V. Le. 2019. [XLNet: Generalized Autoregressive Pretraining for Language Understanding](https://md.research.elamur.ai/free/en/papers/yang-2019-xlnet-generalized-autoregressive-pretraining-language/). Computing Research Repository, arXiv:1906.08237. [↗](https://arxiv.org/abs/1906.08237)
34. Rowan Zellers, Ari Holtzman, Yonatan Bisk, Ali Farhadi, and Yejin Choi. 2019. HellaSwag: Can a Machine Really Finish Your Sentence? In Adam Poliak, Jason Naradowsky, Aparajita Haldar, Rachel Rudinger and Benjamin Van Durme. 2018. Hypothesis only baselines in natural language Proceedings of the 57 th Annual Meeting of the Association for Computational Linguistics. Association for Computational Linguistics, pages 4791–4800. [↗](https://doi.org/10.48550/arxiv.1905.07830)
35. Sheng Zhang, Rachel Rudinger, Kevin Duh, and Benjamin Van Durme. 2017. Ordinal Commonsense Inference. Transactions of the Association for Computational Linguistics, 5:379–395. [find](https://scholar.google.com/scholar?q=Sheng+Zhang%2C+Rachel+Rudinger%2C+Kevin+Duh%2C+and+Benjamin+Van+Durme%2E+2017%2E+Ordinal+Commonsense+Inference%2E+Transactions+of+the+Association+for+Computational+Linguistics%2C+5%3A379%E2%80%93395%2E)

*Updated 24 October 2026*