---
title: "PRover: Proof Generation for Interpretable Reasoning over Rules"
description: "Clark et al. (2020) showed that RuleTakers, a fine-tuned RoBERTa, can decide whether a statement… 44 references, cited by 54 papers here."
authors: ["Swarnadeep Saha", "Sayan Ghosh", "Shashank Srivastava", "Mohit Bansal"]
year: 2020
venue: "Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)"
source: "https://aclanthology.org/2020.emnlp-main.9/"
doi: "10.18653/v1/2020.emnlp-main.9"
licence: "CC BY 4.0 (ACL Anthology)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/saha-2020-prover-proof-generation-interpretable-reasoning/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/saha-2020-prover-proof-generation-interpretable-reasoning/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/) / PRover: Proof Generation for Interpretable Reasoning over Rules

# PRover: Proof Generation for Interpretable Reasoning over Rules

**Authors:** [Swarnadeep Saha](https://md.research.elamur.ai/free/en/authors/swarnadeep-saha/), Sayan Ghosh, [Shashank Srivastava](https://md.research.elamur.ai/free/en/authors/shashank-srivastava/) and [Mohit Bansal](https://md.research.elamur.ai/free/en/authors/mohit-bansal/) · **Year:** 2020 · **Venue:** [Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)](https://md.research.elamur.ai/free/en/venues/emnlp/) ([EMNLP 2020](https://md.research.elamur.ai/free/en/venues/emnlp/2020/)) · **Publisher:** [Association for Computational Linguistics](https://md.research.elamur.ai/free/en/publishers/association-for-computational-linguistics/) · **Licence:** [CC BY 4.0 (ACL Anthology)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://aclanthology.org/2020.emnlp-main.9/) · [PDF](https://aclanthology.org/2020.emnlp-main.9.pdf) · [DOI 10.18653/v1/2020.emnlp-main.9](https://doi.org/10.18653/v1/2020.emnlp-main.9) · [OpenAlex](https://openalex.org/W3092597885)

**Citations:** 47 citations (OpenAlex, 2026-10-14) · [54](https://md.research.elamur.ai/free/en/papers/saha-2020-prover-proof-generation-interpretable-reasoning/cited-by/) in our library, of the 141,840 papers whose bibliographies we read

## Summary
Clark et al. (2020) showed that RuleTakers, a fine-tuned RoBERTa, can decide whether a statement follows from facts and rules written in English. However, it gives no justification for its answers. PRover addresses this by producing an answer together with an explicit proof. Each proof is a directed graph whose nodes are facts, rules or a special NAF (negation as failure) node, and whose edges show how facts feed rules and how rule outputs feed other rules.

A shared RoBERTa encoder supports three components:
- a binary QA head;
- a node classifier that selects proof elements;
- an edge classifier that predicts links and their direction.

The three are trained with a joint cross-entropy loss. During training, edges that cannot occur are masked out. At inference, an ILP enforces global validity constraints, including connectivity checked through a max-flow formulation.

On the synthetic, hand-authored and paraphrased rule-bases from Clark et al., PRover matches or exceeds RuleTakers on QA, generates exactly correct proofs about 87% of the time, and generalizes better to deeper reasoning. This makes it a step toward interpretable, structured explanations in NLP.

## Key points
- Exact proof accuracy is about 87%, falling to roughly 65% for depth-5 questions.
- Zero-shot QA improves by up to 6% over RuleTakers, and unlike RuleTakers, PRover needs no extra fine-tuning on RACE.
- When trained on shallow questions and tested on deeper ones, QA accuracy is up to 15% higher; with only 40% of the training data, PRover still reaches about 98% QA accuracy.
- Error analysis on DU5 depth 5:
  - NAF is predicted correctly 95% of the time.
  - Among wrong node sets, 42% are subsets of the gold set and 25% are supersets, so the model tends to miss needed rules and facts.
  - Edge direction is almost never the cause of errors.
- Code and models are available at github.com/swarnaHub/PRover.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [IBM](https://md.research.elamur.ai/free/en/companies/ibm/) — Company, Armonk, United States · 85 works · 66,118 citations
- [University of North Carolina at Chapel Hill](https://md.research.elamur.ai/free/en/universities/university-of-north-carolina-at-chapel-hill/) — University, Chapel Hill, United States · 37 works · 14,332 citations
- [Toyota Technological Institute at Chicago](https://md.research.elamur.ai/free/en/universities/toyota-technological-institute-at-chicago/) — University, Chicago, United States · 29 works · 10,273 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 9,086 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 8,889 works · [🎭 Arts and Humanities](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/) · 9,296 works · [🗣️ Language and Linguistics](https://md.research.elamur.ai/free/en/shelves/arts-and-humanities/language-and-linguistics/) · 9,206 works

🧭 Topics (1): [🧮 Reasoning](https://md.research.elamur.ai/free/en/topics/reasoning/) · 510 works

## Cite

Cite as: Swarnadeep Saha, Sayan Ghosh, Shashank Srivastava and Mohit Bansal. 2020. PRover: Proof Generation for Interpretable Reasoning over Rules. Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{saha2020prover,
  title = {{PRover: Proof Generation for Interpretable Reasoning over Rules}},
  author = {Swarnadeep Saha, Sayan Ghosh, Shashank Srivastava and Mohit Bansal},
  booktitle = {Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)},
  year = {2020},
  doi = {10.18653/v1/2020.emnlp-main.9},
  url = {https://doi.org/10.18653/v1/2020.emnlp-main.9}
}
```

- [← previous MRP 2020: The Second Shared Task on Cross-Framework and Cross-Lingual Meaning Representation Parsing](https://md.research.elamur.ai/free/en/papers/oepen-2020-mrp-2020-second-shared-task/)
- [next → Findings of the WMT 2020 Shared Task on Parallel Corpus Filtering and Alignment](https://md.research.elamur.ai/free/en/papers/koehn-2020-findings-wmt-2020-shared-task/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/page/5/)

## References

44 references, 21 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Ralph Abboud, Ismail Ilkan Ceylan, and Thomas Lukasiewicz. 2020. Learning to reason: Leveraging neural networks for approximate DNF counting. In Proceedings of the 34th AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Ralph+Abboud%2C+Ismail+Ilkan+Ceylan%2C+and+Thomas+Lukasiewicz%2E+2020%2E+Learning+to+reason%3A+Leveraging+neural+networks+for+approximate+DNF+counting%2E+In+Proceedings+of+the+34th+AAAI+Conference+on+Artificial+I)
2. Ibrahim Abdelaziz, Veronika Thost, Maxwell Crouse, and Achille Fokoue. 2020. An experimental study of formula embeddings for automated theorem proving in first-order logic. arXiv preprint arXiv:2002.00423. — [arXiv:2002.00423](https://arxiv.org/abs/2002.00423) [find](https://scholar.google.com/scholar?q=Ibrahim+Abdelaziz%2C+Veronika+Thost%2C+Maxwell+Crouse%2C+and+Achille+Fokoue%2E+2020%2E+An+experimental+study+of+formula+embeddings+for+automated+theorem+proving+in+first%2Dorder+logic%2E+arXiv+preprint+arXiv%3A2002%2E0)
3. Mohit Bansal, Claire Cardie, and Lillian Lee. 2008. The power of negative thinking: Exploiting label disagreement in the min-cut classification framework. In COLING 2008: Companion Volume: Posters, pages 15–18. [find](https://scholar.google.com/scholar?q=Mohit+Bansal%2C+Claire+Cardie%2C+and+Lillian+Lee%2E+2008%2E+The+power+of+negative+thinking%3A+Exploiting+label+disagreement+in+the+min%2Dcut+classification+framework%2E+In+COLING+2008%3A+Companion+Volume%3A+Posters%2C+pa)
4. Regina Barzilay and Mirella Lapata. 2005. [Collective content selection for concept-to-text generation](https://md.research.elamur.ai/free/en/papers/barzilay-2005-collective-content-selection-concept-text/). In Proceedings of the conference on Human Language Technology and Empirical Methods in Natural Language Processing, pages 331–338. Association for Computational Linguistics.
5. Jonathan Berant, Andrew Chou, Roy Frostig, and Percy Liang. 2013. [Semantic parsing on freebase from question-answer pairs](https://md.research.elamur.ai/free/en/papers/berant-2013-semantic-parsing-freebase-question-answer/). In Proceedings of the 2013 conference on empirical methods in natural language processing, pages 1533–1544.
6. Jonathan Berant and Percy Liang. 2014. [Semantic parsing via paraphrasing](https://md.research.elamur.ai/free/en/papers/berant-2014-semantic-parsing-paraphrasing/). In Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 1415– 1425.
7. Oana-Maria Camburu, Tim Rocktäschel, Thomas Lukasiewicz, and Phil Blunsom. 2018. [e-SNLI: Natural language inference with natural language explanations](https://md.research.elamur.ai/free/en/papers/camburu-2018-e-snli-natural-language-inference/). In Advances in Neural Information Processing Systems, pages 9539–9549.
8. Peter Clark, Oyvind Tafjord, and Kyle Richardson. 2020. [Transformers as soft reasoners over language](https://md.research.elamur.ai/free/en/papers/clark-2020-transformers-soft-reasoners-over-language/). In International Joint Conference on Artificial Intelligence.
9. Ido Dagan, Dan Roth, Mark Sammons, and Fabio Massimo Zanzotto. 2013. Recognizing textual entailment: Models and applications. Synthesis Lectures on Human Language Technologies, 6(4):1–220. [find](https://scholar.google.com/scholar?q=Ido+Dagan%2C+Dan+Roth%2C+Mark+Sammons%2C+and+Fabio+Massimo+Zanzotto%2E+2013%2E+Recognizing+textual+entailment%3A+Models+and+applications%2E+Synthesis+Lectures+on+Human+Language+Technologies%2C+6%284%29%3A1%E2%80%93220%2E)
10. Finale Doshi-Velez and Been Kim. 2017. [Towards a rigorous science of interpretable machine learning](https://md.research.elamur.ai/free/en/papers/doshi-velez-2017-towards-rigorous-science-interpretable-machine/). arXiv preprint arXiv:1702.08608.
11. Shimon Even and R Endre Tarjan. 1975. Network flow and testing graph connectivity. SIAM journal on computing, 4(4):507–518. [find](https://scholar.google.com/scholar?q=Shimon+Even+and+R+Endre+Tarjan%2E+1975%2E+Network+flow+and+testing+graph+connectivity%2E+SIAM+journal+on+computing%2C+4%284%29%3A507%E2%80%93518%2E)
12. Peter Hase and Mohit Bansal. 2020. [Evaluating explainable AI: Which algorithmic explanations help users predict model behavior?](https://md.research.elamur.ai/free/en/papers/hase-2020-evaluating-explainable-ai-which-algorithmic/) In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics.
13. Han He and Jinho Choi. 2020. Establishing strong baselines for the new decade: Sequence tagging, syntactic and semantic parsing with bert. In The Thirty-Third International Flairs Conference. [find](https://scholar.google.com/scholar?q=Han+He+and+Jinho+Choi%2E+2020%2E+Establishing+strong+baselines+for+the+new+decade%3A+Sequence+tagging%2C+syntactic+and+semantic+parsing+with+bert%2E+In+The+Thirty%2DThird+International+Flairs+Conference%2E)
14. Shalmali Joshi, Oluwasanmi Koyejo, Been Kim, and Joydeep Ghosh. 2018. xGEMs: Generating examplars to explain black-box models. arXiv preprint arXiv:1806.08867. — [arXiv:1806.08867](https://arxiv.org/abs/1806.08867) [find](https://scholar.google.com/scholar?q=Shalmali+Joshi%2C+Oluwasanmi+Koyejo%2C+Been+Kim%2C+and+Joydeep+Ghosh%2E+2018%2E+xGEMs%3A+Generating+examplars+to+explain+black%2Dbox+models%2E+arXiv+preprint+arXiv%3A1806%2E08867%2E+%E2%80%94+%5BarXiv%3A1806%2E08867%5D%28https%3A%2F%2Farxiv%2Eorg%2Fa)
15. Aishwarya Kamath and Rajarshi Das. 2019. A survey on semantic parsing. In Automated Knowledge Base Construction (AKBC). [find](https://scholar.google.com/scholar?q=Aishwarya+Kamath+and+Rajarshi+Das%2E+2019%2E+A+survey+on+semantic+parsing%2E+In+Automated+Knowledge+Base+Construction+%28AKBC%29%2E)
16. Guokun Lai, Qizhe Xie, Hanxiao Liu, Yiming Yang, and Eduard Hovy. 2017. [RACE: Large-scale reading comprehension dataset from examinations](https://md.research.elamur.ai/free/en/papers/lai-2017-race-large-scale-reading-comprehension/). In Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, pages 785–794.
17. Guillaume Lample and François Charton. 2020. Deep learning for symbolic mathematics. In 8th International Conference on Learning Representations, ICLR 2020. OpenReview.net. [find](https://scholar.google.com/scholar?q=Guillaume+Lample+and+Franc%CC%A7ois+Charton%2E+2020%2E+Deep+learning+for+symbolic+mathematics%2E+In+8th+International+Conference+on+Learning+Representations%2C+ICLR+2020%2E+OpenReview%2Enet%2E)
18. Tom Leighton and Satish Rao. 1999. Multicommodity max-flow min-cut theorems and their use in designing approximation algorithms. Journal of the ACM (JACM), 46(6):787–832. [find](https://scholar.google.com/scholar?q=Tom+Leighton+and+Satish+Rao%2E+1999%2E+Multicommodity+max%2Dflow+min%2Dcut+theorems+and+their+use+in+designing+approximation+algorithms%2E+Journal+of+the+ACM+%28JACM%29%2C+46%286%29%3A787%E2%80%93832%2E)
19. Kevin Lin, Oyvind Tafjord, Peter Clark, and Matt Gardner. 2019. [Reasoning over paragraph effects in situations](https://md.research.elamur.ai/free/en/papers/lin-2019-reasoning-over-paragraph-effects-situations/). In Proceedings of the 2nd Workshop on Machine Reading for Question Answering, pages 58– 62, Hong Kong, China. Association for Computational Linguistics.
20. Hui Liu, Qingyu Yin, and William Yang Wang. 2019a. Towards explainable NLP: A generative explanation framework for text classification. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 5570–5581. [find](https://scholar.google.com/scholar?q=Hui+Liu%2C+Qingyu+Yin%2C+and+William+Yang+Wang%2E+2019a%2E+Towards+explainable+NLP%3A+A+generative+explanation+framework+for+text+classification%2E+In+Proceedings+of+the+57th+Annual+Meeting+of+the+Association+for)
21. Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov. 2019b. [find](https://scholar.google.com/scholar?q=Yinhan+Liu%2C+Myle+Ott%2C+Naman+Goyal%2C+Jingfei+Du%2C+Mandar+Joshi%2C+Danqi+Chen%2C+Omer+Levy%2C+Mike+Lewis%2C+Luke+Zettlemoyer%2C+and+Veselin+Stoyanov%2E+2019b%2E)
22. RoBERTa: A robustly optimized bert pretraining approach. arXiv preprint arXiv:1907.11692. [find](https://scholar.google.com/scholar?q=RoBERTa%3A+A+robustly+optimized+bert+pretraining+approach%2E+arXiv+preprint+arXiv%3A1907%2E11692%2E)
23. Bill MacCartney and Christopher D Manning. 2014. Natural logic and natural language inference. In Computing meaning, pages 129–147. Springer. [find](https://scholar.google.com/scholar?q=Bill+MacCartney+and+Christopher+D+Manning%2E+2014%2E+Natural+logic+and+natural+language+inference%2E+In+Computing+meaning%2C+pages+129%E2%80%93147%2E+Springer%2E)
24. Mark A Musen and Johan Van Der Lei. 1988. Of brittleness and bottlenecks: Challenges in the creation of pattern-recognition and expert-system models. In Machine Intelligence and Pattern Recognition, volume 7, pages 335–352. Elsevier. [find](https://scholar.google.com/scholar?q=Mark+A+Musen+and+Johan+Van+Der+Lei%2E+1988%2E+Of+brittleness+and+bottlenecks%3A+Challenges+in+the+creation+of+pattern%2Drecognition+and+expert%2Dsystem+models%2E+In+Machine+Intelligence+and+Pattern+Recognition%2C+v)
25. Arvind Neelakantan, Quoc V. Le, and Ilya Sutskever. 2016. Neural programmer: Inducing latent programs with gradient descent. In 4th International Conference on Learning Representations, ICLR 2016, Conference Track Proceedings. [find](https://scholar.google.com/scholar?q=Arvind+Neelakantan%2C+Quoc+V%2E+Le%2C+and+Ilya+Sutskever%2E+2016%2E+Neural+programmer%3A+Inducing+latent+programs+with+gradient+descent%2E+In+4th+International+Conference+on+Learning+Representations%2C+ICLR+2016%2C+Con)
26. Allen Newell and Herbert Simon. 1956. The logic theory machine–a complex information processing system. IRE Transactions on information theory, 2(3):61–79. [find](https://scholar.google.com/scholar?q=Allen+Newell+and+Herbert+Simon%2E+1956%2E+The+logic+theory+machine%E2%80%93a+complex+information+processing+system%2E+IRE+Transactions+on+information+theory%2C+2%283%29%3A61%E2%80%9379%2E)
27. Bo Pang and Lillian Lee. 2004. [A sentimental education: Sentiment analysis using subjectivity summarization based on minimum cuts](https://md.research.elamur.ai/free/en/papers/pang-2004-sentimental-education-sentiment-analysis-subjectivity/). In Proceedings of the 42nd annual meeting on Association for Computational Linguistics, page 271. Association for Computational Linguistics.
28. Nazneen Fatema Rajani, Bryan McCann, Caiming Xiong, and Richard Socher. 2019. [Explain yourself! leveraging language models for commonsense reasoning](https://md.research.elamur.ai/free/en/papers/rajani-2019-explain-yourself-leveraging-language-models/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 4932–4942.
29. Scott E. Reed and Nando de Freitas. 2016. Neural programmer-interpreters. In 4th International Conference on Learning Representations, ICLR 2016, San Juan, Puerto Rico, May 2-4, 2016, Conference Track Proceedings. [find](https://scholar.google.com/scholar?q=Scott+E%2E+Reed+and+Nando+de+Freitas%2E+2016%2E+Neural+programmer%2Dinterpreters%2E+In+4th+International+Conference+on+Learning+Representations%2C+ICLR+2016%2C+San+Juan%2C+Puerto+Rico%2C+May+2%2D4%2C+2016%2C+Conference+Track)
30. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2016. [”Why should I trust you?”: Explaining the predictions of any classifier](https://md.research.elamur.ai/free/en/papers/ribeiro-2016-why-should-i-trust-explaining/). In Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pages 1135–1144.
31. Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. 2018. Anchors: High-precision model-agnostic explanations. In Thirty-Second AAAI Conference on Artificial Intelligence. [find](https://scholar.google.com/scholar?q=Marco+Tulio+Ribeiro%2C+Sameer+Singh%2C+and+Carlos+Guestrin%2E+2018%2E+Anchors%3A+High%2Dprecision+model%2Dagnostic+explanations%2E+In+Thirty%2DSecond+AAAI+Conference+on+Artificial+Intelligence%2E)
32. Kyle Richardson, Hai Hu, Lawrence S Moss, and Ashish Sabharwal. 2020. [Probing natural language inference models through semantic fragments](https://md.research.elamur.ai/free/en/papers/richardson-2019-probing-natural-language-inference-models/). In Thirty-Fourth AAAI Conference on Artificial Intelligence.
33. Cynthia Rudin. 2019. [Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead](https://md.research.elamur.ai/free/en/papers/rudin-2018-stop-explaining-black-box/). Nature Machine Intelligence, 1(5):206–215.
34. Pouya Samangouei, Ardavan Saeedi, Liam Nakagawa, and Nathan Silberman. 2018. ExplainGAN: Model explanation via decision boundary crossing transformations. In Proceedings of the European Conference on Computer Vision (ECCV), pages 666–681. [find](https://scholar.google.com/scholar?q=Pouya+Samangouei%2C+Ardavan+Saeedi%2C+Liam+Nakagawa%2C+and+Nathan+Silberman%2E+2018%2E+ExplainGAN%3A+Model+explanation+via+decision+boundary+crossing+transformations%2E+In+Proceedings+of+the+European+Conference+on+)
35. David Saxton, Edward Grefenstette, Felix Hill, and Pushmeet Kohli. 2019. [Analysing mathematical reasoning abilities of neural models](https://md.research.elamur.ai/free/en/papers/saxton-2019-analysing-mathematical-reasoning-abilities-neural/). In International Conference on Learning Representations.
36. Daniel Selsam, Matthew Lamm, Benedikt Bünz, Percy Liang, Leonardo de Moura, and David L. Dill. 2019. Learning a SAT solver from single-bit supervision. In 7th International Conference on Learning Representations, ICLR 2019, New Orleans, LA, USA, May 6-9, 2019. OpenReview.net. [find](https://scholar.google.com/scholar?q=Daniel+Selsam%2C+Matthew+Lamm%2C+Benedikt+Bu%CC%88nz%2C+Percy+Liang%2C+Leonardo+de+Moura%2C+and+David+L%2E+Dill%2E+2019%2E+Learning+a+SAT+solver+from+single%2Dbit+supervision%2E+In+7th+International+Conference+on+Learning+Rep)
37. Oyvind Tafjord, Matt Gardner, Kevin Lin, and Peter Clark. 2019. QuaRTz: An open-domain dataset of qualitative relationship questions. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (E… [find](https://scholar.google.com/scholar?q=Oyvind+Tafjord%2C+Matt+Gardner%2C+Kevin+Lin%2C+and+Peter+Clark%2E+2019%2E+QuaRTz%3A+An+open%2Ddomain+dataset+of+qualitative+relationship+questions%2E+In+Proceedings+of+the+2019+Conference+on+Empirical+Methods+in+Natu)
38. Alon Talmor, Yanai Elazar, Yoav Goldberg, and Jonathan Berant. 2019. [oLMpics–on what language model pre-training captures](https://md.research.elamur.ai/free/en/papers/talmor-2019-olmpics-language-model-pre-training/). arXiv preprint arXiv:1912.13283.
39. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in neural information processing systems, pages 5998–6008.
40. Qiang Wang, Bei Li, Tong Xiao, Jingbo Zhu, Changliang Li, Derek F. Wong, and Lidia S. Chao. 2019. [Learning deep transformer models for machine translation](https://md.research.elamur.ai/free/en/papers/wang-2019-learning-deep-transformer-models-machine/). In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 1810–1822, Florence, Italy. Association for Co…
41. Leon Weber, Pasquale Minervini, Jannes Münchmeyer, Ulf Leser, and Tim Rocktäschel. 2019. NLProlog: Reasoning with weak unification for question answering in natural language. In Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pages 6151–6161, Florence, Ital… [find](https://scholar.google.com/scholar?q=Leon+Weber%2C+Pasquale+Minervini%2C+Jannes+Mu%CC%88nchmeyer%2C+Ulf+Leser%2C+and+Tim+Rockta%CC%88schel%2E+2019%2E+NLProlog%3A+Reasoning+with+weak+unification+for+question+answering+in+natural+language%2E+In+Proceedings+of+the+5)
42. Jason Weston, Antoine Bordes, Sumit Chopra, Alexander M Rush, Bart van Merriënboer, Armand Joulin, and Tomas Mikolov. 2015. [Towards AI-complete question answering: A set of prerequisite toy tasks](https://md.research.elamur.ai/free/en/papers/weston-2015-towards-ai-complete-question-answering/). arXiv preprint arXiv:1502.05698.
43. Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pier-ric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al. 2019. Huggingface’s [transformers: State-of-the-art natural language processing](https://md.research.elamur.ai/free/en/papers/wolf-2020-transformers-state-art-natural-language/). arXiv preprint arXiv:1910.03771. train, validation and test split…
44. Zhilin Yang, Peng Qi, Saizheng Zhang, Yoshua Bengio, William Cohen, Ruslan Salakhutdinov, and Christopher D Manning. 2018. [HotpotQA: A dataset for diverse, explainable multi-hop question answering](https://md.research.elamur.ai/free/en/papers/yang-2018-hotpotqa-dataset-diverse-explainable-multi/). In Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.

*Updated 24 October 2026*