---
title: "Context-Aware Attentive Knowledge Tracing — Ghosh 2020"
description: "The authors introduce attentive knowledge tracing (AKT), built entirely from attention… 6% AUC. 36 references, cited by 21 papers here."
authors: ["Aritra Ghosh", "Neil Heffernan", "Andrew S. Lan"]
year: 2020
venue: "Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining"
source: "https://arxiv.org/abs/2007.12324"
doi: "10.1145/3394486.3403282"
arxiv: "2007.12324"
licence: "CC BY 4.0 (arXiv)"
licence_url: "https://creativecommons.org/licenses/by/4.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/ghosh-2020-context-aware-attentive-knowledge-tracing/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/ghosh-2020-context-aware-attentive-knowledge-tracing/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2020](https://md.research.elamur.ai/free/en/papers/2020/) / Context-Aware Attentive Knowledge Tracing

# Context-Aware Attentive Knowledge Tracing

**Authors:** Aritra Ghosh, Neil Heffernan and [Andrew S. Lan](https://md.research.elamur.ai/free/en/authors/andrew-lan/) · **Year:** 2020 · **Venue:** [Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining](https://md.research.elamur.ai/free/en/venues/acm-sigkdd-international-conference-on-knowledge-discovery-data-mining-kdd/) ([KDD 2020](https://md.research.elamur.ai/free/en/venues/acm-sigkdd-international-conference-on-knowledge-discovery-data-mining-kdd/2020/)) · **Licence:** [CC BY 4.0 (arXiv)](https://creativecommons.org/licenses/by/4.0/)

**Links:** [Publisher's page](https://doi.org/10.1145/3394486.3403282) · [PDF](https://arxiv.org/pdf/2007.12324) · [DOI 10.1145/3394486.3403282](https://doi.org/10.1145/3394486.3403282) · [arXiv 2007.12324](https://arxiv.org/abs/2007.12324) · [OpenAlex](https://openalex.org/W3043869244)

**Citations:** 580 citations (OpenAlex, 2026-10-14) · [21](https://md.research.elamur.ai/free/en/papers/ghosh-2020-context-aware-attentive-knowledge-tracing/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
Knowledge tracing (KT) forecasts how a learner will answer upcoming assessment questions from their earlier answers. Deep models such as DKT and DKVMN predict accurately but are opaque. Classic approaches like BKT and IRT-based models are interpretable but weaker. The authors introduce attentive knowledge tracing (AKT), built entirely from attention networks. Two self-attentive encoders produce contextualized representations of questions and of question–response pairs. An attention-based knowledge retriever links the current question to the learner's history, and a feed-forward network outputs the predicted response. The attention is monotonic: an exponential decay over a context-aware time distance reduces the weight of old interactions, mimicking forgetting. Question embeddings are regularized with the Rasch model, so items on the same concept differ through a learned difficulty scalar without a large parameter count. On several real-world benchmark datasets, AKT outperforms prior KT methods, by as much as 6% AUC in some cases. Case studies show that its attention patterns and difficulty estimates are meaningful. This makes the model useful for automated feedback and question recommendation in personalized learning.

## Key points
- AKT has four parts: a question encoder, a knowledge encoder, a knowledge retriever, and a response prediction network. Baselines include DKT, DKVMN, and SAKT. In the authors' experiments, SAKT did not beat DKT or DKVMN.
- Attention weights combine question similarity with an exponential decay over a context-aware distance. Learners' attention concentrates on recent practice of the same concept, echoing hand-crafted count features used in DASH and PFA-style models.
- The Rasch model-based embeddings give each question a difficulty parameter (µ_q) within its concept. On ASSISTments2009, t-SNE plots show questions of one concept arranged along a curve from easy to hard.
- Learned difficulties match intuition. For "Probability of a Single Event," the easy, average, and hard items received µ_q values of −0.0515, 0.0088, and 0.0548.
- Code and datasets were announced at github.com/arghosh/AKT. Future work includes using question text and testing the model on language-learning data that involves memory decay.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Massachusetts Amherst](https://md.research.elamur.ai/free/en/universities/university-of-massachusetts-amherst/) — University, Amherst Center, United States · 122 works · 69,574 citations
- [Worcester Polytechnic Institute](https://md.research.elamur.ai/free/en/universities/worcester-polytechnic-institute/) — University, Worcester, United States · 14 works · 1,103 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,888 works · [🖥️ Computer Science Applications](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-science-applications/) · 184 works

## Cite

Cite as: Aritra Ghosh, Neil Heffernan and Andrew S. Lan. 2020. Context-Aware Attentive Knowledge Tracing. Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{ghosh2020context,
  title = {{Context-Aware Attentive Knowledge Tracing}},
  author = {Aritra Ghosh, Neil Heffernan and Andrew S. Lan},
  booktitle = {Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery \& Data Mining},
  year = {2020},
  doi = {10.1145/3394486.3403282},
  eprint = {2007.12324},
  archivePrefix = {arXiv},
  url = {https://doi.org/10.1145/3394486.3403282}
}
```

- [← previous A Joint Neural Model for Information Extraction with Global Features](https://md.research.elamur.ai/free/en/papers/lin-2020-joint-neural-model-information-extraction/)
- [next → Named Entity Recognition as Dependency Parsing](https://md.research.elamur.ai/free/en/papers/yu-2020-named-entity-recognition-as-dependency/)
- [↑ back to papers of 2020](https://md.research.elamur.ai/free/en/papers/2020/)

## References

36 references, 11 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Hao Cen, Kenneth Koedinger, and Brian Junker. 2006. [Learning factors analysis– A general method for cognitive model evaluation and improvement](https://md.research.elamur.ai/free/en/papers/cen-2006-learning-factors-analysis-general-method/). In Proc. International Conference on Intelligent Tutoring Systems. 164–175.
2. Benoît Choffin, Fabrice Popineau, Yolaine Bourda, and Jill-Jênn Vie. 2019. DAS3H: Modeling student learning and forgetting for pptimally scheduling distributed practice of skills. In Proc. International Conference on Educational Data Mining. 29–38. [find](https://scholar.google.com/scholar?q=Beno%C3%AEt+Choffin%2C+Fabrice+Popineau%2C+Yolaine+Bourda%2C+and+Jill%2DJ%C3%AAnn+Vie%2E+2019%2E+DAS3H%3A+Modeling+student+learning+and+forgetting+for+pptimally+scheduling+distributed+practice+of+skills%2E+In+Proc%2E+Internation)
3. Albert Corbett and John Anderson. 1994. Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-adapted Interaction 4, 4 (Dec. 1994), 253–278. [find](https://scholar.google.com/scholar?q=Albert+Corbett+and+John+Anderson%2E+1994%2E+Knowledge+tracing%3A+Modeling+the+acquisition+of+procedural+knowledge%2E+User+Modeling+and+User%2Dadapted+Interaction+4%2C+4+%28Dec%2E+1994%29%2C+253%E2%80%93278%2E)
4. Xavier Glorot and Yoshua Bengio. 2010. Understanding the difficulty of training deep feedforward neural networks. In Proc. International Conference on Artificial Intelligence and Statistics. 249–256. [find](https://scholar.google.com/scholar?q=Xavier+Glorot+and+Yoshua+Bengio%2E+2010%2E+Understanding+the+difficulty+of+training+deep+feedforward+neural+networks%2E+In+Proc%2E+International+Conference+on+Artificial+Intelligence+and+Statistics%2E+249%E2%80%93256%2E)
5. Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. [find](https://scholar.google.com/scholar?q=Ian+Goodfellow%2C+Yoshua+Bengio%2C+and+Aaron+Courville%2E+2016%2E+Deep+Learning%2E+MIT+Press%2E)
6. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. [Deep residual learning for image recognition](https://md.research.elamur.ai/free/en/papers/he-2015-deep-residual-learning-image-recognition/). In Proc. IEEE Conference on Computer Vision and Pattern Recognition. 770–778.
7. Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long short-term memory. Neural Computation 9, 8 (Nov. 1997), 1735–1780. [find](https://scholar.google.com/scholar?q=Sepp+Hochreiter+and+J%C3%BCrgen+Schmidhuber%2E+1997%2E+Long+short%2Dterm+memory%2E+Neural+Computation+9%2C+8+%28Nov%2E+1997%29%2C+1735%E2%80%931780%2E)
8. MM Khajah, Y Huang, JP González-Brenes, MC Mozer, and P Brusilovsky. 2014. Integrating knowledge tracing and item response theory: A tale of two frameworks. In Proc. International Workshop on Personalization Approaches in Learning Environments, Vol. 1181. 7–15. [find](https://scholar.google.com/scholar?q=MM+Khajah%2C+Y+Huang%2C+JP+Gonz%C3%A1lez%2DBrenes%2C+MC+Mozer%2C+and+P+Brusilovsky%2E+2014%2E+Integrating+knowledge+tracing+and+item+response+theory%3A+A+tale+of+two+frameworks%2E+In+Proc%2E+International+Workshop+on+Personal)
9. Mohammad Khajah, Robert Lindsey, and Michael Mozer. 2016. [How deep is knowledge tracing?](https://md.research.elamur.ai/free/en/papers/khajah-2016-deep-knowledge-tracing/). In Proc. International Conference on Educational Data Mining. 94–101.
10. Diederik P Kingma and Jimmy Ba. 2015. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). In Proc. International Conference on Learning Representations.
11. Andrew Lan and Richard Baraniuk. 2016. A Contextual Bandits Framework for Personalized Learning Action Selection. In Proc. International Conference on Educational Data Mining. 424–429. [find](https://scholar.google.com/scholar?q=Andrew+Lan+and+Richard+Baraniuk%2E+2016%2E+A+Contextual+Bandits+Framework+for+Personalized+Learning+Action+Selection%2E+In+Proc%2E+International+Conference+on+Educational+Data+Mining%2E+424%E2%80%93429%2E)
12. Andrew Lan, Tom Goldstein, Richard Baraniuk, and Christoph Studer. 2016. Dealbreaker: A nonlinear latent variable model for educational data. In Proc. International Conference on Machine Learning. 266–275. [find](https://scholar.google.com/scholar?q=Andrew+Lan%2C+Tom+Goldstein%2C+Richard+Baraniuk%2C+and+Christoph+Studer%2E+2016%2E+Dealbreaker%3A+A+nonlinear+latent+variable+model+for+educational+data%2E+In+Proc%2E+International+Conference+on+Machine+Learning%2E+266)
13. Andrew Lan, Christoph Studer, and Richard Baraniuk. 2014. Time-varying learning and content analytics via sparse factor analysis. In Proc. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 452–461. [find](https://scholar.google.com/scholar?q=Andrew+Lan%2C+Christoph+Studer%2C+and+Richard+Baraniuk%2E+2014%2E+Time%2Dvarying+learning+and+content+analytics+via+sparse+factor+analysis%2E+In+Proc%2E+ACM+SIGKDD+International+Conference+on+Knowledge+Discovery+an)
14. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. [Layer normalization](https://md.research.elamur.ai/free/en/papers/ba-2016-layer-normalization/). arXiv preprint arXiv:1607.06450 (July 2016).
15. Robert Lindsey, Jeffery Shroyer, Harold Pashler, and Michael Mozer. 2014. Improving students’ long-term knowledge retention through personalized review. Psychological Science 25, 3 (Jan. 2014), 639–647. [find](https://scholar.google.com/scholar?q=Robert+Lindsey%2C+Jeffery+Shroyer%2C+Harold+Pashler%2C+and+Michael+Mozer%2E+2014%2E+Improving+students%E2%80%99+long%2Dterm+knowledge+retention+through+personalized+review%2E+Psychological+Science+25%2C+3+%28Jan%2E+2014%29%2C+639%E2%80%9364)
16. Frederick Lord. 1980. Applications of Item Response Theory to Practical Testing Problems. Erlbaum Associates. [find](https://scholar.google.com/scholar?q=Frederick+Lord%2E+1980%2E+Applications+of+Item+Response+Theory+to+Practical+Testing+Problems%2E+Erlbaum+Associates%2E)
17. Michael Mozer, Denis Kazakov, and Robert Lindsey. 2017. Discrete-event continuous-time recurrent nets. arXiv preprint arXiv:1710.04110 (Oct. 2017). — [arXiv:1710.04110](https://arxiv.org/abs/1710.04110) [find](https://scholar.google.com/scholar?q=Michael+Mozer%2C+Denis+Kazakov%2C+and+Robert+Lindsey%2E+2017%2E+Discrete%2Devent+continuous%2Dtime+recurrent+nets%2E+arXiv+preprint+arXiv%3A1710%2E04110+%28Oct%2E+2017%29%2E+%E2%80%94+%5BarXiv%3A1710%2E04110%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1710%2E0411)
18. Shalini Pandey and George Karypis. 2019. [A self attentive model for knowledge tracing](https://md.research.elamur.ai/free/en/papers/pandey-2019-self-attentive-model-knowledge-tracing/). In Proc. International Conference on Educational Data Mining. 384–389.
19. Zachary Pardos and Neil Heffernan. 2010. Modeling individualization in a Bayesian networks implementation of knowledge tracing. In Proc. International Conference on User Modeling, Adaptation, and Personalization. 255–266. [find](https://scholar.google.com/scholar?q=Zachary+Pardos+and+Neil+Heffernan%2E+2010%2E+Modeling+individualization+in+a+Bayesian+networks+implementation+of+knowledge+tracing%2E+In+Proc%2E+International+Conference+on+User+Modeling%2C+Adaptation%2C+and+Pers)
20. Razvan Pascanu, Tomas Mikolov, and Yoshua Bengio. 2013. On the difficulty of training recurrent neural networks. In Proc. International Conference on Machine Learning. 1310–1318. [find](https://scholar.google.com/scholar?q=Razvan+Pascanu%2C+Tomas+Mikolov%2C+and+Yoshua+Bengio%2E+2013%2E+On+the+difficulty+of+training+recurrent+neural+networks%2E+In+Proc%2E+International+Conference+on+Machine+Learning%2E+1310%E2%80%931318%2E)
21. Harold Pashler, Nicholas Cepeda, Robert Lindsey, Ed Vul, and Michael Mozer. 2009. Predicting the optimal spacing of study: A multiscale context model of memory. In Proc. Conference on Advances in Neural Information Processing Systems. 1321–1329. [find](https://scholar.google.com/scholar?q=Harold+Pashler%2C+Nicholas+Cepeda%2C+Robert+Lindsey%2C+Ed+Vul%2C+and+Michael+Mozer%2E+2009%2E+Predicting+the+optimal+spacing+of+study%3A+A+multiscale+context+model+of+memory%2E+In+Proc%2E+Conference+on+Advances+in+Neur)
22. Philip Pavlik Jr, Hao Cen, and Kenneth Koedinger. 2009. Performance factors analysis–A new alternative to knowledge tracing. In Proc. International Conference on Artificial Intelligence in Education. 531–538. [find](https://scholar.google.com/scholar?q=Philip+Pavlik+Jr%2C+Hao+Cen%2C+and+Kenneth+Koedinger%2E+2009%2E+Performance+factors+analysis%E2%80%93A+new+alternative+to+knowledge+tracing%2E+In+Proc%2E+International+Conference+on+Artificial+Intelligence+in+Education%2E+)
23. Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J Guibas, and Jascha Sohl-Dickstein. 2015. [Deep knowledge tracing](https://md.research.elamur.ai/free/en/papers/piech-2015-deep-knowledge-tracing/). In Proc. Conference on Advances in Neural Information Processing Systems. 505–513.
24. Chris Piech, Jonathan Huang, Andy Nguyen, Mike Phulsuksombati, Mehran Sahami, and Leonidas Guibas. 2015. Learning Program Embeddings to Propagate Feedback on Student Code. In Proc. International Conference on Machine Learning. 1093–1102. [find](https://scholar.google.com/scholar?q=Chris+Piech%2C+Jonathan+Huang%2C+Andy+Nguyen%2C+Mike+Phulsuksombati%2C+Mehran+Sahami%2C+and+Leonidas+Guibas%2E+2015%2E+Learning+Program+Embeddings+to+Propagate+Feedback+on+Student+Code%2E+In+Proc%2E+International+Confe)
25. Georg Rasch. 1993. Probabilistic Models for Some Intelligence and Attainment Tests. MESA Press. [find](https://scholar.google.com/scholar?q=Georg+Rasch%2E+1993%2E+Probabilistic+Models+for+Some+Intelligence+and+Attainment+Tests%2E+MESA+Press%2E)
26. Siddharth Reddy, Igor Labutov, Siddhartha Banerjee, and Thorsten Joachims. 2016. Unbounded human learning: Optimal scheduling for spaced repetition. In Proc. ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. 1815–1824. [find](https://scholar.google.com/scholar?q=Siddharth+Reddy%2C+Igor+Labutov%2C+Siddhartha+Banerjee%2C+and+Thorsten+Joachims%2E+2016%2E+Unbounded+human+learning%3A+Optimal+scheduling+for+spaced+repetition%2E+In+Proc%2E+ACM+SIGKDD+International+Conference+on+Kno)
27. Nitish Srivastava, Geoffrey Hinton, Alex Krizhevsky, Ilya Sutskever, and Ruslan Salakhutdinov. 2014. [Dropout: A simple way to prevent neural networks from overfitting](https://md.research.elamur.ai/free/en/papers/srivastava-2014-dropout-simple-way-prevent/). The Journal of Machine Learning Research 15, 1 (June 2014), 1929– 1958.
28. Laurens Van Der Maaten. 2014. Accelerating t-SNE using tree-based algorithms. The Journal of Machine Learning Research 15, 1 (2014), 3221–3245. [find](https://scholar.google.com/scholar?q=Laurens+Van+Der+Maaten%2E+2014%2E+Accelerating+t%2DSNE+using+tree%2Dbased+algorithms%2E+The+Journal+of+Machine+Learning+Research+15%2C+1+%282014%29%2C+3221%E2%80%933245%2E)
29. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Proc. Conference on Advances in Neural Information Processing Systems. 5998–6008.
30. Jill-Jênn Vie and Hisashi Kashima. 2019. Knowledge tracing machines: Factorization machines for knowledge tracing. In Proc. AAAI Conference on Artificial Intelligence, Vol. 33. 750–757. [find](https://scholar.google.com/scholar?q=Jill%2DJ%C3%AAnn+Vie+and+Hisashi+Kashima%2E+2019%2E+Knowledge+tracing+machines%3A+Factorization+machines+for+knowledge+tracing%2E+In+Proc%2E+AAAI+Conference+on+Artificial+Intelligence%2C+Vol%2E+33%2E+750%E2%80%93757%2E)
31. Kevin Wilson, Yan Karklin, Bojian Han, and Chaitanya Ekanadham. 2016. Back to the basics: Bayesian extensions of IRT outperform neural networks for proficiency estimation. In Proc. International Conference on Educational Data Mining. 539–544. [find](https://scholar.google.com/scholar?q=Kevin+Wilson%2C+Yan+Karklin%2C+Bojian+Han%2C+and+Chaitanya+Ekanadham%2E+2016%2E+Back+to+the+basics%3A+Bayesian+extensions+of+IRT+outperform+neural+networks+for+proficiency+estimation%2E+In+Proc%2E+International+Confe)
32. Beverly Park Woolf. 2010. Building Intelligent Interactive Tutors: Student-centered Strategies for Revolutionizing E-learning. Morgan Kaufmann. [find](https://scholar.google.com/scholar?q=Beverly+Park+Woolf%2E+2010%2E+Building+Intelligent+Interactive+Tutors%3A+Student%2Dcentered+Strategies+for+Revolutionizing+E%2Dlearning%2E+Morgan+Kaufmann%2E)
33. Xiaolu Xiong, Siyuan Zhao, Eric G Van Inwegen, and Joseph E Beck. 2016. [Going deeper with deep knowledge tracing](https://md.research.elamur.ai/free/en/papers/xiong-2016-going-deeper-deep-knowledge-tracing/). International Educational Data Mining Society (2016).
34. Chun-Kit Yeung and Dit-Yan Yeung. 2018. Addressing two problems in deep knowledge tracing via prediction-consistent regularization. In Proc. ACM Conference on Learning at Scale. ACM, 5. [find](https://scholar.google.com/scholar?q=Chun%2DKit+Yeung+and+Dit%2DYan+Yeung%2E+2018%2E+Addressing+two+problems+in+deep+knowledge+tracing+via+prediction%2Dconsistent+regularization%2E+In+Proc%2E+ACM+Conference+on+Learning+at+Scale%2E+ACM%2C+5%2E)
35. Michael Yudelson, Kenneth Koedinger, and Geoffrey Gordon. 2013. Individualized Bayesian knowledge tracing models. In Proc. International Conference on Artificial Intelligence in Education. 171–180. [find](https://scholar.google.com/scholar?q=Michael+Yudelson%2C+Kenneth+Koedinger%2C+and+Geoffrey+Gordon%2E+2013%2E+Individualized+Bayesian+knowledge+tracing+models%2E+In+Proc%2E+International+Conference+on+Artificial+Intelligence+in+Education%2E+171%E2%80%93180%2E)
36. Jiani Zhang, Xingjian Shi, Irwin King, and Dit-Yan Yeung. 2017. [Dynamic key-value memory networks for knowledge tracing](https://md.research.elamur.ai/free/en/papers/zhang-2017-dynamic-key-value-memory-networks/). In Proc. International Conference on World Wide Web. 765–774.

*Updated 3 November 2026*