---
title: "A self-attentive model for knowledge tracing — Pandey 2019"
description: "Pandey and Karypis propose SAKT (Self Attentive Knowledge Tracing), a Transformer-style model… 11 references, cited by 21 papers here."
authors: ["Pandey"]
year: 2019
venue: "arXiv (Cornell University)"
source: "https://arxiv.org/abs/1907.06837"
arxiv: "1907.06837"
licence: "arXiv — non-exclusive licence to distribute"
licence_url: "http://arxiv.org/licenses/nonexclusive-distrib/1.0/"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/pandey-2019-self-attentive-model-knowledge-tracing/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/pandey-2019-self-attentive-model-knowledge-tracing/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/) / A self-attentive model for knowledge tracing

# A self-attentive model for knowledge tracing

**Authors:** Pandey · **Year:** 2019 · **Venue:** arXiv (Cornell University) · **Licence:** [arXiv — non-exclusive licence to distribute](http://arxiv.org/licenses/nonexclusive-distrib/1.0/)

**Links:** [Publisher's page](https://arxiv.org/abs/1907.06837) · [PDF](https://arxiv.org/pdf/1907.06837) · [arXiv 1907.06837](https://arxiv.org/abs/1907.06837) · [OpenAlex](https://openalex.org/W2957747000)

**Citations:** 192 citations (OpenAlex, 2026-10-14) · [21](https://md.research.elamur.ai/free/en/papers/pandey-2019-self-attentive-model-knowledge-tracing/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This paper addresses knowledge tracing: estimating how well a learner has mastered knowledge concepts (KCs) by predicting whether they will answer the next exercise correctly. Earlier RNN-based approaches such as DKT and DKVMN generalize poorly on sparse real-world interaction logs, where each student touches only a small share of the KCs. Pandey and Karypis propose SAKT (Self Attentive Knowledge Tracing), a Transformer-style model that uses no recurrence. Past exercise–answer pairs are embedded as keys and values, and the exercise being predicted serves as the query. Causal scaled dot-product attention selects the few earlier interactions most relevant to the current exercise. The architecture adds learned positional embeddings, multiple heads, a feed-forward layer, residual connections, layer normalization and a sigmoid output, and is trained with cross-entropy. The authors evaluate SAKT on one synthetic and four real-world datasets. It beats prior state-of-the-art models by 4.43% AUC on average and trains roughly an order of magnitude faster. The learned attention weights also reveal interpretable relations between exercises.

## Key points
- **Data:** Synthetic (4,000 simulated students, 50 exercises from 5 hidden concepts), ASSIST2009 (skill-builder, density 0.06), ASSIST2015 (19,917 students, 708,631 interactions, 100 skills, density 0.05), ASSISTChall and STATICS.
- **Input encoding:** each interaction is encoded as a single index, e + r×E, which gives 2E possible values. Sequences are left-padded or split into chunks of length n.
- **Interpretability:** on Synthetic, attention weights cluster exercises perfectly by their hidden concept, even linking exercises that sit far apart in the sequence (e.g., exercise 22 to exercise 5).
- **Ablation (d = 200, 5 heads by default):** removing the attention block hurts the most. A single head is consistently worse. A second block brings no gain. Residual connections matter little, and dropping them slightly improves ASSIST2015.
- **Speed (NVIDIA Titan V):** SAKT takes 1.4 s per epoch, compared with 65 s for DKT+, 45 s for DKT and 26 s for DKVMN.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [University of Minnesota](https://md.research.elamur.ai/free/en/universities/university-of-minnesota/) — University, Minneapolis, United States · 30 works · 2,466 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,888 works · [🖥️ Computer Science Applications](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-science-applications/) · 184 works

## Cite

Cite as: Pandey. 2019. A self-attentive model for knowledge tracing. arXiv (Cornell University).

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@misc{pandey2019self,
  title = {{A self-attentive model for knowledge tracing}},
  author = {Pandey},
  year = {2019},
  eprint = {1907.06837},
  archivePrefix = {arXiv},
  url = {https://arxiv.org/abs/1907.06837}
}
```

- [← previous Extractive Summarization of Long Documents by Combining Global and Local Context](https://md.research.elamur.ai/free/en/papers/xiao-2019-extractive-summarization-long-documents-combining/)
- [next → Do Massively Pretrained Language Models Make Better Storytellers?](https://md.research.elamur.ai/free/en/papers/see-2019-massively-pretrained-language-models-make/)
- [↑ back to papers of 2019](https://md.research.elamur.ai/free/en/papers/2019/page/4/)

## References

11 references, 7 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Jimmy Lei Ba, Jamie Ryan Kiros, and Geoffrey E Hinton. 2016. [Layer normalization](https://md.research.elamur.ai/free/en/papers/ba-2016-layer-normalization/). arXiv preprint Figure 4: Training Efficiency on ASSIST2009 dataset. arXiv:1607.06450 (2016).
2. Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. [Deep residual learning for image recognition](https://md.research.elamur.ai/free/en/papers/he-2015-deep-residual-learning-image-recognition/). In Proceedings of the IEEE conference on computer vision and pattern recognition. 770–778.
3. Wang-Cheng Kang and Julian McAuley. 2018. Self-Attentive Sequential Recommendation. CoRR abs/1808.09781 (2018). — [arXiv:1808.09781](https://arxiv.org/abs/1808.09781) [find](https://scholar.google.com/scholar?q=Wang%2DCheng+Kang+and+Julian+McAuley%2E+2018%2E+Self%2DAttentive+Sequential+Recommendation%2E+CoRR+abs%2F1808%2E09781+%282018%29%2E+%E2%80%94+%5BarXiv%3A1808%2E09781%5D%28https%3A%2F%2Farxiv%2Eorg%2Fabs%2F1808%2E09781%29)
4. Mohammad Khajah, Robert V Lindsey, and Michael C Mozer. 2016. [How deep is knowledge tracing?](https://md.research.elamur.ai/free/en/papers/khajah-2016-deep-knowledge-tracing/) arXiv preprint arXiv:1604.02416 (2016).
5. Diederik P Kingma and Jimmy Ba. 2014. [Adam: A method for stochastic optimization](https://md.research.elamur.ai/free/en/papers/kingma-2014-adam-method-stochastic-optimization/). arXiv preprint arXiv:1412.6980 (2014).
6. Chris Piech, Jonathan Bassen, Jonathan Huang, Surya Ganguli, Mehran Sahami, Leonidas J Guibas, and Jascha Sohl-Dickstein. 2015. [Deep knowledge tracing](https://md.research.elamur.ai/free/en/papers/piech-2015-deep-knowledge-tracing/). In Advances in Neural Information Processing Systems. 505–513.
7. Adam Santoro, Sergey Bartunov, Matthew Botvinick, Daan Wierstra, and Timothy Lillicrap. 2016. One-shot learning with memory-augmented neural networks. arXiv preprint arXiv:1605.06065 (2016). [find](https://scholar.google.com/scholar?q=Adam+Santoro%2C+Sergey+Bartunov%2C+Matthew+Botvinick%2C+Daan+Wierstra%2C+and+Timothy+Lillicrap%2E+2016%2E+One%2Dshot+learning+with+memory%2Daugmented+neural+networks%2E+arXiv+preprint+arXiv%3A1605%2E06065+%282016%29%2E)
8. John Self. 1990. Theoretical foundations for intelligent tutoring systems. Journal of Artificial Intelligence in Education 1, 4 (1990), 3–14. [find](https://scholar.google.com/scholar?q=John+Self%2E+1990%2E+Theoretical+foundations+for+intelligent+tutoring+systems%2E+Journal+of+Artificial+Intelligence+in+Education+1%2C+4+%281990%29%2C+3%E2%80%9314%2E)
9. Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Lukasz Kaiser, and Illia Polosukhin. 2017. [Attention is all you need](https://md.research.elamur.ai/free/en/papers/vaswani-2017-attention-all-need/). In Advances in Neural Information Processing Systems. 5998–6008.
10. Chun-Kit Yeung and Dit-Yan Yeung. 2018. Addressing two problems in deep knowledge tracing via prediction-consistent regularization. arXiv preprint arXiv:1806.02180 (2018). — [arXiv:1806.02180](https://arxiv.org/abs/1806.02180) [find](https://scholar.google.com/scholar?q=Chun%2DKit+Yeung+and+Dit%2DYan+Yeung%2E+2018%2E+Addressing+two+problems+in+deep+knowledge+tracing+via+prediction%2Dconsistent+regularization%2E+arXiv+preprint+arXiv%3A1806%2E02180+%282018%29%2E+%E2%80%94+%5BarXiv%3A1806%2E02180%5D%28https%3A%2F)
11. Jiani Zhang, Xingjian Shi, Irwin King, and Dit-Yan Yeung. [Dynamic key-value memory networks for knowledge tracing](https://md.research.elamur.ai/free/en/papers/zhang-2017-dynamic-key-value-memory-networks/). In Proceedings of the 26th International Conference on World Wide Web.

*Updated 3 November 2026*