---
title: "Going deeper with deep knowledge tracing — Xiong 2016"
description: "This paper re-examines Deep Knowledge Tracing (DKT), a recurrent neural network method for… 16 references, cited by 7 papers here."
authors: ["Xiong"]
year: 2016
venue: "Educational Data Mining"
source: "https://eric.ed.gov/?id=ED592679"
licence: "All rights reserved — free to read at the source"
summary_licence: "Elamur Cosmic License"
canonical: "https://research.elamur.ai/free/en/papers/xiong-2016-going-deeper-deep-knowledge-tracing/"
---
[Elamur Research](https://md.research.elamur.ai/free/en/) · [this page in HTML](https://research.elamur.ai/free/en/papers/xiong-2016-going-deeper-deep-knowledge-tracing/)

[home](https://md.research.elamur.ai/free/en/) / [papers](https://md.research.elamur.ai/free/en/papers/) / [2016](https://md.research.elamur.ai/free/en/papers/2016/page/3/) / Going deeper with deep knowledge tracing

# Going deeper with deep knowledge tracing

**Authors:** Xiong · **Year:** 2016 · **Venue:** Educational Data Mining · **Licence:** All rights reserved — free to read at the source

**Links:** [Publisher's page](https://eric.ed.gov/?id=ED592679) · [PDF](https://files.eric.ed.gov/fulltext/ED592679.pdf) · [OpenAlex](https://openalex.org/W2574518178)

**Citations:** 115 citations (OpenAlex, 2026-10-14) · [7](https://md.research.elamur.ai/free/en/papers/xiong-2016-going-deeper-deep-knowledge-tracing/cited-by/) in our library, of the 149,439 papers whose bibliographies we read

## Summary
This paper re-examines Deep Knowledge Tracing (DKT), a recurrent neural network method for predicting whether students will answer intelligent-tutoring problems correctly. DKT had been reported to beat classic student models by a wide margin. The WPI authors rebuilt DKT in Python with Google's TensorFlow, feeding an LSTM-based network one-hot vectors that encode skill and correctness. They compared it with Bayesian Knowledge Tracing (BKT) and Performance Factors Analysis (PFA). They repeated the original experiment on the ASSISTments 2009-2010 skill-builder data and added an ASSISTments 2014-2015 dataset and a KDD Cup 2010 dataset.

Inspecting the data, they found three factors that inflated DKT's earlier score:
- randomly duplicated rows;
- multi-skill responses repeated once per skill, which an RNN can exploit by recalling exact sequence patterns;
- scaffolding questions that BKT and PFA did not use.

With cleaned data limited to main questions, DKT clearly beats BKT but shows no overall advantage over PFA. The work serves as a reproducibility check and a warning that preprocessing choices and data glitches can create apparent algorithmic gains.

## Key points
- The original Lua/Torch DKT reported an AUC of 0.86 on ASSISTments 2009-2010, versus 0.67 for BKT. That dataset has 525,535 response rows, 4,217 students and 124 skills.
- The three issues found were duplicated records, inconsistent multi-skill tagging (rows repeated per skill) and the inclusion of scaffolding questions ignored by BKT and PFA.
- The TensorFlow DKT uses 200 hidden units, mini-batch SGD with batches of 100 students, and dropout on the hidden layer. Code is at github.com/siyuanzhao/2016-EDM.
- On properly prepared data, DKT performs roughly on par with PFA and substantially better than BKT, while using only response sequences.
- Proposed future work includes a context-dependent DKT with student and content features, and applying the approach to wheel spinning, mastery speed and affect detection.

*Our summary of the paper, in our own words — not its text. The work belongs to its authors; read it at the links above.*

## Institutions

- [Worcester Polytechnic Institute](https://md.research.elamur.ai/free/en/universities/worcester-polytechnic-institute/) — University, Worcester, United States · 14 works · 1,103 citations

📚 Shelves (4): [🖥️ Computer Science](https://md.research.elamur.ai/free/en/shelves/computer-science/) · 19,503 works · [🤖 Artificial Intelligence](https://md.research.elamur.ai/free/en/shelves/computer-science/artificial-intelligence/) · 18,805 works · [🗄️ Information Systems](https://md.research.elamur.ai/free/en/shelves/computer-science/information-systems/) · 1,888 works · [🖥️ Computer Science Applications](https://md.research.elamur.ai/free/en/shelves/computer-science/computer-science-applications/) · 184 works

## Cite

Cite as: Xiong. 2016. Going deeper with deep knowledge tracing. Educational Data Mining.

The paper itself, not this page. BibTeX from the library's record of the work.

```bibtex
@inproceedings{xiong2016going,
  title = {{Going deeper with deep knowledge tracing}},
  author = {Xiong},
  booktitle = {Educational Data Mining},
  year = {2016},
  url = {https://eric.ed.gov/?id=ED592679}
}
```

- [← previous Hammering towards QED](https://md.research.elamur.ai/free/en/papers/blanchette-2016-hammering-qed/)
- [next → A Latent Variable Recurrent Neural Network for Discourse-Driven Language Models](https://md.research.elamur.ai/free/en/papers/ji-2016-latent-variable-recurrent-neural-network/)
- [↑ back to papers of 2016](https://md.research.elamur.ai/free/en/papers/2016/page/3/)

## References

16 references, 4 of them with a card of ours. ↗ leads to the work, «find» searches for one we have no id for yet.

1. Piech, C., Bassen, J., Huang, J., Ganguli, S., Sahami, M., Guibas, L. J., & Sohl-Dickstein, J. (2015). [Deep Knowledge Tracing](https://md.research.elamur.ai/free/en/papers/piech-2015-deep-knowledge-tracing/). In Advances in Neural Information Processing Systems (pp. 505-513).
2. Mikolov, T., & Zweig, G. (2012, July). Context dependent recurrent neural network language model. In SLT (pp. 234-239). [find](https://scholar.google.com/scholar?q=Mikolov%2C+T%2E%2C+%26+Zweig%2C+G%2E+%282012%2C+July%29%2E+Context+dependent+recurrent+neural+network+language+model%2E+In+SLT+%28pp%2E+234%2D239%29%2E)
3. Abadi, M., Agarwal, A., Barham, P., Brevdo, E., Chen, Z., Citro, C., & Ghemawat, S. (2015). TensorFlow: Large-scale machine learning on heterogeneous systems. White paper, Google Research. [find](https://scholar.google.com/scholar?q=Abadi%2C+M%2E%2C+Agarwal%2C+A%2E%2C+Barham%2C+P%2E%2C+Brevdo%2C+E%2E%2C+Chen%2C+Z%2E%2C+Citro%2C+C%2E%2C+%26+Ghemawat%2C+S%2E+%282015%29%2E+TensorFlow%3A+Large%2Dscale+machine+learning+on+heterogeneous+systems%2E+White+paper%2C+Google+Research%2E)
4. Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). [Dropout: A simple way to prevent neural networks from overfitting](https://md.research.elamur.ai/free/en/papers/srivastava-2014-dropout-simple-way-prevent/). The Journal of Machine Learning Research, 15(1), 1929-1958.
5. Open Science Collaboration. (2015). Estimating the reproducibility of psychological science. Science, 349(6251), aac4716. [find](https://scholar.google.com/scholar?q=Open+Science+Collaboration%2E+%282015%29%2E+Estimating+the+reproducibility+of+psychological+science%2E+Science%2C+349%286251%29%2C+aac4716%2E)
6. Stamper, J., Niculescu-Mizil, A., Ritter, S., Gordon, G.J., & Koedinger, K.R. (2010). Algebra I 2005-2006. Challenge data set from KDD Cup 2010 Educational Data Mining Challenge. Find it at [http://pslcdatashop.web.cmu.edu/KDDCup/downloads.jsp](http://pslcdatashop.web.cmu.edu/KDDCup/downloads.jsp) [↗](http://pslcdatashop.web.cmu.edu/KDDCup/downloads.jsp)
7. Silver, D., Huang, A., Maddison, C. J., Guez, A., Sifre, L., van den Driessche, G., ... & Dieleman, S. (2016). Mastering the game of Go with deep neural networks and tree search. Nature, 529(7587), 484-489. [find](https://scholar.google.com/scholar?q=Silver%2C+D%2E%2C+Huang%2C+A%2E%2C+Maddison%2C+C%2E+J%2E%2C+Guez%2C+A%2E%2C+Sifre%2C+L%2E%2C+van+den+Driessche%2C+G%2E%2C+%2E%2E%2E+%26+Dieleman%2C+S%2E+%282016%29%2E+Mastering+the+game+of+Go+with+deep+neural+networks+and+tree+search%2E+Nature%2C+529%287587%29%2C+48)
8. Bengio, Y., Louradour, J., Collobert, R., & Weston, J. (2009, June). [Curriculum learning](https://md.research.elamur.ai/free/en/papers/bengio-2009-curriculum-learning/). In Proceedings of the 26th annual international conference on machine learning (pp. 41-48). ACM.
9. Pavlik Jr, P. I., Cen, H., & Koedinger, K. R. (2009). Performance Factors Analysis-A New Alternative to Knowledge Tracing. Online Submission. [find](https://scholar.google.com/scholar?q=Pavlik+Jr%2C+P%2E+I%2E%2C+Cen%2C+H%2E%2C+%26+Koedinger%2C+K%2E+R%2E+%282009%29%2E+Performance+Factors+Analysis%2DA+New+Alternative+to+Knowledge+Tracing%2E+Online+Submission%2E)
10. Corbett, A. T., & Anderson, J. R. (1994). Knowledge tracing: Modeling the acquisition of procedural knowledge. User modeling and user-adapted interaction, 4(4), 253-278. [find](https://scholar.google.com/scholar?q=Corbett%2C+A%2E+T%2E%2C+%26+Anderson%2C+J%2E+R%2E+%281994%29%2E+Knowledge+tracing%3A+Modeling+the+acquisition+of+procedural+knowledge%2E+User+modeling+and+user%2Dadapted+interaction%2C+4%284%29%2C+253%2D278%2E)
11. Gong, Y., Beck, J. E., & Heffernan, N. T. (2010, June). Comparing knowledge tracing and performance factor analysis by using multiple model fitting procedures. In Intelligent tutoring systems (pp. 35-44). Springer Berlin Heidelberg. [find](https://scholar.google.com/scholar?q=Gong%2C+Y%2E%2C+Beck%2C+J%2E+E%2E%2C+%26+Heffernan%2C+N%2E+T%2E+%282010%2C+June%29%2E+Comparing+knowledge+tracing+and+performance+factor+analysis+by+using+multiple+model+fitting+procedures%2E+In+Intelligent+tutoring+systems+%28pp%2E+35%2D)
12. Heathcote, A., Brown, S., & Mewhort, D. J. K. (2000). The power law repealed: The case for an exponential law of practice. Psychonomic bulletin & review, 7(2), 185-207. [find](https://scholar.google.com/scholar?q=Heathcote%2C+A%2E%2C+Brown%2C+S%2E%2C+%26+Mewhort%2C+D%2E+J%2E+K%2E+%282000%29%2E+The+power+law+repealed%3A+The+case+for+an+exponential+law+of+practice%2E+Psychonomic+bulletin+%26+review%2C+7%282%29%2C+185%2D207%2E)
13. Cen, H., Koedinger, K., & Junker, B. (2006, June). Learning factors analysis–a general method for cognitive model evaluation and improvement. In Intelligent tutoring systems (pp. 164-175). Springer Berlin Heidelberg. — [Learning Factors Analysis – A General Method for Cognitive Model Evaluation and Improvement](https://md.research.elamur.ai/free/en/papers/cen-2006-learning-factors-analysis-general-method/)
14. Hochreiter, S., & Schmidhuber, J. (1997). Long short-term memory. Neural computation, 9(8), 1735-1780. [find](https://scholar.google.com/scholar?q=Hochreiter%2C+S%2E%2C+%26+Schmidhuber%2C+J%2E+%281997%29%2E+Long+short%2Dterm+memory%2E+Neural+computation%2C+9%288%29%2C+1735%2D1780%2E)
15. Pardos, Z. A., & Heffernan, N. T. (2011). KT-IDEM: Introducing item difficulty to the knowledge tracing model. In User Modeling, Adaption and Personalization (pp. 243-254). Springer Berlin Heidelberg. [find](https://scholar.google.com/scholar?q=Pardos%2C+Z%2E+A%2E%2C+%26+Heffernan%2C+N%2E+T%2E+%282011%29%2E+KT%2DIDEM%3A+Introducing+item+difficulty+to+the+knowledge+tracing+model%2E+In+User+Modeling%2C+Adaption+and+Personalization+%28pp%2E+243%2D254%29%2E+Springer+Berlin+Heidelberg%2E)
16. ASSISTments Data. (2015). Retrieved March 07, 2016, from [https://sites.google.com/site/assistmentsdata/home/assistment](https://sites.google.com/site/assistmentsdata/home/assistment) -2009-2010-data/skill-builder-data-2009-2010 [↗](https://sites.google.com/site/assistmentsdata/home/assistment)

*Updated 3 November 2026*