Skip to content
BY 4.0 license Open Access Published by De Gruyter Open Access May 16, 2022

Knowledge tracing for adaptive learning in a metacognitive tutor

May Kristine Jonson Carlon and Jeffrey S. Cross EMAIL logo
From the journal Open Education Studies


Adaptive learning is provided in intelligent tutoring systems (ITS) to enable learners with varying abilities to meet their expected learning outcomes. Despite the personalized learning afforded by ITSes using adaptive learning, learners are still susceptible to shallow learning. Introducing metacognitive tutoring to teach learners how to be aware of their knowledge can enable deeper learning. However, metacognitive tutoring on top of cognitive tutoring can lead to unsustainable cognitive loads. Using metacognitive inputs for knowledge tracing was explored for managing cognitive loads. Hidden Markov models (HMM) and artificial neural networks were used to train models on a synthetic dataset created from predetermined learner personas. The models created with metacognitive inputs were compared with the models created without said inputs. The models using metacognitive inputs performed better than the standard models while still following learning intuitions. This indicates that combining knowledge tracing and metacognitive tutoring is a viable option for improving learning outcomes. This is an important finding since online learning, which demands metacognitive skills, is becoming popular for various topics, including those that are challenging even with immediate teacher assistance.


Agustianto, K., Permanasari, A. E., Kusumawardani, S. S., & Hidayah, I. (2016). Design adaptive learning system using metacognitive strategy path for learning in classroom and intelligent tutoring systems. AIP Conference Proceedings 1755 (pp. 070012-1–070012-6. doi: 10.1063/1.4958507). AIP Publishing.Search in Google Scholar

Aleven, V. A., & Koedinger, K. R. (2000). Limitations of student control: Do students know when they need help? International Conference on Intelligent Tutoring Systems (pp. 292-303). Springer, Berlin, Heidelberg.Search in Google Scholar

Aleven, V. A., & Koedinger, K. R. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computerbased cognitive tutor. Cognitive Science, 26(2), 147-179.10.1207/s15516709cog2602_1Search in Google Scholar

Azevedo, R. (2005, December). Computer environments as metacognitive tools for enhancing learning. Educational Psychologist, 40(4), 193-197. doi: 10.1207/s15326985ep4004_1.10.1207/s15326985ep4004_1Search in Google Scholar

Baker, R. S., Corbett, A. T., & Aleven, V. (2008). More accurate student modeling through contextual estimation of slip and guess probabilities in Bayesian knowledge tracing. International Conference on Intelligent Tutoring Systems (pp. 406-415). Springer Berlin Heidelberg.10.1007/978-3-540-69132-7_44Search in Google Scholar

Baker, R. S., Gowda, S. M., Corbett, A. T., & Ocumpaugh, J. (2012). Towards Automatically Detecting Whether Student Learning Is Shallow. International Conference on Intelligent Tutoring Systems (pp. 444-453). Springer.10.1007/978-3-642-30950-2_57Search in Google Scholar

Baum, L. E. (1972). An inequality and associated maximization technique in statistical estimation for probabilistic functions of Markov processes. Inequalities, 3(1), 1-8.Search in Google Scholar

Choudhury, S., & Pattnaik, S. (2020, January). Emerging themes in e-learning: A review from the stakeholders’ perspective. Computers & Education, 144, 103657. doi: 10.1016/j.compedu.2019.103657.10.1016/j.compedu.2019.103657Search in Google Scholar

Colbeck, C. L., Cabrera, A. F., & Terenzini, P. T. (2001). Learning professional confidence: Linking teaching practices, students’ self-perceptions, and gender. The Review of Higher Education, 24(2), 173-191. doi: 10.1353/rhe.2000.0028.10.1353/rhe.2000.0028Search in Google Scholar

Corbett, A. T., & Anderson, J. R. (1994). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4), 253-278.10.1007/BF01099821Search in Google Scholar

Dempster, A. P., Laird, N. N., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39(1), 1-22. doi: 10.1111/j.2517-6161.1977.tb01600.x.10.1111/j.2517-6161.1977.tb01600.xSearch in Google Scholar

Deonovic, B., Yudelson, M., Bolsinova, M., Attali, M., & Maris, G. (2018). Learning meets assessment. Behaviormetrika, 45(2), 457-474.10.1007/s41237-018-0070-zSearch in Google Scholar

Doroudi, S., & Brunskill, E. (2017). The misidentified identifiability problem of Bayesian knowledge tracing. 10th International Conference on Educational Data Mining (pp. 143-149). Wuhan, China: International Educational Data Mining Society.Search in Google Scholar

Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34(10), 906-911. doi: 10.1037/0003-066X.34.10.906.10.1037/0003-066X.34.10.906Search in Google Scholar

Friedler, S. A., Scheidegger, C., Venkatasubramanian, S., Choudhary, S., Hamilton, E. P., & Roth, D. (2019). A comparative study of fairness-enhancing interventions in machine learning. Conference on Fairness, Accountability, and Transparency (pp. 329-338. doi: 10.1145/3287560.3287589). Association for Computing Machinery.Search in Google Scholar

Gallagher, S., & Palmer, J. (2020, September 29). The Pandemic Pushed Universities Online. The Change Was Long Overdue. Retrieved October 2020, from Harvard Business Review: in Google Scholar

Gama, C. (2004). Metacognition in interactive learning environments: The Reflection Assistant model. International Conference on Intelligent Tutoring Systems (pp. 668-677). Springer.10.1007/978-3-540-30139-4_63Search in Google Scholar

Halpern, D. F. (2013). Thought and knowledge: An introduction to critical thinking. Psychology Press.Search in Google Scholar

Halpern, D., Tubridy, S., Wang, H. Y., Gasser, C., Popp, P. O., Davachi, L., & Gureckis, T. M. (2018). Knowledge tracing using the brain. International Conference on Educational Data Mining (pp. 219-228). International Educational Data Mining Society.10.31234/ in Google Scholar

Heffernan, N. T., & Heffernan, C. L. (2014). The ASSISTments ecosystem: Building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. International Journal of Artificial Intelligence in Education, 24(4), 470-497.10.1007/s40593-014-0024-xSearch in Google Scholar

Himmelmann, L. (2010). HMM: HMM - hidden Markov models. Retrieved May 2020, from The Comprehensive R Archive Network: in Google Scholar

Ho, E. S.-C. (2009). Characteristics of East Asian learners: What we learned from PISA. Educational Research Journal, 24(2), 327-348.Search in Google Scholar

Kuhn, M. (2008). Building Predictive Models in R Using the caret Package. Journal of Statistical Software, 25(5), 1-26, doi:10.18637/jss.v028.i05.10.18637/jss.v028.i05Search in Google Scholar

Mann, H. B., & Whitney, D. R. (1947). On a test of whether two random variables is stochastically larger than the other. The Annals of Mathematical Statistics, 18(1), 50-60.10.1214/aoms/1177730491Search in Google Scholar

McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics, 114-133. doi:10.1007/BF02478259.10.1007/BF02478259Search in Google Scholar

McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2), 153-157.10.1007/BF02295996Search in Google Scholar

Nixton, T., & Yudelson, M. (2013). Functions for fitting Bayesian knowledge tracing (BKT) models from data. Retrieved May 2020, from Github: in Google Scholar

Ohtani, K., & Hisasaka, T. (2018). Beyond intelligence: a metaanalytic review of the relationship among metacognition, intelligence, and academic performance. Metacognition and Learning, 13(2), 179-212.10.1007/s11409-018-9183-8Search in Google Scholar

Pardos, Z. A., & Heffernan, N. T. (2010). Modeling individualization in a Bayesian networks implementation of knowledge tracing. International Conference on User Modeling, Adaptation, and Personalization (pp. 255-266). Springer.10.1007/978-3-642-13470-8_24Search in Google Scholar

Pardos, Z. A., & Heffernan, N. T. (2011). KT-IDEM: introducing item difficulty to the knowledge tracing model. International Conference on User Modeling, Adaptation, and Personalization (pp. 243-254). Springer.10.1007/978-3-642-22362-4_21Search in Google Scholar

Pardos, Z. A., Bergner, Y., Seaton, D. T., & Pritchard, D. E. (2013). Adapting Bayesian knowledge tracing to a massive open online course in edX. Sixth International Conference on Educational Data Mining (pp. 137-144). International Educational Data Mining Society.Search in Google Scholar

Piech, C., Bassen, J., Ganguli, S., Sahami, M., Guibas, L. J., & Sohl- Dickstein, J. (2015). Deep knowledge tracing. Advances in Neural Information Processing Systems, (pp. 505-513).Search in Google Scholar

Reise, S. P., Ainsworth, A. T., & Haviland, M. G. (2005). Item response theory: Fundamentals, applications, and promise in psychological research. Current Directions in Psychological Science, 14(2), 95-101. doi: 10.1111/j.0963-7214.2005.00342.x.10.1111/j.0963-7214.2005.00342.xSearch in Google Scholar

Roll, I., Aleven, V., McLaren, B. M., & Koedinger, K. R. (2007). Designing for metacognition—applying cognitive tutor principles to the tutoring of help seeking. Metacognition and Learning, 2(2-3), 125-140.10.1007/s11409-007-9010-0Search in Google Scholar

Rong, X. (2014). deepnet: deep learning toolkit in R. Retrieved from The Comprehensive R Archive Network: in Google Scholar

Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1), 1929-1958.Search in Google Scholar

Sweeny, K., & Shepperd, J. A. (2010). The costs of optimism and the benefits of pessimism. Emotion, 10(5), 750-753. doi: 10.1037/a0019016.10.1037/a0019016Search in Google Scholar

Tobias, S., & Everson, H. T. (2002). Knowing what you know and what you don’t: Further research on metacognitive knowledge monitoring. Research Report No. 2002-3. College Entrance Examination Board. ERIC.Search in Google Scholar

Yu, L., Schwier, J. M., Craven, R. M., Brooks, R. R., & Griffin, C. (2012). Inferring statistically significant hidden Markov models. IEEE Transactions on Knowledge and Data Engineering, 25(7), 1548-1558.10.1109/TKDE.2012.93Search in Google Scholar

Yudelson, M. V., Koedinger, K. R., & Gordon, G. J. (2013). Individualized Bayesian knowledge tracing models. International Conference on Artificial Intelligence in Education (pp. 171-180). Springer.10.1007/978-3-642-39112-5_18Search in Google Scholar

Zhao, L., & Ye, C. (2020, July). Time and Performance in Online Learning: Applying the Theoretical Perspective of Metacognition. Decision Sciences Journal of Innovative Education, 18(3), 435-455. doi:10.1111/dsji.12216.10.1111/dsji.12216Search in Google Scholar

Received: 2020-10-29
Accepted: 2022-03-09
Published Online: 2022-05-16

© 2022 May Kristine Jonson Carlon et al., published by De Gruyter

This work is licensed under the Creative Commons Attribution 4.0 International License.

Downloaded on 29.1.2023 from
Scroll Up Arrow