Abstract
Adaptive learning is provided in intelligent tutoring systems (ITS) to enable learners with varying abilities to meet their expected learning outcomes. Despite the personalized learning afforded by ITSes using adaptive learning, learners are still susceptible to shallow learning. Introducing metacognitive tutoring to teach learners how to be aware of their knowledge can enable deeper learning. However, metacognitive tutoring on top of cognitive tutoring can lead to unsustainable cognitive loads. Using metacognitive inputs for knowledge tracing was explored for managing cognitive loads. Hidden Markov models (HMM) and artificial neural networks were used to train models on a synthetic dataset created from predetermined learner personas. The models created with metacognitive inputs were compared with the models created without said inputs. The models using metacognitive inputs performed better than the standard models while still following learning intuitions. This indicates that combining knowledge tracing and metacognitive tutoring is a viable option for improving learning outcomes. This is an important finding since online learning, which demands metacognitive skills, is becoming popular for various topics, including those that are challenging even with immediate teacher assistance.
References
Agustianto, K., Permanasari, A. E., Kusumawardani, S. S., & Hidayah, I. (2016). Design adaptive learning system using metacognitive strategy path for learning in classroom and intelligent tutoring systems. AIP Conference Proceedings 1755 (pp. 070012-1–070012-6. doi: 10.1063/1.4958507). AIP Publishing.Search in Google Scholar
Aleven, V. A., & Koedinger, K. R. (2000). Limitations of student control: Do students know when they need help? International Conference on Intelligent Tutoring Systems (pp. 292-303). Springer, Berlin, Heidelberg.Search in Google Scholar
Aleven, V. A., & Koedinger, K. R. (2002). An effective metacognitive strategy: Learning by doing and explaining with a computerbased cognitive tutor. Cognitive Science, 26(2), 147-179.10.1207/s15516709cog2602_1Search in Google Scholar
Azevedo, R. (2005, December). Computer environments as metacognitive tools for enhancing learning. Educational Psychologist, 40(4), 193-197. doi: 10.1207/s15326985ep4004_1.10.1207/s15326985ep4004_1Search in Google Scholar
Baker, R. S., Corbett, A. T., & Aleven, V. (2008). More accurate student modeling through contextual estimation of slip and guess probabilities in Bayesian knowledge tracing. International Conference on Intelligent Tutoring Systems (pp. 406-415). Springer Berlin Heidelberg.10.1007/978-3-540-69132-7_44Search in Google Scholar
Baker, R. S., Gowda, S. M., Corbett, A. T., & Ocumpaugh, J. (2012). Towards Automatically Detecting Whether Student Learning Is Shallow. International Conference on Intelligent Tutoring Systems (pp. 444-453). Springer.10.1007/978-3-642-30950-2_57Search in Google Scholar
Baum, L. E. (1972). An inequality and associated maximization technique in statistical estimation for probabilistic functions of Markov processes. Inequalities, 3(1), 1-8.Search in Google Scholar
Choudhury, S., & Pattnaik, S. (2020, January). Emerging themes in e-learning: A review from the stakeholders’ perspective. Computers & Education, 144, 103657. doi: 10.1016/j.compedu.2019.103657.10.1016/j.compedu.2019.103657Search in Google Scholar
Colbeck, C. L., Cabrera, A. F., & Terenzini, P. T. (2001). Learning professional confidence: Linking teaching practices, students’ self-perceptions, and gender. The Review of Higher Education, 24(2), 173-191. doi: 10.1353/rhe.2000.0028.10.1353/rhe.2000.0028Search in Google Scholar
Corbett, A. T., & Anderson, J. R. (1994). Knowledge tracing: Modeling the acquisition of procedural knowledge. User Modeling and User-Adapted Interaction, 4(4), 253-278.10.1007/BF01099821Search in Google Scholar
Dempster, A. P., Laird, N. N., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society: Series B (Methodological), 39(1), 1-22. doi: 10.1111/j.2517-6161.1977.tb01600.x.10.1111/j.2517-6161.1977.tb01600.xSearch in Google Scholar
Deonovic, B., Yudelson, M., Bolsinova, M., Attali, M., & Maris, G. (2018). Learning meets assessment. Behaviormetrika, 45(2), 457-474.10.1007/s41237-018-0070-zSearch in Google Scholar
Doroudi, S., & Brunskill, E. (2017). The misidentified identifiability problem of Bayesian knowledge tracing. 10th International Conference on Educational Data Mining (pp. 143-149). Wuhan, China: International Educational Data Mining Society.Search in Google Scholar
Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34(10), 906-911. doi: 10.1037/0003-066X.34.10.906.10.1037/0003-066X.34.10.906Search in Google Scholar
Friedler, S. A., Scheidegger, C., Venkatasubramanian, S., Choudhary, S., Hamilton, E. P., & Roth, D. (2019). A comparative study of fairness-enhancing interventions in machine learning. Conference on Fairness, Accountability, and Transparency (pp. 329-338. doi: 10.1145/3287560.3287589). Association for Computing Machinery.Search in Google Scholar
Gallagher, S., & Palmer, J. (2020, September 29). The Pandemic Pushed Universities Online. The Change Was Long Overdue. Retrieved October 2020, from Harvard Business Review: https://hbr.org/2020/09/the-pandemic-pushed-universitiesonline-the-change-was-long-overdueSearch in Google Scholar
Gama, C. (2004). Metacognition in interactive learning environments: The Reflection Assistant model. International Conference on Intelligent Tutoring Systems (pp. 668-677). Springer.10.1007/978-3-540-30139-4_63Search in Google Scholar
Halpern, D. F. (2013). Thought and knowledge: An introduction to critical thinking. Psychology Press.Search in Google Scholar
Halpern, D., Tubridy, S., Wang, H. Y., Gasser, C., Popp, P. O., Davachi, L., & Gureckis, T. M. (2018). Knowledge tracing using the brain. International Conference on Educational Data Mining (pp. 219-228). International Educational Data Mining Society.10.31234/osf.io/fmj48Search in Google Scholar
Heffernan, N. T., & Heffernan, C. L. (2014). The ASSISTments ecosystem: Building a platform that brings scientists and teachers together for minimally invasive research on human learning and teaching. International Journal of Artificial Intelligence in Education, 24(4), 470-497.10.1007/s40593-014-0024-xSearch in Google Scholar
Himmelmann, L. (2010). HMM: HMM - hidden Markov models. Retrieved May 2020, from The Comprehensive R Archive Network: https://CRAN.R-project.org/package=HMMSearch in Google Scholar
Ho, E. S.-C. (2009). Characteristics of East Asian learners: What we learned from PISA. Educational Research Journal, 24(2), 327-348.Search in Google Scholar
Kuhn, M. (2008). Building Predictive Models in R Using the caret Package. Journal of Statistical Software, 25(5), 1-26, doi:10.18637/jss.v028.i05.10.18637/jss.v028.i05Search in Google Scholar
Mann, H. B., & Whitney, D. R. (1947). On a test of whether two random variables is stochastically larger than the other. The Annals of Mathematical Statistics, 18(1), 50-60.10.1214/aoms/1177730491Search in Google Scholar
McCulloch, W. S., & Pitts, W. (1943). A logical calculus of the ideas immanent in nervous activity. The bulletin of mathematical biophysics, 114-133. doi:10.1007/BF02478259.10.1007/BF02478259Search in Google Scholar
McNemar, Q. (1947). Note on the sampling error of the difference between correlated proportions or percentages. Psychometrika, 12(2), 153-157.10.1007/BF02295996Search in Google Scholar
Nixton, T., & Yudelson, M. (2013). Functions for fitting Bayesian knowledge tracing (BKT) models from data. Retrieved May 2020, from Github: https://github.com/IEDMS/REDM/tree/master/bktSearch in Google Scholar
Ohtani, K., & Hisasaka, T. (2018). Beyond intelligence: a metaanalytic review of the relationship among metacognition, intelligence, and academic performance. Metacognition and Learning, 13(2), 179-212.10.1007/s11409-018-9183-8Search in Google Scholar
Pardos, Z. A., & Heffernan, N. T. (2010). Modeling individualization in a Bayesian networks implementation of knowledge tracing. International Conference on User Modeling, Adaptation, and Personalization (pp. 255-266). Springer.10.1007/978-3-642-13470-8_24Search in Google Scholar
Pardos, Z. A., & Heffernan, N. T. (2011). KT-IDEM: introducing item difficulty to the knowledge tracing model. International Conference on User Modeling, Adaptation, and Personalization (pp. 243-254). Springer.10.1007/978-3-642-22362-4_21Search in Google Scholar
Pardos, Z. A., Bergner, Y., Seaton, D. T., & Pritchard, D. E. (2013). Adapting Bayesian knowledge tracing to a massive open online course in edX. Sixth International Conference on Educational Data Mining (pp. 137-144). International Educational Data Mining Society.Search in Google Scholar
Piech, C., Bassen, J., Ganguli, S., Sahami, M., Guibas, L. J., & Sohl- Dickstein, J. (2015). Deep knowledge tracing. Advances in Neural Information Processing Systems, (pp. 505-513).Search in Google Scholar
Reise, S. P., Ainsworth, A. T., & Haviland, M. G. (2005). Item response theory: Fundamentals, applications, and promise in psychological research. Current Directions in Psychological Science, 14(2), 95-101. doi: 10.1111/j.0963-7214.2005.00342.x.10.1111/j.0963-7214.2005.00342.xSearch in Google Scholar
Roll, I., Aleven, V., McLaren, B. M., & Koedinger, K. R. (2007). Designing for metacognition—applying cognitive tutor principles to the tutoring of help seeking. Metacognition and Learning, 2(2-3), 125-140.10.1007/s11409-007-9010-0Search in Google Scholar
Rong, X. (2014). deepnet: deep learning toolkit in R. Retrieved from The Comprehensive R Archive Network: https://CRAN.R-project.org/package=deepnetSearch in Google Scholar
Srivastava, N., Hinton, G., Krizhevsky, A., Sutskever, I., & Salakhutdinov, R. (2014). Dropout: a simple way to prevent neural networks from overfitting. Journal of Machine Learning Research, 15(1), 1929-1958.Search in Google Scholar
Sweeny, K., & Shepperd, J. A. (2010). The costs of optimism and the benefits of pessimism. Emotion, 10(5), 750-753. doi: 10.1037/a0019016.10.1037/a0019016Search in Google Scholar
Tobias, S., & Everson, H. T. (2002). Knowing what you know and what you don’t: Further research on metacognitive knowledge monitoring. Research Report No. 2002-3. College Entrance Examination Board. ERIC.Search in Google Scholar
Yu, L., Schwier, J. M., Craven, R. M., Brooks, R. R., & Griffin, C. (2012). Inferring statistically significant hidden Markov models. IEEE Transactions on Knowledge and Data Engineering, 25(7), 1548-1558.10.1109/TKDE.2012.93Search in Google Scholar
Yudelson, M. V., Koedinger, K. R., & Gordon, G. J. (2013). Individualized Bayesian knowledge tracing models. International Conference on Artificial Intelligence in Education (pp. 171-180). Springer.10.1007/978-3-642-39112-5_18Search in Google Scholar
Zhao, L., & Ye, C. (2020, July). Time and Performance in Online Learning: Applying the Theoretical Perspective of Metacognition. Decision Sciences Journal of Innovative Education, 18(3), 435-455. doi:10.1111/dsji.12216.10.1111/dsji.12216Search in Google Scholar
© 2022 May Kristine Jonson Carlon et al., published by De Gruyter
This work is licensed under the Creative Commons Attribution 4.0 International License.
