Abstract
Learning processes often occur at a situational level. Changes in learning context have implications on how students are motivated or are able to cognitively process information.
To study such situational phenomena, Experience Sampling Method (ESM) can help assess psychological variables in the moment and in context. However, data collected via ESM is voluminous and imbalanced. Special types of statistical modeling are needed to handle this unique data structure in order to maximize its potential for scientific discovery. The purpose of this paper is to illustrate how Latent State-Trait modeling used within a multilevel framework can help model complex data as derived by ESM. A study of situational engagement is presented as an illustrative case. We describe methodological considerations which facilitated the following analyses: (1) Decomposition of trait-level and state-level engagement; (2) Group differences in variance decomposition, and (3) Predicting state component of engagement. Discussions include the relative advantages and disadvantages of ESM and multilevel Latent State-Trait modeling in facilitating situational psychological research.
References
Archambault, I., Janosz, M., & Chouinard, R. (2012). Teacher beliefs as predictors of adolescents’ cognitive engagement and achievement in mathematics. Journal of Educational Research, 105(5), 319–328. https://doi.org/10.1080/00220671.2011.629694Search in Google Scholar
Bandura, A. (1997). Self-efficacy: The exercise of control. Freeman.Search in Google Scholar
Bandura, A. (2001). Social Cognitive Theory : An Agentic Perspective. Annu. Rev. Psychol., 21, 331–343.10.1146/annurev.psych.52.1.1Search in Google Scholar
Bland, J. M., & Altman, D. G. (1995). Correlation between subjects. BMJ, 310(March), 633.Search in Google Scholar
Buhs, E. S., & Ladd, G. W. (2001). Peer rejection as an antecedent of young children’s school adjustment: an examination of mediating processes. Developmental Psychology, 37(4), 550–560. https://doi.org/10.1037/0012-1649.37.4.550Search in Google Scholar
Cheung, M. W. L., & Au, K. (2005). Applications of multilevel structural equation modeling to cross-cultural research. Structural Equation Modeling, 12(4), 598–619. https://doi.org/10.1207/s15328007sem1204_5Search in Google Scholar
Crompton, H. (2013). Mobile learning: New approach, new theory. In Z.. Berge & L. Muilenburg (Eds.), Handbook of mobile learning (Vol. 1, pp. 47–58). New York: Routledge.Search in Google Scholar
Csikszentmihalyi, M., & Larson, R. (2014). Validity and reliability of the experience-sampling method. In Flow and the foundations of positive psychology (pp. 35–54). Springer.10.1007/978-94-017-9088-8_3Search in Google Scholar
Curran, P. J., & Bauer, D. J. (2011). The disaggregation of within-person and between-person effects in longitudinal models of change. Annual Review of Psychology, 62, 583–619. https://doi.org/10.1146/annurev.psych.093008.100356Search in Google Scholar
Gefen, D., Straub, D., & Boudreau, M.-C. (2000). Structural Equation Modeling and Regression: Guidelines for Research Practice. Communications of the Association for Information Systems, 4(August). https://doi.org/10.17705/1cais.00407Search in Google Scholar
Geiser, C. (2021). Longitudinal Structural Equation Modeling with Mplus: A Latent State-Trait Perspective. Guilford Press.Search in Google Scholar
Geiser, C., Bishop, J., Lockhart, G., Shiffman, S., & Grenard, J. L. (2013). Analyzing latent state-trait and multiple-indicator latent growth curve models as multilevel structural equation models. Frontiers in Psychology, 4(DEC), 1–23. https://doi.org/10.3389/fpsyg.2013.00975Search in Google Scholar
Greene, B. A. (2015). Measuring Cognitive Engagement With Self-Report Scales: Reflections From Over 20 Years of Research. Educational Psychologist, 50(1), 14–30. https://doi.org/10.1080/00461520.2014.989230Search in Google Scholar
Heddy, B. C., & Sinatra, G. M. (2017). Transformative parents: Facilitating transformative experiences and interest with a parent involvement intervention. Science Education, 101(5), 765–786. https://doi.org/10.1002/sce.21292Search in Google Scholar
Hox, J. J., Maas, C. J. M., & Brinkhuis, M. J. S. (2010). The effect of estimation method and sample size in multilevel structural equation modeling. Statistica Neerlandica, 64(2), 157–170. https://doi.org/10.1111/j.1467-9574.2009.00445.xSearch in Google Scholar
Lavigne, G. L., & Vallerand, R. J. (2010). The dynamic processes of influence between contextual and situational motivation: A test of the hierarchical model in a science education setting. Journal of Applied Social Psychology, 40(9), 2343–2359. https://doi.org/10.1111/j.1559-1816.2010.00661.xSearch in Google Scholar
Little, T. D. (2013). Longitudinal structural equation modeling. Guilford press.Search in Google Scholar
Muthen, B. (1991). Multilevel Factor Analysis of Class and Student Achievement Components Author (s): Bengt O. Muthén Source : Journal of Educational Measurement, Vol. 28, No. 4 (Winter, 1991), pp. 338-354 Published by : National Council on Measurement in Educati. Journal of Education Measurement, 28(4), 338–354.Search in Google Scholar
Preacher, K. J., Zyphur, M. J., & Zhang, Z. (2010). A general multilevel SEM framework for assessing multilevel mediation. Psychological Methods, 15(3), 209–233. https://doi.org/10.1037/a0020141Search in Google Scholar
Rabe-Hesketh, S., & Skrondal, A. (2012). Multilevel and longitudinal modeling using Stata. 3rd. College Station, Tex: Stata Press Publication.Search in Google Scholar
Ratelle, C. F., Guay, F., Vallerand, R. J., Larose, S., & Senécal, C. (2007). Autonomous, Controlled, and Amotivated Types of Academic Motivation: A Person-Oriented Analysis. Journal of Educational Psychology, 99(4), 734–746. https://doi.org/10.1037/0022-0663.99.4.734Search in Google Scholar
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). sage.Search in Google Scholar
Schmidt, J. A., Rosenberg, J. M., & Beymer, P. N. (2018). A person-in-context approach to student engagement in science: Examining learning activities and choice. Journal of Research in Science Teaching, 55(1), 19–43. https://doi.org/10.1002/tea.21409Search in Google Scholar
Schmidt, J., Kackar-Cam, H., Strati, A., & Shumow, L. (2015). The Role of Challenge in Students’ Engagement and Competence in High School Science Classrooms: Hispanic and Non-Hispanic Whites Compared. NCSSS Journal, 20(1), 20–26.Search in Google Scholar
Shumow, L., Schmidt, J. A., & Kackar, H. (2008). Reading In Class \& Out of Class: An Experience Sampling Method Study. Middle Grades Research Journal, 3(3).Search in Google Scholar
Shumow, L., Schmidt, J. A., & Zaleski, D. J. (2013). Multiple Perspectives on Student Learning, Engagement, and Motivation in High School Biology Labs Published by : University of North Carolina Press Stable URL : https://www.jstor.org/stable/23351976 REFERENCES Linked references are available on JSTOR fo. 96(3), 232–252.Search in Google Scholar
Sinatra, G. M., Heddy, B. C., & Lombardi, D. (2015). The Challenges of Defining and Measuring Student Engagement in Science. Educational Psychologist, 50(1). https://doi.org/10.1080/00461520.2014.1002924Search in Google Scholar
Steyer, R., Ferring, D., & Schmitt, M. J. (1992). States and traits in psychological assessment. European Journal of Psychological Assessment.Search in Google Scholar
Steyer, R., Mayer, A., Geiser, C., & Cole, D. A. (2015). A theory of states and traits—Revised. Annual Review of Clinical Psychology, 11, 71–98.10.1146/annurev-clinpsy-032813-153719Search in Google Scholar
Walker, C. O., Greene, B. A., & Mansell, R. A. (2006). Identification with academics, intrinsic/extrinsic motivation, and self-efficacy as predictors of cognitive engagement. Learning and Individual Differences, 16(1), 1–12. https://doi.org/10.1016/j.lindif.2005.06.004Search in Google Scholar
Wang, M. Te, & Eccles, J. S. (2012). Social Support Matters: Longitudinal Effects of Social Support on Three Dimensions of School Engagement From Middle to High School. Child Development, 83(3), 877–895. https://doi.org/10.1111/j.1467-8624.2012.01745.xSearch in Google Scholar
Wang, M. Te, Fredricks, J. A., Ye, F., Hofkens, T. L., & Linn, J. S. (2016). The Math and Science Engagement Scales: Scale development, Validation, And psychometric properties. Learning and Instruction, 43, 16–26. https://doi.org/10.1016/j.learninstruc.2016.01.008Search in Google Scholar
Xie, K., Heddy, B. C., & Greene, B. A. (2019). Affordances of using mobile technology to support experience-sampling method in examining college students’ engagement. Computers & Education, 128, 183-198.10.1016/j.compedu.2018.09.020Search in Google Scholar
Xie, K., Heddy, B. C., & Vongkulluksn, V. W. (2019). Examining engagement in context using experience-sampling method with mobile technology. Contemporary Educational Psychology, 59, 101788.10.1016/j.cedpsych.2019.101788Search in Google Scholar
Xie, K., Hensley, L. C., Law, V., & Sun, Z. (2019). Self-regulation as a function of perceived leadership and cohesion in small group online collaborative learning. British Journal of Educational Technology, 50(1), 456-468.10.1111/bjet.12594Search in Google Scholar
Xie, K., Vongkulluksn, V. W., Lu, L., & Cheng, S. L. (2020). A person-centered approach to examining high-school students’ motivation, engagement and academic performance. Contemporary Educational Psychology, 62, 101877.10.1016/j.cedpsych.2020.101877Search in Google Scholar
Zhu, X., Chen, A., Ennis, C., Sun, H., Hopple, C., Bonello, M., Bae, M., & Kim, S. (2009). Situational interest, cognitive engagement, and achievement in physical education. Contemporary Educational Psychology, 34(3), 221–229. https://doi.org/10.1016/j.cedpsych.2009.05.002Search in Google Scholar
Zirkel, S., Garcia, J. A., & Murphy, M. C. (2015). Experience-Sampling Research Methods and Their Potential for Education Research. Educational Researcher, 44(1), 7–16. https://doi.org/10.3102/0013189X14566879Search in Google Scholar
© 2022 Vanessa W. Vongkulluksn et al., published by De Gruyter
This work is licensed under the Creative Commons Attribution 4.0 International License.
