Skip to content
BY 4.0 license Open Access Published by De Gruyter Open Access July 5, 2022

Multilevel Latent State-Trait Models with Experience Sampling Data: An Illustrative Case of Examining Situational Engagement

  • Vanessa W. Vongkulluksn EMAIL logo and Kui Xie
From the journal Open Education Studies


Learning processes often occur at a situational level. Changes in learning context have implications on how students are motivated or are able to cognitively process information.

To study such situational phenomena, Experience Sampling Method (ESM) can help assess psychological variables in the moment and in context. However, data collected via ESM is voluminous and imbalanced. Special types of statistical modeling are needed to handle this unique data structure in order to maximize its potential for scientific discovery. The purpose of this paper is to illustrate how Latent State-Trait modeling used within a multilevel framework can help model complex data as derived by ESM. A study of situational engagement is presented as an illustrative case. We describe methodological considerations which facilitated the following analyses: (1) Decomposition of trait-level and state-level engagement; (2) Group differences in variance decomposition, and (3) Predicting state component of engagement. Discussions include the relative advantages and disadvantages of ESM and multilevel Latent State-Trait modeling in facilitating situational psychological research.


Archambault, I., Janosz, M., & Chouinard, R. (2012). Teacher beliefs as predictors of adolescents’ cognitive engagement and achievement in mathematics. Journal of Educational Research, 105(5), 319–328. in Google Scholar

Bandura, A. (1997). Self-efficacy: The exercise of control. Freeman.Search in Google Scholar

Bandura, A. (2001). Social Cognitive Theory : An Agentic Perspective. Annu. Rev. Psychol., 21, 331–343.10.1146/annurev.psych.52.1.1Search in Google Scholar

Bland, J. M., & Altman, D. G. (1995). Correlation between subjects. BMJ, 310(March), 633.Search in Google Scholar

Buhs, E. S., & Ladd, G. W. (2001). Peer rejection as an antecedent of young children’s school adjustment: an examination of mediating processes. Developmental Psychology, 37(4), 550–560. in Google Scholar

Cheung, M. W. L., & Au, K. (2005). Applications of multilevel structural equation modeling to cross-cultural research. Structural Equation Modeling, 12(4), 598–619. in Google Scholar

Crompton, H. (2013). Mobile learning: New approach, new theory. In Z.. Berge & L. Muilenburg (Eds.), Handbook of mobile learning (Vol. 1, pp. 47–58). New York: Routledge.Search in Google Scholar

Csikszentmihalyi, M., & Larson, R. (2014). Validity and reliability of the experience-sampling method. In Flow and the foundations of positive psychology (pp. 35–54). Springer.10.1007/978-94-017-9088-8_3Search in Google Scholar

Curran, P. J., & Bauer, D. J. (2011). The disaggregation of within-person and between-person effects in longitudinal models of change. Annual Review of Psychology, 62, 583–619. in Google Scholar

Gefen, D., Straub, D., & Boudreau, M.-C. (2000). Structural Equation Modeling and Regression: Guidelines for Research Practice. Communications of the Association for Information Systems, 4(August). in Google Scholar

Geiser, C. (2021). Longitudinal Structural Equation Modeling with Mplus: A Latent State-Trait Perspective. Guilford Press.Search in Google Scholar

Geiser, C., Bishop, J., Lockhart, G., Shiffman, S., & Grenard, J. L. (2013). Analyzing latent state-trait and multiple-indicator latent growth curve models as multilevel structural equation models. Frontiers in Psychology, 4(DEC), 1–23. in Google Scholar

Greene, B. A. (2015). Measuring Cognitive Engagement With Self-Report Scales: Reflections From Over 20 Years of Research. Educational Psychologist, 50(1), 14–30. in Google Scholar

Heddy, B. C., & Sinatra, G. M. (2017). Transformative parents: Facilitating transformative experiences and interest with a parent involvement intervention. Science Education, 101(5), 765–786. in Google Scholar

Hox, J. J., Maas, C. J. M., & Brinkhuis, M. J. S. (2010). The effect of estimation method and sample size in multilevel structural equation modeling. Statistica Neerlandica, 64(2), 157–170. in Google Scholar

Lavigne, G. L., & Vallerand, R. J. (2010). The dynamic processes of influence between contextual and situational motivation: A test of the hierarchical model in a science education setting. Journal of Applied Social Psychology, 40(9), 2343–2359. in Google Scholar

Little, T. D. (2013). Longitudinal structural equation modeling. Guilford press.Search in Google Scholar

Muthen, B. (1991). Multilevel Factor Analysis of Class and Student Achievement Components Author (s): Bengt O. Muthén Source : Journal of Educational Measurement, Vol. 28, No. 4 (Winter, 1991), pp. 338-354 Published by : National Council on Measurement in Educati. Journal of Education Measurement, 28(4), 338–354.Search in Google Scholar

Preacher, K. J., Zyphur, M. J., & Zhang, Z. (2010). A general multilevel SEM framework for assessing multilevel mediation. Psychological Methods, 15(3), 209–233. in Google Scholar

Rabe-Hesketh, S., & Skrondal, A. (2012). Multilevel and longitudinal modeling using Stata. 3rd. College Station, Tex: Stata Press Publication.Search in Google Scholar

Ratelle, C. F., Guay, F., Vallerand, R. J., Larose, S., & Senécal, C. (2007). Autonomous, Controlled, and Amotivated Types of Academic Motivation: A Person-Oriented Analysis. Journal of Educational Psychology, 99(4), 734–746. in Google Scholar

Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (Vol. 1). sage.Search in Google Scholar

Schmidt, J. A., Rosenberg, J. M., & Beymer, P. N. (2018). A person-in-context approach to student engagement in science: Examining learning activities and choice. Journal of Research in Science Teaching, 55(1), 19–43. in Google Scholar

Schmidt, J., Kackar-Cam, H., Strati, A., & Shumow, L. (2015). The Role of Challenge in Students’ Engagement and Competence in High School Science Classrooms: Hispanic and Non-Hispanic Whites Compared. NCSSS Journal, 20(1), 20–26.Search in Google Scholar

Shumow, L., Schmidt, J. A., & Kackar, H. (2008). Reading In Class \& Out of Class: An Experience Sampling Method Study. Middle Grades Research Journal, 3(3).Search in Google Scholar

Shumow, L., Schmidt, J. A., & Zaleski, D. J. (2013). Multiple Perspectives on Student Learning, Engagement, and Motivation in High School Biology Labs Published by : University of North Carolina Press Stable URL : REFERENCES Linked references are available on JSTOR fo. 96(3), 232–252.Search in Google Scholar

Sinatra, G. M., Heddy, B. C., & Lombardi, D. (2015). The Challenges of Defining and Measuring Student Engagement in Science. Educational Psychologist, 50(1). in Google Scholar

Steyer, R., Ferring, D., & Schmitt, M. J. (1992). States and traits in psychological assessment. European Journal of Psychological Assessment.Search in Google Scholar

Steyer, R., Mayer, A., Geiser, C., & Cole, D. A. (2015). A theory of states and traits—Revised. Annual Review of Clinical Psychology, 11, 71–98.10.1146/annurev-clinpsy-032813-153719Search in Google Scholar

Walker, C. O., Greene, B. A., & Mansell, R. A. (2006). Identification with academics, intrinsic/extrinsic motivation, and self-efficacy as predictors of cognitive engagement. Learning and Individual Differences, 16(1), 1–12. in Google Scholar

Wang, M. Te, & Eccles, J. S. (2012). Social Support Matters: Longitudinal Effects of Social Support on Three Dimensions of School Engagement From Middle to High School. Child Development, 83(3), 877–895. in Google Scholar

Wang, M. Te, Fredricks, J. A., Ye, F., Hofkens, T. L., & Linn, J. S. (2016). The Math and Science Engagement Scales: Scale development, Validation, And psychometric properties. Learning and Instruction, 43, 16–26. in Google Scholar

Xie, K., Heddy, B. C., & Greene, B. A. (2019). Affordances of using mobile technology to support experience-sampling method in examining college students’ engagement. Computers & Education, 128, 183-198.10.1016/j.compedu.2018.09.020Search in Google Scholar

Xie, K., Heddy, B. C., & Vongkulluksn, V. W. (2019). Examining engagement in context using experience-sampling method with mobile technology. Contemporary Educational Psychology, 59, 101788.10.1016/j.cedpsych.2019.101788Search in Google Scholar

Xie, K., Hensley, L. C., Law, V., & Sun, Z. (2019). Self-regulation as a function of perceived leadership and cohesion in small group online collaborative learning. British Journal of Educational Technology, 50(1), 456-468.10.1111/bjet.12594Search in Google Scholar

Xie, K., Vongkulluksn, V. W., Lu, L., & Cheng, S. L. (2020). A person-centered approach to examining high-school students’ motivation, engagement and academic performance. Contemporary Educational Psychology, 62, 101877.10.1016/j.cedpsych.2020.101877Search in Google Scholar

Zhu, X., Chen, A., Ennis, C., Sun, H., Hopple, C., Bonello, M., Bae, M., & Kim, S. (2009). Situational interest, cognitive engagement, and achievement in physical education. Contemporary Educational Psychology, 34(3), 221–229. in Google Scholar

Zirkel, S., Garcia, J. A., & Murphy, M. C. (2015). Experience-Sampling Research Methods and Their Potential for Education Research. Educational Researcher, 44(1), 7–16. in Google Scholar

Received: 2021-07-12
Accepted: 2022-04-28
Published Online: 2022-07-05

© 2022 Vanessa W. Vongkulluksn et al., published by De Gruyter

This work is licensed under the Creative Commons Attribution 4.0 International License.

Downloaded on 6.2.2023 from
Scroll Up Arrow