Disentangling objective characteristics of learning situations from subjective perceptions thereof, using an experience sampling method design
Main Article Content
Abstract
This article proposes a study design developed to disentangle the objective characteristics of a learning situation from individuals’ subjective perceptions of that situation. The term ‘objective characteristics’ refers to the agreement across students, whereas ‘subjective perceptions’ refers to inter-individual heterogeneity. We describe a novel strategy for assessing and disentangling objective situation characteristics and subjective perceptions thereof, propose methods for analyzing the resulting data, and illustrate the procedure with an example of a first study using this design to examine situational interest in 155 university students. Situational interest was assessed nine times per weekly lecture with three measurement time points per person and a rotated multi-group schedule. Assessments took place over the course of an entire semester of ten weeks.
One of the advantages of the proposed design is that ‘objective’ group agreements can be disentangled from subjective deviations from the group’s average at each of the nine measurement time points per weekly lecture. Furthermore, the proposed design makes it possible to study the development of both subjective and objective parameters across the time span of one weekly lecture and an entire semester, while the burden for each person is kept relatively low with three beeps per lecture.
Article Details
FLR adopts the Attribution-NonCommercial-NoDerivs Creative Common License (BY-NC-ND). That is, Copyright for articles published in this journal is retained by the authors with, however, first publication rights granted to the journal. By virtue of their appearance in this open access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.
References
Altrabsheh, N., Gaber, M. M., & Cocea, M. (2013). SA-E: Sentiment analysis for education. In: R. Neves-Silva, J. Watada, G. Philipps-Wren, L. C. Jain, & R. J. Howlett (Eds.), Intelligent Decision Technologies (pp. 353 - 362), Amsterdam: IOS Press. doi: 10.3233/978-1-61499-264-6-353
Asparouhov, T., & Muthén, B. (2012). Multiple group multilevel analysis. Mplus Web Notes: No. 16. Retrieved March 5, 2020 from https://www.statmodel.com/examples/webnotes/webnote16.pdf
Asparouhov, T. & Muthén, B. (2019). Comparison of models for the analysis of intensive longitudinal data, Structural Equation Modeling: A Multidisciplinary Journal, 00: 1–23. https://doi.org/10.1080/10705511.2019.1626733
Banister, S., Reinhart, R., & Ross, C. (2014). Using digital resources to support personalized learning experiences in K-12 classrooms: The evolution of mobile devices as innovations in schools in Northwest Ohio. In M. Searson & M. Ochoa (Eds.), Proceedings of Society for Information Technology & Teacher Education International Conference 2014 (pp. 2715-2721). Chesapeake, VA: Association for the Advancement of Computing in Education. Retrieved March 5, 2020 from https://www.learntechlib.org/primary/p/131202/.
Bastiaansen, J. A., Kunkels, Y. K., Blaauw, F., Boker, S. M., Ceulemans, E., Chen, M., … Bringmann, L. F. (2019, March 21). Time to get personal? The impact of researchers’ choices on the selection of treatment targets using the experience sampling methodology. Preprint retrieved on August 24, 2019 from https://doi.org/10.31234/osf.io/c8vp7
Bingham, A. J., Pane, J. F., Steiner, E. D., & Hamilton, L. S. (2018). Ahead of the curve: Implementation challenges in personalised learning school models. Educational Policy, 32(3), 454 – 489. https://doi.org/10.1177/0895904816637688
Barrett, L. F. (2018). How emotions are made. The secret life of the brain. Mariner Books: New York.
Battle, A., & Wigfield, A. (2003). College women’s value orientations toward family, career, and graduate school. Journal of Vocational Behavior, 62, 56–75. https://doi.org/10.1016/S0001-8791(02)00037-4
Beretvas, S. N. (2010). Cross-classified and multiple membership models. In J. J. Hox & J. K. Roberts (Eds.), Handbook of advanced multilevel analysis (pp. 313–334). New York, NY: Routledge.
Bieg, M., Goetz, T., Wolter, I., & Hall, N. C. (2015). Gender stereotype endorsement differentially predicts girls' and boys' trait-state discrepancy in math anxiety. Frontiers in Psychology, 6, 1404. https://doi.org/10.3389/fpsyg.2015.01404
Carroll, E. A., Czerwinski, M., Roseway, A., Kapoor, A., Johns, P., Rowan, K., & Schraefel, M. C. (2013). Food and mood: Just-in-time support for emotional eating. 2013 Humaine Association Conference of Affective Computing and Intelligent Interaction. Geneva, Switzerland.
Chauliac, M; Catrysse, L. ; Gijbels, D. & Donche V. (2020). It is all in the surv-eye: can eye tracking data shed light on the internal consistency in self-report questionnaires on cognitive processing strategies? Frontline Learning Research. 8 (3), 26 – 39. https://doi.org/10.14786/flr.v8i3.489
Chung, H., Kim, J., Park, R., & Jean, H. (2018). The impact of sample size in cross-classified multiple membership multilevel models. Journal of Modern Applied Statistical Methods, 17 (1), Article 26. https://doi.org/10.22237/jmasm/1542209860
Cole, J. S., Bergin, D. A., & Whittaker, T. A. (2008). Predicting student achievement for low stakes tests with effort and task value. Contemporary Educational Psychology, 33, 609–624. https://doi.org/10.1016/j.cedpsych.2007.10.002
Corno, L. (2008). On teaching adaptively. Educational Psychologist, 43(3), 161–173. https://doi.org/10.1080/00461520802178466
Creswell, J. W. & Guetterman, T. C. (2019). Educational research: Planning, conducting, and evaluating quantitative and qualitative research, 6th edition, Pearson.
Dietrich, J., Viljaranta, J., Moeller, J., & Kracke, B. (2017). Situational expectancies and task values: Associations with students' effort. Learning and Instruction, 47, 53–64. https://doi.org/10.1016/j.learninstruc.2016.10.009
Dietrich, J., Moeller, J., Guo, J., Viljaranta, J., & Kracke, B. (2019a). In-the-moment profiles of expectancies, task values, and costs. Frontiers in Psychology, 10:1662. https://doi.org/10.3389/fpsyg.2019.01662
Durik, A. M. & Jenkins J. S. (2020). Variability in Certainty of Self-Reported Interest: Implications for Theory and Research. Frontline Learning Research. 8 (3) 85-103. https://doi.org/10.14786/flr.v8i3.491
Douglas, H., (2011). Facts, values, and objectivity. In: I. Jarvie & J. Zamora Bonilla (eds.), The SAGE Handbook of Philosophy of Social Science, 513–529, London: SAGE Publications.
Durik, A. M., Vida, M., & Eccles, J. S. (2006). Task values and ability beliefs as predictors of high school literacy choices: A developmental analysis. Journal of Educational Psychology, 98, 382–393. https://doi.org/10.1037/0022-0663.98.2.382
Eccles, J. S., Adler, T. F., Futterman, R., Goff, S. B., Kaczala, C. M., Meece, J. L., & Midgley, C. (1983). Expectancies, values, and academic behaviors. In J. T. Spence (Ed.), Achievement and achievement motives (pp.74–146). San Francisco, CA: Freeman.
Eccles, J. S., & Wigfield, A. (2002). Motivational beliefs, values, and goals. Annual Review of Psychology, 53, 109–132. https://doi.org/10.1146/annurev.psych.53.100901.135153
Efron, B. and Tibshirani, R. (1993) An Introduction to the Bootstrap. Chapman and Hall, New York, London.
Eisner, E. (1992). Objectivity in educational research. Curriculum Inquiry, 22(1), 9-15. https://doi.org/10.1080/03626784.1992.11075389
Enders, C. K. (2010). Applied missing data analysis. New York, NY: The Guilford Press.
Fahrenberg, J. (1996). Ambulatory assessment: Issues and perspectives. In: Fahrenberg, J. & Myrtek, M. (Eds.). (1996). Ambulatory Assessment: Computer-assisted Psychological and Psychophysiological Methods in Monitoring and Field Studies (pp. 3 – 20). Seattle, WA: Hogrefe and Huber. University of Freiburg i. Br., Germany
Fink, B. (1991). Interest development as structural change in person-object relationships. In: Oppenheimer L., Valsiner J. (eds) The Origins of Action. Springer, New York, NY. https://doi.org/10.1007
Firstbeat (2012). Heart beat based recovery analysis for athletic training. Firstbeat Whitepapers. Retrieved from: http://www.firstbeat.fi/physiology/white-papers
Fisher W. P. Jr. (2000). Objectivity in psychosocial measurement: what, why, how. Journal of Outcome Measurement, 4(2), 527-563.
Fryer, L. K. & Nakao K. (2020). The Future of Survey Self-report: An experiment contrasting Likert, VAS, Slide, and Swipe touch interfaces. Frontline Learning Research, 8 (3),10-25. https://doi.org/10.14786/flr.v8i3.501
Glanzberg, M. (2018). Truth. In: Edward N. Zalta (ed.), The Stanford Encyclopedia of Philosophy (Fall 2018 Edition), Retrieved from https://plato.stanford.edu/archives/fall2018/entries/truth/
Göllner, R., Wagner, W., Eccles, J. S., & Trautwein, U. (2018). Students’ idiosyncratic perceptions of teaching quality in mathematics: A result of rater tendency alone or an expression of dyadic effects between students and teachers? Journal of Educational Psychology, 110(5), 709–725. https://doi.org/10.1037/edu0000236
Goetz, T., Bieg, M., Lüdtke, O., Pekrun, R., & Hall, N. C. (2013). Do girls really experience more anxiety in mathematics? Psychological Science, 24(10), 2079-2087. https://doi.org/10.1177/0956797613486989
Green, A. S., Rafaeli, E., Bolger, N., Shrout, P. E., & Reis, H. T. (2006). Paper or plastic? Data equivalence in paper and electronic diaries. Psychological Methods, 11, 87–105. https://doi.org/10.1037/1082-989X.11.1.87
Hartigan, J. A., & Hartigan, P. M. (1985) The dip test of unimodality. Annals of Statistics, 13, 70–84.
Hektner, J. M., Schmidt, J. A., & Csikszentmihalyi, M. (2007). Experience sampling method. Measuring the quality of everyday life. Thousand Oaks, CA, US: Sage Publications.
Hidi, S., & Renninger, K. A. (2006). The four-phase model of interest development. Educational Psychologist, 41, 111-127. https://doi.org/10.1207/s15326985ep4102_4
Ketonen, E., Dietrich, J., Moeller, J., Salmela-Aro, K., & Lonka, K. (2018). The influence of autonomous and controlled daily goals on positive and negative emotional states: An experience sampling approach. Learning and Instruction, 53, 10-20. https://doi.org/10.1016/j.learninstruc.2017.07.003
Krapp, A. (1998). Entwicklung und Förderung von Interessen im Unterricht [Development and promotion of interest in instruction]. Psychologie in Erziehung und Unterricht, 45, 186-203.
Krapp, A. (2002). Structural and dynamic aspects of interest development: theoretical considerations from an ontogenetic perspective. Learning and Instruction, 12(4), 383-409. https://doi.org/10.1016/S0959-4752(01)00011-1
Krapp, A., & Fink, B. (1992). The development and function of interests during the critical transition from home to preschool. In K. A. Renninger, S. Hidi, & A. Krapp (Eds.), The role of interest in learning and development (pp. 397–429). Hillsdale, NJ: Lawrence Erlbaum Associates.
Krapp, A., Hidi, S., & Renninger, K. A. (1992). Interest, learning and development. In K. A. Renninger, S. Hidi, & A. Krapp (Eds.), The role of interest in learning and development (pp. 3–25). Hillsdale, NJ: Lawrence Erlbaum Associates.
Krothapalli, K. S. & Koolagudi, S. G. (2013). Emotion recognition using speech features. London: Springer
Lüdtke, O., Robitzsch, A., Trautwein, U., Kunter. M. (2009). Assessing the impact of learning environments: How to use student ratings of classroom or school characteristics in multilevel modeling. Contemporary Educational Psychology 34, 120–131. https://doi.org/10.1016/j.cedpsych.2008.12.001
Maechler, M. (2016). Package ‘diptest’. Hartigan's dip test statistic for unimodality – Corrected. R package. Retrieved March 5, 2020 from https://cran.r-project.org/web/packages/diptest/diptest.pdf
Moeller, J., Dietrich, J., Viljaranta, J., & Kracke, B. (2019). Data, R and Mplus codes for disentangling objective characteristics of learning situations from subjective perceptions thereof, using an experience sampling method design. Rerieved from https://osf.io/yszvm/. https://doi.org/10.17605/OSF.IO/YSZVM
Moeller, J., Ivcevic, Z., White, A., & Brackett, M. A. (2018). Mixed emotions: network analyses of intra-individual co-occurrences within and across situations. Emotion,18(8), 1106-1121. https://doi.org/10.1037/emo0000419
Popper, K. R. (1934 [2002]), Logik der Forschung [The Logic of Scientific Discovery], Berlin: Akademie Verlag.
Prenzel, M., Krapp, A. & Schiefele, H. (1986). Grundzüge einer pädagogischen Interessentheorie [Outline of an educational interest theory]. Zeitschrift für Pädagogik, 32(2), 163-173.
Reitzle, M. & Dietrich, J. (2019). From between-person statstics to within-person dynamics. Diskurs Kindheits- und Jugendforschung, 3-2019, 319-339. https://doi.org/10.3224/diskurs.v14i3.06
Schmitz, B. & Perels, F. (2011). Self-monitoring of self-regulation during math homework behaviour using standardized diaries. Metacognition & Learning, 6, 255-273. https://doi.org/10.1007/s11409-011-9076-6
Schönbrodt, F. D. & Perugini, M. (2013). At what sampe size do correlations stabilize? Journal of Research on Personality, 47, 609-612. https://doi.org/10.1016/j.jrp.2013.05.009
Shiffman, S., Stone, A. A., & Hufford, M. R. (2008). Ecological momentary assessment. Annual Review of Clinical Psychology, 4, 1-32. https://doi.org/10.1146/annurev.clinpsy.3.022806.091415
Silge, J. & Robinson, D. (2017). Text mining with R: A tidy approach. Sebastopol, CA: O’Reilly
Takarangi, M. K. T., Garry, M., & Loftus, E. F. (2006). Dear diary, is plastic better than paper? I can’t remember: Comment on Green, Rafaeli, Bolger, Shrout, and Reis (2006). Psychological Methods, 11, 119 –122. https://doi.org/10.1037/1082-989X.11.1.119
Tibshirani, R. & Leisch, F. (2019). bootstrap: Functions for the Book "An Introduction to the Bootstrap. R package. https://cran.r-project.org/web/packages/bootstrap/index.html
Wickham, H., Chang, W., Henry, L., Pedersen, T. L., Takahashi, K., Wilke, C., & Woo, K. (n.d.). Jittered points. Retrieved from: https://ggplot2.tidyverse.org/reference/geom_jitter.html
Wickham, H. (2016). Ggplot2: Elegant graphics for data analysis. Springer-Verlag New York.