Addressing boundary conditions of cognitive and motivational effects of gamified learning
Main Article Content
Abstract
There is a growing interest in developing gamified learning solutions to address educational challenges. However, learning is highly influenced by the conditions in which it takes place (e.g., does gamified learning in a laboratory setting replicate the outcomes of gamified learning online at home?). Hence, it is crucial to understand the boundary conditions of different learning contexts to effectively implement gamified interventions that provide optimal learner support. This work contributes to such an understanding by assessing how general contextual aspects of three studies on gamified learning influence cognitive learning and motivational outcomes. Therefore, we re-examined the results of two earlier published online studies (Study 1: n=285; Study 2: n=61) and compared the results to a recently conducted laboratory study (Study 3: n=121), all of which employed the same associative learning task. Comparing results through a Bayesian lens, we find that motivational outcomes induced by gamification differ substantially between contexts. In contrast, cognitive learning outcomes seem comparatively robust across different contextual factors, with some indication of subtle influences in agreement with cognitive learning theories. Implications are discussed for future empirical research on learning, highlighting how a better understanding of boundary conditions of gamified learning interventions could open perspectives for context-aware educational interventions.
Article Details
FLR adopts the Attribution-NonCommercial-NoDerivs Creative Common License (BY-NC-ND). That is, Copyright for articles published in this journal is retained by the authors with, however, first publication rights granted to the journal. By virtue of their appearance in this open access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.
References
Alotaibi, M. S. (2024). Game-based learning in early childhood education: A systematic review and meta-analysis. Frontiers in Psychology, 15, 1307881. https://doi.org/10.3389/fpsyg.2024.1307881
Alyahyan, E., & Düştegör, D. (2020). Predicting academic success in higher education: Literature review and best practices. International Journal of Educational Technology in Higher Education, 17(1), 3. https://doi.org/10.1186/s41239-020-0177-7
Arztmann, M., Hornstra, L., Jeuring, J., & Kester, L. (2023). Effects of games in STEM education: A meta-analysis on the moderating role of student background characteristics. Studies in Science Education, 59(1), 109–145. https://doi.org/10.1080/03057267.2022.2057732
Azevedo, R., & Wiedbusch, M. (2023). Theories of metacognition and pedagogy applied to AIED systems. In B. Du Boulay, A. Mitrovic, & K. Yacef (Eds.), Handbook of Artificial Intelligence in Education (pp. 45–67). Edward Elgar Publishing. https://doi.org/10.4337/9781800375413.00013
Bai, S., Hew, K. F., & Huang, B. (2020). Does gamification improve student learning outcome? Evidence from a meta-analysis and synthesis of qualitative data in educational contexts. Educational Research Review, 30, 100322. https://doi.org/10.1016/j.edurev.2020.100322
Bandura, A. (1997). Self-efficacy: The exercise of control. W. H. Freeman.
Barrett, L. F., Adolphs, R., Marsella, S., Martinez, A. M., & Pollak, S. D. (2019). Emotional Expressions Reconsidered: Challenges to Inferring Emotion From Human Facial Movements. Psychological Science in the Public Interest, 20(1), 1–68. https://doi.org/10.1177/1529100619832930
Barz, N., Benick, M., Dörrenbächer-Ulrich, L., & Perels, F. (2023). The Effect of Digital Game-Based Learning Interventions on Cognitive, Metacognitive, and Affective-Motivational Learning Outcomes in School: A Meta-Analysis. Review of Educational Research, 003465432311677. https://doi.org/10.3102/00346543231167795
Baten, E., Vansteenkiste, M., De Muynck, G.-J., De Poortere, E., & Desoete, A. (2020). How can the blow of math difficulty on elementary school children’s motivational, cognitive, and affective experiences be dampened? The critical role of autonomy-supportive instructions. Journal of Educational Psychology, 112(8), 1490–1505. https://doi.org/10.1037/edu0000444
Bernecker, K., & Ninaus, M. (2021). No Pain, no Gain? Investigating motivational mechanisms of game elements in cognitive tasks. Computers in Human Behavior, 114, 106542. https://doi.org/10.1016/j.chb.2020.106542
Blikstein, P. (2013). Multimodal learning analytics. Proceedings of the Third International Conference on Learning Analytics and Knowledge, 102–106. https://doi.org/10.1145/2460296.2460316
Booth, B. M., Bosch, N., & D’Mello, S. K. (2023). Engagement Detection and Its Applications in Learning: A Tutorial and Selective Review. Proceedings of the IEEE, 111(10), 1398–1422. https://doi.org/10.1109/JPROC.2023.3309560
Borenstein, M., Hedges, L. V., Higgins, J. P. T., & Rothstein, H. R. (2009). Introduction to Meta‐Analysis (1st ed.). Wiley. https://doi.org/10.1002/9780470743386
Chi, M. T. H., & Wylie, R. (2014). The ICAP Framework: Linking Cognitive Engagement to Active Learning Outcomes. Educational Psychologist, 49(4), 219–243. https://doi.org/10.1080/00461520.2014.965823
Cloude, E. B., Dever, D. A., Hahs-Vaughn, D. L., Emerson, A. J., Azevedo, R., & Lester, J. (2022). Affective Dynamics and Cognition During Game-Based Learning. IEEE Transactions on Affective Computing, 13(4), 1705–1717. https://doi.org/10.1109/TAFFC.2022.3210755
Clow, D. (2012). The learning analytics cycle: Closing the loop effectively. Proceedings of the 2nd International Conference on Learning Analytics and Knowledge, 134–138. https://doi.org/10.1145/2330601.2330636
Conijn, R., Snijders, C., Kleingeld, A., & Matzat, U. (2017). Predicting Student Performance from LMS Data: A Comparison of 17 Blended Courses Using Moodle LMS. IEEE Transactions on Learning Technologies, 10(1), 17–29. https://doi.org/10.1109/TLT.2016.2616312
Conte, P. D. (2019). A gender study on the effects of the “high five game” on the math learning performance of children. International Journal of Scientific and Technology Research, 8(12), 2063–2066.
Deterding, S., Dixon, D., Khaled, R., & Nacke, L. (2011). From game design elements to gamefulness: Defining “gamification.” Proceedings of the 15th International Academic MindTrek Conference: Envisioning Future Media Environments, 9–15. https://doi.org/10.1145/2181037.2181040
D’Mello, S. K., & Booth, B. M. (2023). Affect Detection From Wearables in the “Real” Wild: Fact, Fantasy, or Somewhere In between? IEEE Intelligent Systems, 38(1), 76–84. https://doi.org/10.1109/MIS.2022.3221854
D’Mello, S. K., Kappas, A., & Gratch, J. (2018). The Affective Computing Approach to Affect Measurement. Emotion Review, 10(2), 174–183. https://doi.org/10.1177/1754073917696583
Domjan, M. (2010). The Principles of Learning and Behavior. Wadsworth Cengage Learning.
Eccles, J. S., & Wigfield, A. (2020). From expectancy-value theory to situated expectancy-value theory: A developmental, social cognitive, and sociocultural perspective on motivation. Contemporary Educational Psychology, 61, 101859. https://doi.org/10.1016/j.cedpsych.2020.101859
Engeser, S., & Rheinberg, F. (2008). Flow, performance and moderators of challenge-skill balance. Motivation and Emotion, 32(3), 158–172. https://doi.org/10.1007/s11031-008-9102-4
Fadda, D., Pellegrini, M., Vivanet, G., & Zandonella Callegher, C. (2022). Effects of digital games on student motivation in mathematics: A meta‐analysis in K‐12. Journal of Computer Assisted Learning, 38(1), 304–325. https://doi.org/10.1111/jcal.12618
Fiedler, K., & Beier, S. (2014). Affect and Cognitive Processes in Educational Contexts. In R. Pekrun & L. Linnenbrank-Garcia (Eds.), International Handbook of Emotions in Education (1st ed.). Routledge.
Fryer, L. K., & Dinsmore, D. L. (2020). The Promise and Pitfalls of Self-report. Frontline Learning Research, 8(3), 1–9. https://doi.org/10.14786/flr.v8i3.623
Fulmer, S. M., & Frijters, J. C. (2009). A Review of Self-Report and Alternative Approaches in the Measurement of Student Motivation. Educational Psychology Review, 21(3), 219–246. https://doi.org/10.1007/s10648-009-9107-x
Gaspard, H., Jiang, Y., Piesch, H., Nagengast, B., Jia, N., Lee, J., & Bong, M. (2020). Assessing students’ values and costs in three countries: Gender and age differences within countries and structural differences across countries. Learning and Individual Differences, 79, 101836. https://doi.org/10.1016/j.lindif.2020.101836
Greipl, S., Klein, E., Lindstedt, A., Kiili, K., Moeller, K., Karnath, H.-O., Bahnmueller, J., Bloechle, J., & Ninaus, M. (2021). When the brain comes into play: Neurofunctional correlates of emotions and reward in game-based learning. Computers in Human Behavior, 125, 106946. https://doi.org/10.1016/j.chb.2021.106946
Greipl, S., Moeller, K., & Ninaus, M. (2020). Potential and limits of game-based learning. International Journal of Technology Enhanced Learning, 12(4), 363. https://doi.org/10.1504/IJTEL.2020.110047
Guay, F., Vallerand, R. J., & Blanchard, C. (2000). On the Assessment of Situational Intrinsic and Extrinsic Motivation: The Situational Motivation Scale (SIMS). Motivation and Emotion, 24(3), 175–213. https://doi.org/10.1023/A:1005614228250
Harmon-Jones, E., & Mills, J. (2019). An introduction to cognitive dissonance theory and an overview of current perspectives on the theory. In E. Harmon-Jones (Ed.), Cognitive dissonance: Reexamining a pivotal theory in psychology (2nd ed.). (pp. 3–24). American Psychological Association. https://doi.org/10.1037/0000135-001
Hatano, A., Ogulmus, C., Shigemasu, H., & Murayama, K. (2022). Thinking about thinking: People underestimate how enjoyable and engaging just waiting is. Journal of Experimental Psychology: General, 151(12), 3213–3229. https://doi.org/10.1037/xge0001255
Heckhausen, J., & Heckhausen, H. (2018). Motivation and Action: Introduction and Overview. In J. Heckhausen & H. Heckhausen (Eds.), Motivation and Action (pp. 1–14). Springer International Publishing. https://doi.org/10.1007/978-3-319-65094-4_1
Hoerger, M. (2010). Participant Dropout as a Function of Survey Length in Internet-Mediated University Studies: Implications for Study Design and Voluntary Participation in Psychological Research. Cyberpsychology, Behavior, and Social Networking, 13(6), 697–700. https://doi.org/10.1089/cyber.2009.0445
Hoerger, M., & Currell, C. (2012). Ethical issues in Internet research. In S. J. Knapp, M. C. Gottlieb, M. M. Handelsman, & L. D. VandeCreek (Eds.), APA handbook of ethics in psychology, Vol 2: Practice, teaching, and research (pp. 385–400). American Psychological Association.
Hu, Y., Gallagher, T., Wouters, P., Van Der Schaaf, M., & Kester, L. (2022). Game‐based learning has good chemistry with chemistry education: A three‐level meta‐analysis. Journal of Research in Science Teaching, 59(9), 1499–1543. https://doi.org/10.1002/tea.21765
Huang, R., Ritzhaupt, A. D., Sommer, M., Zhu, J., Stephen, A., Valle, N., Hampton, J., & Li, J. (2020). The impact of gamification in educational settings on student learning outcomes: A meta-analysis. Educational Technology Research and Development, 68(4), 1875–1901. https://doi.org/10.1007/s11423-020-09807-z
Huber, S. E., Cortez, R., Kiili, K., Lindstedt, A., & Ninaus, M. (2023). Game elements enhance engagement and mitigate attrition in online learning tasks. Computers in Human Behavior, 149, 107948. https://doi.org/10.1016/j.chb.2023.107948
Huber, S. E., Edlinger, M., Lindstedt, A., Kiili, K., & Ninaus, M. (2024). Game elements improve affect and motivation in a learning task. International Journal of Serious Games, 11(4), 103–126. https://doi.org/10.17083/ijsg.v11i4.769
JASP Team. (2024). JASP (Version 0.19.1.0) [Computer software].
Koskinen, A., McMullen, J., Ninaus, M., & Kiili, K. (2023). Does the emotional design of scaffolds enhance learning and motivational outcomes in game‐based learning? Journal of Computer Assisted Learning, 39(1), 77–93. https://doi.org/10.1111/jcal.12728
Krauspe, J., Ebersbach, M., Ludwig, A., & Scharf, F. (2025). Do worked examples boost the spacing effect on lasting learning? Learning and Instruction, 97, 102103. https://doi.org/10.1016/j.learninstruc.2025.102103
Kuratomi, K., Johnsen, L., Kitagami, S., Hatano, A., & Murayama, K. (2023). People underestimate their capability to motivate themselves without performance-based extrinsic incentives. Motivation and Emotion, 47(4), 509–523. https://doi.org/10.1007/s11031-022-09996-5
Lampropoulos, G. & Kinshuk. (2024). Virtual reality and gamification in education: A systematic review. Educational Technology Research and Development, 72(3), 1691–1785. https://doi.org/10.1007/s11423-024-10351-3
Laugwitz, B., Held, T., & Schrepp, M. (2008). Construction and Evaluation of a User Experience Questionnaire. In A. Holzinger (Ed.), HCI and Usability for Education and Work (Vol. 5298, pp. 63–76). Springer Berlin Heidelberg. https://doi.org/10.1007/978-3-540-89350-9_6
Lewin, K. (1936). Principles of topological psychology. McGraw-Hill.
Lewin, K. (1946). Behavior and development as a function of the total situation. In L. Carmichael (Ed.), Manual of child psychology. (pp. 791–844). John Wiley & Sons Inc. https://doi.org/10.1037/10756-016
Li, L., Hew, K. F., & Du, J. (2024). Gamification enhances student intrinsic motivation, perceptions of autonomy and relatedness, but minimal impact on competency: A meta-analysis and systematic review. Educational Technology Research and Development. https://doi.org/10.1007/s11423-023-10337-7
Long, P. D., & Siemens, G. (2011). Penetrating the Fog: Analytics in Learning and Education. EDUCAUSE Review. https://er.educause.edu/articles/2011/9/penetrating-the-fog-analytics-in-learning-and-education
Mavridis, A., Katmada, A., & Tsiatsos, T. (2017). Impact of online flexible games on students’ attitude towards mathematics. Educational Technology Research and Development, 65(6), 1451–1470. https://doi.org/10.1007/s11423-017-9522-5
Mayer, R. E. (2011). Applying the Science of Learning. Pearson.
Mayer, R. E. (2014). Computer Games for Learning: An Evidence-Based Approach. MIT Press.
Mayer, R. E. (2019). Computer Games in Education. Annual Review of Psychology, 70(1), 531–549. https://doi.org/10.1146/annurev-psych-010418-102744
Mayer, R. E. (2020). Cognitive foundations of game-based learning. In J. L. Plass, R. E. Mayer, & B. D. Homer (Eds.), Handbook of game-based learning (pp. 83–110). MIT Press.
Mayer, R. E. (2024). The Past, Present, and Future of the Cognitive Theory of Multimedia Learning. Educational Psychology Review, 36(1), 8. https://doi.org/10.1007/s10648-023-09842-1
Molenaar, I., Mooij, S. D., Azevedo, R., Bannert, M., Järvelä, S., & Gašević, D. (2023). Measuring self-regulated learning and the role of AI: Five years of research using multimodal multichannel data. Computers in Human Behavior, 139, 107540. https://doi.org/10.1016/j.chb.2022.107540
Neath, I., & Suprenant, A. M. (2003). Human Memory. Wadsworth/Thomson Learning.
Ninaus, M., Greipl, S., Kiili, K., Lindstedt, A., Huber, S., Klein, E., Karnath, H.-O., & Moeller, K. (2019). Increased emotional engagement in game-based learning – A machine learning approach on facial emotion detection data. Computers & Education, 142, 103641. https://doi.org/10.1016/j.compedu.2019.103641
Ninaus, M., & Sailer, M. (2022). Closing the loop – The human role in artificial intelligence for education. Frontiers in Psychology, 13, 956798. https://doi.org/10.3389/fpsyg.2022.956798
Palha, S., & Jukić Matić, L. (2025). What do teachers anticipate from education in game-based pedagogy? Technology, Pedagogy and Education, 1–14. https://doi.org/10.1080/1475939X.2025.2454453
Plass, J. L., Homer, B. D., & Kinzer, C. K. (2015). Foundations of Game-Based Learning. Educational Psychologist, 50(4), 258–283. https://doi.org/10.1080/00461520.2015.1122533
Plass, J. L., Homer, B. D., Mayer, R. E., & Kinzer, C. K. (2020). Theoretical foundations of game-based and playful learning. In J. L. Plass, R. E. Mayer, & B. D. Homer (Eds.), Handbook of game-based learning (pp. 3–24). MIT Press.
Plass, J. L., & Kaplan, U. (2016). Emotional Design in Digital Media for Learning. In Emotions, Technology, Design, and Learning (pp. 131–161). Elsevier. https://doi.org/10.1016/B978-0-12-801856-9.00007-4
Prenkaj, B., Velardi, P., Stilo, G., Distante, D., & Faralli, S. (2021). A Survey of Machine Learning Approaches for Student Dropout Prediction in Online Courses. ACM Computing Surveys, 53(3), 1–34. https://doi.org/10.1145/3388792
Ritzhaupt, A. D., Huang, R., Sommer, M., Zhu, J., Stephen, A., Valle, N., Hampton, J., & Li, J. (2021). A meta-analysis on the influence of gamification in formal educational settings on affective and behavioral outcomes. Educational Technology Research and Development, 69(5), 2493–2522. https://doi.org/10.1007/s11423-021-10036-1
Rodrigues, L., Pereira, F., Toda, A., Palomino, P., Oliveira, W., Pessoa, M., Carvalho, L., Oliveira, D., Oliveira, E., Cristea, A., & Isotani, S. (2022). Are They Learning or Playing? Moderator Conditions of Gamification’s Success in Programming Classrooms. ACM Transactions on Computing Education, 22(3), 1–27. https://doi.org/10.1145/3485732
Ryan, R. M., & Deci, E. L. (2019). Brick by Brick: The Origins, Development, and Future of Self-Determination Theory. In Advances in Motivation Science (Vol. 6, pp. 111–156). Elsevier. https://doi.org/10.1016/bs.adms.2019.01.001
Ryan, R. M., & Rigby, C. S. (2020). Motivational foundations of game-based learning. In J. L. Plass, R. E. Mayer, & B. D. Homer (Eds.), Handbook of game-based learning (pp. 153–176). MIT Press.
Sailer, M., & Homner, L. (2020). The Gamification of Learning: A Meta-analysis. Educational Psychology Review, 32(1), 77–112. https://doi.org/10.1007/s10648-019-09498-w
Sailer, M., Ninaus, M., Huber, S. E., Bauer, E., & Greiff, S. (2024). The End is the Beginning is the End: The closed-loop learning analytics framework. Computers in Human Behavior, 158, 108305. https://doi.org/10.1016/j.chb.2024.108305
Sailer, M., & Sailer, M. (2021). Gamification of in‐class activities in flipped classroom lectures. British Journal of Educational Technology, 52(1), 75–90. https://doi.org/10.1111/bjet.12948
Sammut, R., Griscti, O., & Norman, I. J. (2021). Strategies to improve response rates to web surveys: A literature review. International Journal of Nursing Studies, 123, 104058. https://doi.org/10.1016/j.ijnurstu.2021.104058
Schlag, R., Sailer, M., Tolks, D., Ninaus, M., & Sailer, M. (2024). Effectiveness of gamification in education. In A. Gegenfurtner & I. Kollar (Eds.), Designing Effective Digital Learning Environments (pp. 143–159). Routledge.
Schrepp, M., Hinderks, A., & Thomaschewski, J. (2017). Die UX KPI - Wunsch und Wirklichkeit. https://doi.org/10.18420/MUC2017-UP-0100
Schwartz, R. N., & Plass, J. L. (2020). Types of engagement in learning with games. In J. L. Plass, R. E. Mayer, & B. D. Homer (Eds.), Handbook of game-based learning (pp. 53–80). MIT Press.
Seo, K., Dodson, S., Harandi, N. M., Roberson, N., Fels, S., & Roll, I. (2021). Active learning with online video: The impact of learning context on engagement. Computers & Education, 165, 104132. https://doi.org/10.1016/j.compedu.2021.104132
Sinatra, G. M., Heddy, B. C., & Lombardi, D. (2015). The Challenges of Defining and Measuring Student Engagement in Science. Educational Psychologist, 50(1), 1–13. https://doi.org/10.1080/00461520.2014.1002924
Skitka, L. J., & Sargis, E. G. (2006). The Internet as Psychological Laboratory. Annual Review of Psychology, 57(1), 529–555. https://doi.org/10.1146/annurev.psych.57.102904.190048
Smith, S. M., & Vela, E. (2001). Environmental context-dependent memory: A review and meta-analysis. Psychonomic Bulletin & Review, 8(2), 203–220. https://doi.org/10.3758/BF03196157
Sweller, J. (2011). Cognitive Load Theory. In Psychology of Learning and Motivation (Vol. 55, pp. 37–76). Elsevier. https://doi.org/10.1016/B978-0-12-387691-1.00002-8
Urhahne, D., & Wijnia, L. (2023). Theories of Motivation in Education: An Integrative Framework. Educational Psychology Review, 35(2), 45. https://doi.org/10.1007/s10648-023-09767-9
Wesenberg, L., Jansen, S., Krieglstein, F., Schneider, S., & Rey, G. D. (2025). The influence of seductive details in learning environments with low and high extrinsic motivation. Learning and Instruction, 96, 102054. https://doi.org/10.1016/j.learninstruc.2024.102054
Wilcox, R. R. (2022). Introduction to robust estimation and hypothesis testing (5th ed.). Academic Press.
Wilde, M., Bätz, K., Kovaleva, A., & Urhahne, D. (2009). Überprüfung einer Kurzskala intrinsicher Motivation (KIM). Zeitschrift Für Didaktik Der Naturwissenschaften, 15, 31–45.
Wong, Z. Y., & Liem, G. A. D. (2022). Student Engagement: Current State of the Construct, Conceptual Refinement, and Future Research Directions. Educational Psychology Review, 34(1), 107–138. https://doi.org/10.1007/s10648-021-09628-3
Wouters, P., Van Nimwegen, C., Van Oostendorp, H., & Van Der Spek, E. D. (2013). A meta-analysis of the cognitive and motivational effects of serious games. Journal of Educational Psychology, 105(2), 249–265. https://doi.org/10.1037/a0031311
Xu, Z., Olson, J., Pochinki, N., Zheng, Z., & Yu, R. (2024). Contexts Matter but How? Course-Level Correlates of Performance and Fairness Shift in Predictive Model Transfer. Proceedings of the 14th Learning Analytics and Knowledge Conference, 713–724. https://doi.org/10.1145/3636555.3636936
Zainuddin, Z., Chu, S. K. W., Shujahat, M., & Perera, C. J. (2020). The impact of gamification on learning and instruction: A systematic review of empirical evidence. Educational Research Review, 30, 100326. https://doi.org/10.1016/j.edurev.2020.100326
Zhang, L., Lei, Y., Pelton, T., Pelton, L. F., & Shang, J. (2024). An exploration of gendered differences in cognitive, motivational and emotional aspects of game‐based math learning. Journal of Computer Assisted Learning, 40(6), 2633–2649. https://doi.org/10.1111/jcal.12956
Zhang, Q., & Yu, Z. (2022). Meta-Analysis on Investigating and Comparing the Effects on Learning Achievement and Motivation for Gamification and Game-Based Learning. Education Research International, 2022, 1–19. https://doi.org/10.1155/2022/1519880