How teachers interpret displays of students’ gaze in reading comprehension assignments
Main Article Content
Abstract
Reading comprehension is a central skill in secondary education. To be able to provide adaptive instruction, teachers need to be able to accurately estimate students’ reading comprehension. However, they tend to experience difficulties doing so. Eye tracking can uncover these reading processes by visualizing what a student looked at, in what order, and for how long, in a gaze display. The question is, however, whether teachers could interpret such displays. We, therefore, examined how teachers interpret gaze displays and perceived their potential use in education to foster tailored support for reading comprehension. Sixty teachers in secondary education were presented with three static gaze displays of students performing a reading comprehension task. Teachers were asked to report how they interpreted these gaze displays and what they considered to be the promises and pitfalls of gaze displays for education. Teachers interpreted in particular reading strategies in the gaze displays quite well, and also interpreted the displays as reflecting other concepts, such as motivation and concentration. Results showed that teachers’ interpretations of the gaze displays were generally consistent across teachers and that teachers discriminated well between displays of different strategies. Teachers were generally positive about potential applications in educational practice. This study provides first insights into how teachers experience the utility of gaze displays as an innovative tool to support reading instruction, which is timely as rapid technological developments already enable eye tracking through webcams on regular laptops. Thus, using gaze displays in an educational setting seems to be an increasingly feasible scenario.
Article Details
FLR adopts the Attribution-NonCommercial-NoDerivs Creative Common License (BY-NC-ND). That is, Copyright for articles published in this journal is retained by the authors with, however, first publication rights granted to the journal. By virtue of their appearance in this open access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.
References
Afflerbach, P., Pearson, P. D., & Paris, S. G. (2008). Clarifying differences between reading skills and reading strategies. The Reading Teacher, 61(5), 364-373. https://doi.org/https://doi.org/10.1598/RT.61.5.1
Almosallam, E. A., & Ouertani, H. C. (2013). Learning analytics: Definitions, applications and related fields. Proceedings of the first international conference on advanced data and information engineering (DaEng-2013),
Andreassen, R., Jensen, M. S., & Bråten, I. (2017). Investigating self-regulated study strategies among postsecondary students with and without dyslexia: A diary method study. Reading and Writing, 30(9), 1891-1916. https://doi.org/10.1007/s11145-017-9758-9
Authors. (under review). reference blinded for review.
Bahle, B., Mills, M., & Dodd, M. D. (2017). Human classifier: Observers can deduce task solely from eye movements. Attention Perception & Psychophysics, 79(5), 1415-1425. https://doi.org/10.3758/s13414-017-1324-7
Benfatto, M. N., Öqvist Seimyr, G., Ygge, J., Pansell, T., Rydberg, A., & Jacobson, C. (2016). Screening for dyslexia using eye tracking during reading. Plos One, 11(12), e0165508. https://doi.org/10.1371/journal.pone.0165508
Chisari, L. B., Mockevičiūtė, A., Ruitenburg, S. K., van Vemde, L., Kok, E. M., & van Gog, T. (2020). Effects of prior knowledge and joint attention on learning from eye movement modelling examples. Journal of Computer Assisted Learning, 36(4), 569-579. https://doi.org/10.1111/jcal.12428
de Koning, B. B., & Jarodzka, H. (2017). Attention guidance strategies for supporting learning from dynamic visualizations. In R. Lowe & R. Ploetzner (Eds.), Learning from Dynamic Visualization: Innovations in Research and Application (pp. 255-278). Springer International Publishing. https://doi.org/10.1007/978-3-319-56204-9_11
Donovan, T., Manning, D. J., & Crawford, T. (2008). Performance changes in lung nodule detection following perceptual feedback of eye movements Proceedings of SPIE - The International Society for Optical Engineering in San Diego, CA, http://dx.doi.org/10.1117/12.768503
Duffy, G. G. (2002). The case for direct explanation of strategies. In C. C. Block & M. Pressley (Eds.), Comprehension instruction: Research-based best practices (pp. 28-41). Guilford.
Duke, N. K., & Pearson, P. D. (2009). Effective practices for developing reading comprehension. Journal of education, 189(1-2), 107-122. https://doi.org/10.1177/0022057409189001-208
Eder, T. F., Richter, J., Scheiter, K., Keutel, C., Castner, N., Kasneci, E., & Huettig, F. (2020). How to support dental students in reading radiographs: effects of a gaze‑based compare‑and‑contrast intervention. Advances in Health Sciences Education: Theory and Practice. https://doi.org/10.1007/s10459-020-09975-w
Emhardt, S. N., van Wermeskerken, M., Scheiter, K., & van Gog, T. (2020). Inferring task performance and confidence from displays of eye movements. Applied Cognitive Psychology, 34(6), 1430-1443. https://doi.org/https://doi.org/10.1002/acp.3721
Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data. MIT Press. http://books.google.nl/books?id=z4HQQgAACAAJ
Foulsham, T., & Lock, M. (2015). How the eyes tell lies: Social gaze during a preference task. Cognitive science, 39(7), 1704-1726. https://doi.org/10.1111/cogs.12211
Gambrell, L. B., Malloy, J. A., & Mazzoni, S. A. (2011). Evidence-based best practices for comprehensive literacy instruction. In L. M. Morrow & L. B. Gambrell (Eds.), Best practices in literacy instruction (fourth edition) (Vol. 4, pp. 11-56). The Guilford Press.
Guthrie, J. T., & Mosenthal, P. (1987). Literacy as multidimensional: Locating information and reading comprehension. Educational Psychologist, 22(3-4), 279-297. https://doi.org/10.1080/00461520.1987.9653053
Hayes, A. F., & Krippendorff, K. (2007). Answering the call for a standard reliability measure for coding data. Communication methods and measures, 1(1), 77-89. https://doi.org/10.1080/19312450709336664
Henneman, E. A., Cunningham, H., Fisher, D. L., Plotkin, K., Nathanson, B. H., Roche, J. P., Marquard, J. L., Reilly, C. A., & Henneman, P. L. (2014). Eye tracking as a debriefing mechanism in the simulated setting improves patient safety practices. Dimensions of Critical Care Nursing, 33(3), 129-135. https://doi.org/10.1097/DCC.0000000000000041
Jarodzka, H., van Gog, T., Dorr, M., Scheiter, K., & Gerjets, P. (2013). Learning to see: Guiding students' attention via a model's eye movements fosters learning. Learning and Instruction, 25(0), 62-70. https://doi.org/10.1016/j.learninstruc.2012.11.004
Jivet, I., Scheffel, M., Drachsler, H., & Specht, M. (2017). Awareness is not enough: Pitfalls of learning analytics dashboards in the educational practice. European conference on technology enhanced learning, Tallinn, Estonia.
Katalayi, G. B., & Sivasubramaniam, S. (2013). Careful reading versus expeditious reading: Investigating the construct validity of a multiple-choice reading test. Theory and Practice in Language Studies, 3, 877-884. https://doi.org/10.4304/tpls.3.6.877-884
Knoop-van Campen, C., & Molenaar, I. (2020). How teachers integrate dashboards into their feedback practices. Frontline Learning Research, 8(4), 37-51. https://doi.org/10.14786/flr.v8i4.641
Knoop-van Campen, C. A. N., Wise, A., & Molenaar, I. (2021). The equalizing effect of teacher dashboards on feedback in K-12 classrooms. Interactive Learning Environments, 1-17. https://doi.org/10.1080/10494820.2021.1931346
Kok, E. M., Aizenman, A. M., Võ, M. L.-H., & Wolfe, J. M. (2017). Even if I showed you where you looked, remembering where you just looked is hard. Journal of vision, 17(12), 1-11. https://doi.org/10.1167/17.12.2
Kostons, D., van Gog, T., & Paas, F. (2009). How do I do? Investigating effects of expertise and performance-process records on self-assessment. Applied Cognitive Psychology, 23(9), 1256–1265. https://doi.org/10.1002/acp.1528
Krishnan, K. S. D. (2011). Careful versus expeditious reading: The case of the IELTS reading test. Academic Research International, 1(3), 25.
Liu, F. (2010). Reading abilities and strategies: A short introduction. International Education Studies, 3(3), 153-157. https://doi.org/10.5539/ies.v3n3p153
Mason, L., Pluchino, P., & Tornatora, M. C. (2015). Eye-movement modeling of integrative reading of an illustrated text: Effects on processing and learning. Contemporary Educational Psychology, 41, 172-187. https://doi.org/10.1016/j.cedpsych.2015.01.004
Molenaar, I., & Knoop-van Campen, C. (2019). How teachers make dashboard information actionable. IEEE Transactions on Learning Technologies, 12(3), 347-355. https://doi.org/10.1109/TLT.2018.2851585
Murnane, R., Sawhill, I., & Snow, C. (2012). Literacy challenges for the twenty-first century: Introducing the issue. The Future of Children, 3-15. https://doi.org/10.1353/foc.2012.0013
Oudman, S., van de Pol, J., Bakker, A., Moerbeek, M., & van Gog, T. (2018). Effects of different cue types on the accuracy of primary school teachers' judgments of students' mathematical understanding. Teaching and Teacher Education, 76, 214-226. https://doi.org/https://doi.org/10.1016/j.tate.2018.02.007
Pikulski, J. J., & Chard, D. J. (2005). Fluency: Bridge between decoding and reading comprehension. The Reading Teacher, 58(6), 510-519. https://doi.org/10.1598/RT.58.6.2
Rienties, B., Herodotou, C., Olney, T., Schencks, M., & Boroowa, A. (2018). Making sense of learning analytics dashboards: A technology acceptance perspective of 95 teachers. International Review of Research in Open and Distributed Learning, 19(5). https://doi.org/10.19173/irrodl.v19i5.3493
Rosengrant, D., Hearrington, D., & O’Brien, J. (2021). Investigating Student Sustained Attention in a Guided Inquiry Lecture Course Using an Eye Tracker. Educational psychology review, 33(1), 11-26. https://doi.org/10.1007/s10648-020-09540-2
Salmerón, L., Naumann, J., García, V., & Fajardo, I. (2017). Scanning and deep processing of information in hypertext: an eye tracking and cued retrospective think‐aloud study. Journal of Computer Assisted Learning, 33(3), 222-233. https://doi.org/10.1111/jcal.12152
Scheiter, K., Schubert, C., & Schüler, A. (2018). Self-regulated learning from illustrated text: Eye movement modelling to support use and regulation of cognitive processes during learning from multimedia. British Journal of Educational Psychology, 88(1), 80-94. https://doi.org/doi:10.1111/bjep.12175
Schutz, K. M., & Rainey, E. C. (2020). Making sense of modeling in elementary literacy instruction. The Reading Teacher, 73(4), 443-451. https://doi.org/10.1002/trtr.1863
Shute, V. J., & Zapata-Rivera, D. (2012). Adaptive educational systems. In P. J. Durlach & A. Lesgold (Eds.), Adaptive technologies for training and education (Vol. 7, pp. 1-35). Cambridge University Press. https://doi.org/10.1017/CBO9781139049580.004
Špakov, O., Siirtola, H., Istance, H., & Räihä, K. (2017). Visualizing the reading activity of people learning to read. Journal of Eye Movement Research, 10(5), 1-12. https://doi.org/10.16910/jemr.10.5.5
Tijmstra, J., & Boeije, H. (2011). Wetenschapsfilosofie in de context van de sociale wetenschappen. Boom Lemma.
Urquhart, S., & Weir, C. (1998). Reading in a Second Language: Process, Product and Practice. . Routledge.
van de Pol, J., de Bruin, A. B. H., van Loon, M. H., & van Gog, T. (2019). Students’ and teachers’ monitoring and regulation of students’ text comprehension: Effects of comprehension cue availability. Contemporary Educational Psychology, 56, 236-249. https://doi.org/10.1016/j.cedpsych.2019.02.001
Van de Pol, J., Van den Boom-Muilenburg, S. N., & Van Gog, T. (in press). Exploring the relations between teachers’ cue-utilization, monitoring and regulation of students’ text learning. Metacognition and Learning.
van Gog, T., & Jarodzka, H. (2013). Eye tracking as a tool to study and enhance cognitive and metacognitive processes in computer-based learning environments. In R. Azevedo & V. Aleven (Eds.), International Handbook of metacognition and learning technologies (pp. 143-156). Springer Science+ Business media.
Van Gog, T., Jarodzka, H., Scheiter, K., Gerjets, P., & Paas, F. (2009). Attention guidance during example study via the model’s eye movements. Computers in Human Behavior, 25(3), 785-791. https://doi.org/10.1016/j.chb.2009.02.007
Van Leeuwen, A., Knoop-van Campen, C., Molenaar, I., & Rummel, N. (in press). How teacher characteristics relate to how teachers use dashboards: results from two case studies. . Journal of Learning Analytics.
van Leeuwen, A., Rummel, N., & van Gog, T. (2019). What information should CSCL teacher dashboards provide to help teachers interpret CSCL situations? International Journal of Computer-Supported Collaborative Learning, 14(3), 261-289. https://doi.org/10.1007/s11412-019-09299-x
van Wermeskerken, M., Litchfield, D., & van Gog, T. (2018). What am i looking at? Interpreting dynamic and static gaze displays. Cognitive science, 42(1), 220-252. https://doi.org/10.1111/cogs.12484
Verbert, K., Govaerts, S., Duval, E., Santos, J. L., Van Assche, F., Parra, G., & Klerkx, J. (2014). Learning dashboards: an overview and future research opportunities. Personal and Ubiquitous Computing, 18(6), 1499-1514. https://doi.org/10.1007/s00779-013-0751-2
Võ, M. L. H., Aizenman, A. M., & Wolfe, J. M. (2016). You think you know where you looked? You better look again. Journal of experimental psychology: Human perception and performance, 42(10), 1477-1481. https://doi.org/10.1037/xhp0000264
Xhakaj, F., Aleven, V., & McLaren, B. M. (2017). Effects of a teacher dashboard for an intelligent tutoring system on teacher knowledge, lesson planning, lessons and student learning. European Conference on Technology Enhanced Learning,
Zelinsky, G. J., Peng, Y., & Samaras, D. (2013). Eye can read your mind: Decoding gaze fixations to reveal categorical search targets. Journal of vision, 13(14), 1-13. https://doi.org/10.1167/13.14.10