Eye Tracking Indicators of Reading Approaches in Text-Picture Comprehension

Article Sidebar

PDF HTML
Published: Oct 27, 2014
DOI: https://doi.org/10.14786/flr.v2i4.98
Keywords:
Multimedia Learning, Item Solving, Mental model

Main Article Content

Fang Zhao
University of Koblenz-Landau
Wolfgang Schnotz
University of Koblenz-Landau
Inga Wagner
University of Koblenz-Landau
Robert Gaschler
University of Koblenz-Landau

Abstract

Despite numerous researches on reading as well as multimedia comprehension, reading approaches in text-picture comprehension have become a focus of research only rarely. The current experiment aims at exploring text-picture comprehension in different reading approaches with item solving. In a within subjects design using eye tracking, seventeen secondary school students processed our blended text and picture materials in three different ways. (1) Unguided processing with text and picture and without the question. (2) Information gathering to answer the question after the prior experience with text and pictures. (3) Comprehending text and pictures to solve the task with the prior information of the question. Eye tracking data showed that text and picture play different roles in comprehension in different reading approaches. The data suggest that (1) text plays the main role to construct the mental model in unguided spontaneous processing of text and picture. (2) Students seem to mainly rely on the picture as an external representation when trying to answer a question after the prior experience with the material. (3)Text and picture are both used heavily when students work out an answer with the prior experience with the question presented. The text likely plays a major part in guiding the processing of meaning, whereas the picture is used as an external representation for information retrieval. Our work provides a first step towards an Item Solving Model in Text-Picture Comprehension. It also provides pedagogical implications for learning in secondary school.

Article Details

How to Cite
Zhao, F., Schnotz, W., Wagner, I., & Gaschler, R. (2014). Eye Tracking Indicators of Reading Approaches in Text-Picture Comprehension. Frontline Learning Research, 2(5), 46–66. https://doi.org/10.14786/flr.v2i4.98
Issue
Vol. 2 No. 5 (2014): Frontline Learning Research
Section
Articles

FLR adopts the Attribution-NonCommercial-NoDerivs Creative Common License (BY-NC-ND). That is, Copyright for articles published in this journal is retained by the authors with, however, first publication rights granted to the journal. By virtue of their appearance in this open access journal, articles are free to use, with proper attribution, in educational and other non-commercial settings.

Author Biographies

Fang Zhao, University of Koblenz-Landau

Department of General and Educational Psychology

Wolfgang Schnotz, University of Koblenz-Landau

Department of General and Educational Psychology

References

Anderson, R. C., & Pearson, P. D. (2002). A schema-theoretic view of basic processes in reading comprehension. In P. D. Pearson, B. Rebecca, M. L. Kamil & P. Mosenthal (Eds.), Handbook of reading research (pp. 255-291). Mahwah, NJ: Lawrence Erlbaum.

Duffy, S. A. (1992). Eye movements and complex comprehension processes. In K. Rayner (Ed.), Eye movements and visual cognition: scene perception and reading (pp. 462-471). New York: Springer-Verlag.

Eitel, A., Scheiter, K., Schüler, A., Nyström, M., & Holmqvist, K. (2013). How a picture facilitates the process of learning from text: Evidence for scaffolding. Learning and Instruction, 28, 48-63. doi: 10.1016/j.learninstruc.2013.05.002

Frase, L. T. (1967). Learning from prose material: Length of passage, knowledge of results, and position of questions. Journal of Educational Psychology, 58(5), 266-272. doi: 10.1037/h0025028

Frase, L. T. (1968). Effect of question location, pacing, and mode upon retention of prose material. Journal of Educational Psychology, 59(4), 244-249. doi: 10.1037/h0025947

Frase, L. T., & Silbiger, F. (1970). Some adaptive consequences of searching for information in a text. American Educational Research Journal, 7(4), 553-560. doi: 10.2307/1161836

Friedman, A., & Liebelt, L. S. (1981). On the time course of viewing pictures with a view towards remembering. In D. F. Fisher, R. A. Monty & J. Senders (Eds.), Eye movements: cognition and visual perception (pp. 137-155). Hillsdale, NJ: Lawrence Erlbaum Associates.

Godau, C., Wirth, M., Hansen, S., Haider, H., & Gaschler, R. (2014). From marbles to numbers—estimation influences looking patterns on arithmetic problems. Psychology, 5, 127-133. doi: 10.4236/psych.2014.52020

Goldberg, J. H., & Kotval, X. P. (1999). Computer interface evaluation using eye movements: methods and constructs. International Journal of Industrial Ergonomics, 24(6), 631-645. doi: http://dx.doi.org/10.1016/S0169-8141(98)00068-7

Hannus, M., & Hyönä, J. (1999). Utilization of illustrations during learning of science textbook passages among low- and high-ability children. Contemporary Educational Psychology, 24(2), 95-123. doi: http://dx.doi.org/10.1006/ceps.1998.0987

Hegarty, M. (1992). Mental animation: Inferring motion from static displays of mechanical systems. Journal of Experimental Psychology: Learning, Memory, and Cognition, 18(5), 1084-1102. doi: 10.1037/0278-7393.18.5.1084

Hochpöchler, U., Schnotz, W., Rasch, T., Ullrich, M., Horz, H., McElvany, N., & Baumert, J. (2012). Dynamics of mental model construction from text and graphics. European Journal of Psychology of Education, 1(22). doi: 10.1007/s10212-012-0156-z

Holmqvist, K., Nyström, M., Andersson, R., Dewhurst, R., Jarodzka, H., & Weijer, J. v. d. (2011). Eye tracking : a comprehensive guide to methods and measures. Oxford; New York: Oxford University Press.

Johnson, C. I., & Mayer, R. E. (2012). An eye movement analysis of the spatial contiguity effect in multimedia learning. Journal of Experimental Psychology: Applied, 18(2), 178-191. doi: 10.1037/a0026923

Just, M. A., & Carpenter, P. A. (1976). Eye fixations and cognitive processes. Cognitive Psychology, 8(4), 441-480.

Kulhavy, R., Stock, W., & Kealy, W. (1993). How geographic maps increase recall of instructional text. Educational Technology Research and Development, 41(4), 47-62. doi: 10.1007/bf02297511

Lee, K., Khng, K. H., Ng, S. F., & Kong, J. N. L. (2013). Longer bars for bigger numbers? Children’s usage and understanding of graphical representations of algebraic problems. Frontline Learning Research, 1 (1), 81-96.

Mason, L., Pluchino, P., Tornatora, M. C., & Ariasi, N. (2013). An eye-tracking study of learning from science text with concrete and abstract illustrations. Journal of Experimental Education, 81(3), 356-384. doi: 10.1080/00220973.2012.727885

Mayer, R. E. (2005). Cognitive theory of multimedia learning. In R. E. Mayer (Ed.), The Cambridge handbook of multimedia learning (pp. 31-48). Cambridge, U.K.; New York: Cambridge University Press.

Paivio, A. (1986). Mental representations: A dual coding approach. New York: Oxford University Press.

Pinker, S. (1990). A theory of graph comprehension. In R. Freedle (Ed.), Artificial intelligence and the future of testing. (pp. 73-126). Hillsdale, NJ England: Lawrence Erlbaum Associates, Inc.

Rayner, K. (1998). Eye movements in reading and information processing: 20 years of research. Psychological Bulletin, 124(3), 372-422. doi: 10.1037/0033-2909.124.3.372

Rothkopf, E. Z. (1964). Learning and the educational process; selected papers from the Research Conference on Learning and the Educational Process, held at Stanford University, June 22-July 31, 1964. In J. D. Krumboltz (Ed.), Research Conference on Learning the Educational Process (pp. 193-221). Chicago: Rand McNally.

Rothkopf, E. Z. (1966). Learning from written instructive materials: An exploration of the control of inspection behavior by test-like events. American Educational Research Journal, 3(4), 241-249. doi: 10.3102/00028312003004241

Schmidt-Weigand, F., Kohnert, A., & Glowalla, U. (2010). Explaining the modality and contiguity effects: New insights from investigating students' viewing behaviour. Applied Cognitive Psychology, 24(2), 226-237. doi: 10.1002/acp.1554

Schnotz, W., Baadte, C., Müller, A., & Rasch, R. (2010). Creative thinking and problem solving with depictive and descriptive representations. In L. Verschaffel, E. De Corte, J. Elen & T. de Jong (Eds.), Use of external representations in reasoning and problem solving (pp. 11-35). Amsterdam: Elsevier.

Schnotz, W., & Bannert, M. (2003). Construction and interference in learning from multiple representation. Learning and Instruction, 13(2003), 141-156. doi: http://dx.doi.org/10.1016/S0959-4752(02)00017-8

Xi, X. (2004). Investigating language performance on the graph description task in a semi-direct oral test. (Doctoral dissertation), University of California, L.A., L.A., California, US. Retrieved from http://search.ebscohost.com/login.aspx?direct=true&db=psyh&AN=2004-99003-039&lang=de&scope=site Available from EBSCOhost psyh database.