1.
Introduction
Learning
from text is one
of the most essential skills in our modern society and the
ability to
understand challenging texts is an important key to success in
education and
beyond (Mason, Tornatora,
& Pluchino, 2013; McNamara,
2004; Moss, Schunn, Schneider, McNamara, & Vanlehn,
2011). One
of the research traditions that is interested in how students
learn from text
is the domain of Student Approaches to Learning (SAL) (Gijbels, Donche,
Richardson, & Vermunt, 2014;
Lonka, Olkinuora, & Mäkinen, 2004; Richardson, 2000). Research
in the SAL domain is founded on the seminal studies by Marton
and his
colleagues in the 1970s in Sweden (Marton et al.,
1975). They investigated how students went about
reading academic texts
in experimental situations by conducting retrospective
interviews (Marton et al.,
1975; Richardson, 2000). A distinction was made between deep processing
strategies and
surface processing strategies, which has been influential in
the later
development of self-report questionnaires to quantify
individual differences in
students’ processing strategies (Biggs, 1987;
Entwistle & McCune, 2004).
Up till
now, empirical
studies in the SAL field have mainly been focused on the use
of self-report
instruments such as interviews and questionnaires to uncover
differences in students’
general preferences towards processing strategies. Although
these offline
measures are claimed to be reliable and valid at this general
level, many
authors argue that the results are poor indicators of the
actual processing at
a task specific level (Perry &
Winne, 2006; Samuelstuen & Braten,
2007; Veenman, 2005; Veenman, Bavelaar, De Wolf, & Van
Haaren, 2014). Recently,
there has been a plea for the use of more direct and online
measurement tools
when it comes to describe students’ processing strategies (Richardson, 2013). In the present
study we will therefore use eye tracking to map individual
differences in
cognitive processing followed by a cued recall. Eye tracking
provides a unique
opportunity to study processing strategies in a level of
detail that no other
measures can provide (Lai et al., 2013;
van Gog & Jarodzka, 2013). In what follows
we will describe how different processing strategies can be
manipulated in
experimental designs by the assessment demands, and how eye
tracking followed
by a cued recall can be useful to investigate differences in
processing
strategies.
2.
Uncovering differences in processing
strategies
Processing
strategies
refer to cognitive activities a student applies whilst
studying (Vermunt &
Vermetten, 2004). In
general, two main types of processing strategies are described
in the
literature namely deep and surface processing strategies (Gijbels et al.,
2014). Research in the SAL domain showed that deep
processors try to
comprehend what the author wants to say about a certain topic,
try to
understand the overall meaning of the text, try to relate the
message to a
wider context and to prior knowledge, identify the main ideas
and adopt a
critical angle to the conclusion. In contrast, surface
processors direct their
attention towards learning the text itself, focus more on
specific comparisons,
focus on the parts of the text in sequence, memorize details
and definitions,
remember introductory sentences and list points (Biggs & Tang,
2007; Entwistle & Ramsden,
1982; Marton et al., 1975; Richardson, 2000).
2.1.
Processing
strategies and task
demands
In the 1960s, Rothkopf (1966) introduced the
concept of mathemagenic
activities, which refers to activities that stimulate students
to actively
engage in learning. The use of adjunct questions in written
texts is one
example of these mathemagenic activities. One possible type of
an adjunct
question is the inserted post question, which is placed within
the text and
follows the text passage containing the information needed.
These questions
result in a change in the processing strategy on subsequent
text passages. They
steer students attention to a specific type of information in
the text (Hamaker, 1986;
Rothkopf, 1966).
Similarly, researchers in the SAL domain
agree that one of the most
salient contextual variables to influence processing
strategies is the
assessment method (Baeten, Kyndt,
Struyven, & Dochy, 2010; Gielen, Dochy, & Dierick,
2003; Marton et al.,
1975; Scouller, 1998; Scouller & Prosser, 1994; Segers,
Nijhuis, &
Gijselaers, 2006). Research showed
that how students
learn is influenced by their initial preference for a
processing strategy (Baeten et al.,
2010), but they can
shift between deep
and surface processing strategies according to the assessment
demands, also
known as the backwash-effect of assessment (Baeten et al.,
2010; Gielen et al., 2003; Segers et al., 2006). In
contrast to adjunct question research (Hamaker, 1986;
Rothkopf, 1966), research in the
SAL domain
evaluated the effect of the assessment method at the end of a
text or study
process, without inserting questions in the text or
interrupting the study
process.
In the experiments
of Marton et al.
(1975), students were asked to read three texts and to
prepare for answering
some questions on the content after reading them. The
questions they received
after the first two texts were the only indication on how to
behave during
reading the third text. Students in the deep condition
received questions at a
deep level (e.g., making a summary statement), while students
in the surface condition
received reproduction-oriented questions. After studying the
third text, a
semi-structured interview was conducted to gather data on the
effect of the
experimental manipulation on the levels of processing. The
results of the
interviews suggested that students tended to adapt the
intended level of
processing (Marton et al.,
1975; Richardson, 2000). This study was the first study in the SAL
domain to confirm the
possibility to manipulate students’ levels of processing by
appropriate
questions or prompts. It shows that the level of processing
depends on the expected
form of assessment (Richardson, 2000). Another study of Scouller and
Prosser (1994) suggested that the assessment method influences
processing
strategies. Their research showed that multiple-choice
questions led to more
surface processing strategies. Also research of Scouller (1998) investigated how students perceived two
assessment methods namely
multiple-choice examination and an assignment essay and which
processing
strategies they used. The findings were in line with Scouller and
Prosser (1994), multiple-choice examination was perceived as
assessing lower
levels of intellectual abilities and students indicated to
engage in more
surface processing strategies. An assignment essay was
perceived as testing higher-level
intellectual abilities and students engaged in more deep
processing strategies.
A last study of Segers et al.
(2006) showed that students who perceive the demands on
a deep level, to
demonstrate a thorough understanding and integration of
knowledge, are more
likely to employ deep processing strategies. In contrast,
students who perceive
the demands of assessment on a surface level, to acquire
passive acquisition
and reproduction of details, are expected to employ more
surface processing
strategies such as rote learning and concentrating on facts
and details.
2.2.
Processing
strategies and eye
tracking
Online
measures to map cognitive
processing strategies include the think aloud method,
observation of behaviour
and eye movement measurement (Schellings, 2011;
Veenman, 2011). The think aloud method provides a rich source
of data, but it is
intrusive and can alter the processing itself (Ericsson &
Simon, 1993; Veenman, 2005). The main limitation of the observation of
behaviour is that it
cannot detect covert cognitive processes (Veenman, 2005). According to Hyönä and Lorch
(2004) eye tracking is an attractive method for
studying cognitive processing
strategies in comparison with other online measures because
eye tracking
collects several indices of processing simultaneously and does
not disrupt
normal processing.
There
are two theoretical
assumptions that make the relation between eye movement and
cognitive
processing clear: the immediacy assumption and the eye-mind
assumption (Just &
Carpenter, 1980). The immediacy assumption states that
information processing is not
postponed and takes place when the information is encountered.
The eye-mind
hypothesis explains that eye movements are closely linked to
the focus of
attention as students process the information in the text.
Therefore, eye
movements can be used to trace cognitive processing when
learning from text (Hyönä, Lorch,
& Rinck, 2003; Just &
Carpenter, 1980). In
eye tracking research, the movement of the eyeball is recorded
and these
movements are related to a stimulus. This allows us to
investigate to what
parts of the text a student allocates visual attention and for
how long (Holmqvist et al.,
2011; van Gog & Jarodzka, 2013). A
distinction is made between two main measures namely fixations
and saccades.
During fixations the eye is almost completely still and
information can be
extracted from the text. In contrast, during saccades the
focus of visual
attention is moved to another location and the eye is rapidly
moving between
fixations, as a result students are not able to extract
information from text
during saccades (Holmqvist et al.,
2011; Lai et al., 2013; van Gog
& Jarodzka, 2013).
Although
eye tracking
methodology seems a promising tool to investigate students’
processing
strategies, we could not find studies that examine eye
movement behaviour that
results from using different cognitive processing strategies
such as deep and
surface processing. In another related research field, namely
research in
reading comprehension, they already adopted the eye tracking
methodology (Hyönä, Lorch,
& Kaakinen, 2002; Ponce &
Mayer, 2014; Rayner, 1998). More
specifically, the perspective driven text comprehension
framework states that
the allocation of visual attention is influenced by the
reading perspective and
this reading perspective shapes the cognitive processing in
learning from text (Kaakinen &
Hyönä, 2005, 2007, 2010; Kaakinen,
Hyönä, & Keenan, 2002). A
reading perspective refers to the mental frame from which the
reader approaches
a text and this perspective makes parts of the text more
important to the
reader than others (Hyönä et al.,
2003; Kaakinen & Hyönä, 2007). Kaakinen and Hyönä
(2007) gave the example that when you read a travel
guide in order to find
information about a specific country (e.g., Finland), you will
approach the
text with a specific reading perspective. This reading
perspective is thus
content related. Alternatively, processing strategies
correspond to the
different aspects of the learning material on which the
learner focuses (Richardson, 2000). So students with different processing
strategies focus on the same
content but search for other types of information (e.g., facts
and details vs.
essences) (Schellings, van
Hout-Wolters, & Vermunt, 1996). Research that investigates the influence of
reading perspective on
eye movements showed that there is more time spent on relevant
words or facts
in the text than on irrelevant words or facts (Kaakinen &
Hyönä, 2007; Kaakinen et al., 2002). Next to that, relevant words attracted more
refixations than
irrelevant words (Kaakinen &
Hyönä, 2007). Research of Kaakinen and Hyönä
(2005) indicated that the extra time spent on relevant
information is used
to rehearse this information in order to encode it to memory.
Particularly
relevant for research on learning from text is that these
refixations reflect
purposeful and effortful strategic eye behaviour (Ariasi &
Mason, 2010).
Eye
tracking is an
interesting method to investigate cognitive processes, but to
reduce the amount
of inferences required by the researcher, eye movement data
should be combined
with other data such as verbal reports (Hyönä, 2010; van
Gog & Jarodzka, 2013). Recent studies have already applied the think
aloud method to
obtain verbal reports on students’ processing strategies
during reading and
learning from text (Dinsmore &
Alexander, 2012, 2015). Concurrent reporting while learning from text
can affect the eye
movement patterns, and therefore cued retrospective reporting
offers a valuable
alternative in combination with eye tracking (van Gog &
Jarodzka, 2013). Besides recording the eye movement, the eye
tracking software
allows replaying the records of eye movements. Using this eye
movement pattern
as a memory cue, it may help learners to recover how they
encoded and
interpreted elements in the text (Hyönä, 2010;
Penttinen, Anto, & Mikkilä-Erdmann,
2012; van Gog, Paas, & Van Merrienboer, 2005). Because
of the small delay after processing the text and the
presentation of the memory
cue, students are still able to report on their cognitive
processes (Veenman, 2005,
2011). For this reason we chose to use cued
retrospective reporting to
triangulate with eye movement measures.
3.
Present study
Our
study aims to extend
current research on processing strategies by using eye
tracking methodology followed
by a cued recall to map differences in processing strategies.
This more direct
and online way of measuring processing strategies allows to
learn more about
the actual processing behaviour of students while learning
from expository text.
As
stated above
processing strategies shape what information is looked for in
a text and what
information is perceived as relevant (Kaakinen &
Hyönä, 2005). Next to that, research using self-report
measures suggested that
deep processors focus more on essences and surface processors
focus more on
details and definitions (Biggs & Tang,
2007; Entwistle & Ramsden,
1982; Marton et al., 1975; Richardson, 2000). Based
on findings from research on perspective driven text
comprehension (Kaakinen &
Hyönä, 2008) and the SAL domain (Lonka et al.,
2004), we suggest the following hypotheses for
students in the deep
condition (after receiving guiding questions at a deep level)
and students in
the surface condition (after receiving reproduction-oriented
questions):
a)
Hypothesis
1: Students in the
deep condition focus their attention longer on the essentials
(e.g., key
phrases and words) in the text compared to students in the
surface condition.
b)
Hypothesis
2: Students in the
deep condition, more often return back to essences compared to
students in the
surface condition.
c)
Hypothesis
3: Students in the
surface condition focus their attention longer on facts and
details (e.g.,
names) compared to students in the deep condition.
d)
Hypothesis
4: Students in the
surface condition, more often return back to facts and details
compared to students
in the deep condition.
4.
Method
4.1.
Participants
Twenty-eight
students (age range:
18-25) enrolled at the University of Antwerp (Belgium),
participated on a voluntary
basis. Participants were randomly divided in either the deep
condition (DC, N =
14) or the surface condition (SC, N = 14).
Unfortunately, data of nine
respondents could not be used due equipment failure and
problems with eye
tracking calibration. Therefore, data of 19 students were
considered in the
statistical analyses (Table 2). All participants had normal or
corrected-to-normal vision and Dutch was their native
language.
Table 1
Participant
characteristics
|
|
DC |
SC |
|
N |
12 |
7 |
|
Gender |
|
|
|
Male |
5 |
5 |
|
Female |
7 |
2 |
4.2.
Materials
In order
to test our
hypotheses, we based our experimental design on the seminal
studies by Marton et al.
(1975). In their experiments they induced either a deep
or surface
processing strategy by giving students questions after they
studied an academic
text. In our experiment, students were asked to study a series
of three
expository texts (± 800 words) on a topic
they were not familiar with, namely research
on happiness. The texts were taken from the Dutch version of
‘The world book of
Happiness’ (Bormans, 2010). After processing each text they received a
number of evaluation questions
on the preceding text (Figure 1). Students in the deep
condition received
questions at a deep level (e.g., give a summary of the text).
In contrast,
students in the surface condition received
reproduction-oriented questions
(e.g., in which country was the research discussed in the text
conducted?). So
in both conditions students processed the same learning
content, but received
different questions. In the original study, Marton et al.
(1975) interviewed and tested the students after the
third text and concluded
that in the surface condition, students adopted more surface
processing
strategies while students in the deep condition adopted more
deep processing
strategies. Similarly, in our study we analysed the eye
tracking data and cued
recalls from the third text.
Figure 1. Experimental
design. (see pdf)
4.3.
Eye
tracking
Eye
movements were
collected using the Tobii TX300 eye tracker (dark pupil
tracking), manufactured
by Tobii Technology (Stockholm, Sweden). It is integrated into
a 23-inch TFT
monitor with a maximum resolution of 1920 x 1080 pixels. The
camera samples
data at the rate of 300 Hz and registration was binocular.
Tobii TX300 does not
require a head stabilization system and allows for more
freedom of head
movement (37 x 17 cm). Gaze accuracy is 0.4° and gaze
precision is 0.15°, as
reported by the hardware producer. The eye tracker latency is
between 1.0 and
3.3 milliseconds. Data were recorded with Tobii-Studio (3.2)
software. Before starting
the experiment, students were seated about 60 cm from the
screen for the eye
tracking calibration. A five point calibration procedure was
used in which
students needed to track five red calibration dots on a plain,
grey background.
Areas of
interest (AOI’s)
define regions in the text that the researcher is interested
in gathering data
about (Holmqvist et al.,
2011). With regard to our hypotheses we are interested
in key phrases and
keywords for the deep condition and in details and facts for
the surface
condition. Six volunteers (master students in educational
sciences) read the
text in a pilot study to determine the key phrases and
keywords. In total 15
deep AOI’s (e.g., a topic sentence with summary statements)
and three surface AOI’s
(e.g., name of a country) were marked. There were only parts
of the text
defined as AOI’s, so not the whole text was covered with
AOI’s. The total size
of the text was 1490 x 1087 pixels, the smallest AOI was 47 x
31 pixels and the
biggest AOI was 684 x 71 pixels. The complete text could be
seen on the screen,
so scrolling was not needed.
In line
with Hyönä et al.
(2002) first pass fixation time, look back fixation
time and total
fixation time were analysed at the level of AOI’s. An overview
of the
definitions is given in Table 2 (Holmqvist et al.,
2011; Hyönä et al., 2003). Students
were able to process the text in a self-paced manner and
therefore we
calculated relative duration measures. Next to that, AOI’s
differed in size
because they sometimes contained phrases, while others
consisted of only words.
Therefore, AOI measures were normalized by calculating the
reading depth
measure (Holmqvist et al.,
2011; Holmqvist & Wartenberg,
2005; Holsanova, Holmqvist, & Rahm, 2006). This
reading depth measure is defined by the total time spent in an
AOI per cm2
and is an indication of how densely an AOI is processed. So
for the three
measures described in Table 2 we calculated relative measures
and reading depth
measures.
Table 2
Overview of eye tracking
measures and
their definitions
|
Measure |
Definition |
|
First pass fixation time |
The time spent in an AOI
when it was visited for the first time. A visit can
consist of more fixations. It reflects early
processing and object recognition. |
|
Look back fixation time |
Duration of all the
regressions back to an AOI. It reflects delayed
processing, for example to integrate information. |
|
Total fixation time |
The time spent in an AOI during the whole
trial, it is the sum of the first pass fixation time
and the look back fixation time in that AOI. |
The
fixation indices were
calculated for either the group of deep AOI’s or the group of
surface AOI’s. We
used the Tobii fixation filter for fixation identification,
which is an
implementation of a classification algorithm proposed by Olsson (2007). It uses a velocity threshold (35 pixels/window)
and a distance
threshold (35 pixels). For all the measures, the means and
standard deviations
were calculated. To compare students in both conditions, we
used non-parametric
tests due to the small sample sizes (van Gog et al.,
2005). Therefore the medians together with the first
and third quartile
were calculated as well. Relative measures and reading depth
measures for the eye
movement measures were compared for students in both
conditions using
Mann-Whitney U tests. We reported the exact two-tailed
significance. Also in
line with van Gog et al.
(2005), we used a less stringent significance level of
0.10 to avoid type
II error and to increase power.
4.4.
Cued
Recall
After
the eye tracking
experiment, a cued recall was conducted. After processing the
third text, the
experimenter informed students that they would watch the
replay of eye
movements of the third text together. The cued recall was
conducted by using
gaze videos produced by Tobii-Studio software (3.2). In the
cued recall, a
video showed the text and a moving red dot representing the
point of fixation.
The bigger the dot, the longer the fixation lasted. Students
saw their gaze
videos at the same speed they processed the text. The
interviewer instructed
students to watch the video and to tell the interviewer what
they were thinking
during processing the text. The interviewer also stated that
she would
occasionally stop the video and ask questions about the
reading process, such
as ‘Here you fixated a lot, what where you doing?’ or ‘Here
you are going back
in the text, what were you doing?’.
Table 3
Coding scheme for the
cued recall
analysis
|
Strategy |
Example |
DC |
SC |
|
Surface processing |
|
66 (65,3%) |
45 (100%) |
|
Rereading |
I tried to understand that part so I was
rereading it. |
|
|
|
Skimming |
Now I am reading it again and just
scanning for important words in the text. |
|
|
|
Guessing meaning word in context |
That was GNP, I was wondering what the
meaning of that word was. |
|
|
|
Rehearsing |
Those countries, I was trying to remember
them. |
|
|
|
Connecting to prior text |
I realise that I go back a lot in the
text and that is because I am trying to link parts of
the text. |
|
|
|
Connecting to the research task |
I guess the first paragraph was going to
give an overview about the rest of the task, so I
thought that was important. |
|
|
|
Detecting mistakes in the text |
I was looking at the ‘n’ that was missing
in that word. |
|
|
|
Deep processing |
|
35 (34,7%) |
0 (0%) |
|
Questioning |
I was wondering what they meant with that
phrase. |
|
|
|
Paraphrasing |
First, they name something and then you
know a summation is coming. Second, they talk about
cross national comparisons, … |
|
|
|
Connecting to personal experiences |
You try to process the text critically
and to take you own findings and personal experience
into account. |
|
|
|
Interpreting and elaborating |
What I do most of the time is reading the
text and then trying to analyse what I just read. In
this way I get a better picture of what the text is
about. |
|
|
The cued
recalls were
transcribed from the audiotapes. Next to that, we linked
comments of the cued
recalls to the part of the text that was discussed. The cued
recalls were coded
based on an initial set of ten codes developed in a study of Dinsmore and
Alexander (2015). Specifically, comments were coded as either a
surface or deep
processing strategy (Table 3). After coding the interviews
deductively, we
added one extra code in the surface processing category namely
“detecting
mistakes in the text”. Transcripts were coded with the
qualitative analysis
software package Nvivo 10. Two judges (authors LC and LG)
coded the cued recalls
and an inter-rater agreement of 73% was reached, which is
considered as
substantial. We compared the number of coded utterances in
each condition
between the two categories (Table 3). We first analysed the
data on a general
level and looked for differences between students in both
conditions. We also
analysed the data at a more fine-grained level to see whether
the reported
strategies are linked to AOI’s and to examine differences at
the AOI level
between groups.
5.
Results
Table 4
shows the means
and standard deviations. Standard deviations for the measures
in the deep
condition are higher than in the surface condition. This may
be an indication
that students in the deep condition differ more from each
other. When we look
at the cued recall results of students in the deep condition,
some students
pointed out that they sometimes took a pause to integrate
processed information
instead of looking back. This may also be an indication that
there are two
types of students in the deep condition, on the one hand
students who process
information immediately and take a pause to integrate
information and on the
other hand students who need to look back to parts in the text
to integrate
this information and to encode it to memory.
“Sometimes I keep staring at the text, because
I try to visualize it
for myself” (R7, DC)
“I sometimes have the feeling that when I am
staring at a word that
I am not processing that word but that I am just taking a
moment to think about
what I have read” (R3, DC)
“Sometimes I have the feeling that I am staring
at something in the
text, to process the things I just read before” (R5,
DC)
The most
reported
processing strategy in the cued recalls, is the surface
processing strategy and
more specifically rereading. Students in both groups indicated
that they reread
parts of the text the most. Only students in the deep
condition reported deep
and surface processing strategies. Students in the surface
condition only
reported surface processing strategies. Deep processing
strategies are reported
on a more general level and are not linked to certain phrases,
paragraphs or
AOI’s in the text.
“When you know you will need to answer questions
after reading the text,
you try to read the text critically and I always try to take
into account my
personal experiences and findings.” (R5, DC)
“I first think about what I read in the text, before
I proceed with the
next part. I try to make a summary for myself of what I read
in the previous
parts.” (R3, DC)
Table 4
Means and standard
deviations
|
|
Essentials |
Facts
and details |
||||||
|
|
DC |
SC |
DC |
SC |
||||
|
|
M |
SD |
M |
SD |
M |
SD |
M |
SD |
|
FPFT r |
2.69 |
2.22 |
3.23 |
1.48 |
0.40 |
0.21 |
0.38 |
0.25 |
|
FPFT rd |
45.53 |
23.77 |
57.50 |
25.19 |
80.60 |
24.49 |
79.02 |
51.30 |
|
LBFT r |
12.65 |
3.63 |
11.22 |
2.69 |
1.09 |
0.49 |
1.94 |
0.77 |
|
LBFT rd |
285.66 |
192.02 |
200.75 |
46.39 |
268.20 |
236.77 |
401.22 |
168.88 |
|
TFT r |
15.35 |
3.01 |
14.46 |
2.85 |
1.50 |
0.57 |
2.32 |
0.84 |
|
TFT rd |
331.19 |
189.72 |
285.25 |
43.45 |
348.80 |
234.62 |
480.24 |
185.68 |
|
FPFT = first pass fixation time; LBFT =
look back fixation time; TFT = total fixation time; r
= relative measure; rd = reading depth measure. |
||||||||
We
compared the total
reading time of students in both conditions with a
Mann-Whitney U test, but no
significant differences were found (U
= 41, p = 0.97). So
students in both
conditions spent on average the same amount of time on
processing the text.
5.1.
Essentials
in the text
Table 5
shows the medians
and quartiles for the essentials in the text for students in
both conditions. We
conducted Mann-Whiney U tests on all these measures but no
significant
differences were found between students in both groups.
Table 5
First quartile, median
and third quartile
for relative measures and reading depth measures.
|
|
DC |
SC |
|
||||||||||||
|
|
|
|
|
|
|
Mann-Whiney
U |
|||||||||
|
|
Q1 |
Mdn |
Q3 |
Q1 |
Mdn |
Q3 |
U |
p |
|||||||
|
FPFT r |
1.53 |
1.92 |
2.86 |
2.13 |
2.57 |
4.43 |
27 |
0.227 |
|||||||
|
FPFT rd |
27.65 |
41.14 |
45.17 |
41.18 |
48.55 |
75.67 |
30 |
0.340 |
|||||||
|
LBFT r |
10.93 |
12.92 |
15.88 |
9.39 |
10.60 |
12.69 |
56 |
0.261 |
|||||||
|
LBFT rd |
173.39 |
215.14 |
354.49 |
170.57 |
178.85 |
215.59 |
53 |
0.385 |
|||||||
|
TFT r |
13.97 |
15.63 |
18.26 |
12.42 |
13.02 |
16.41 |
53 |
0.385 |
|||||||
|
TFT rd |
209.29 |
261.89 |
396.90 |
235.15 |
257.08 |
267.00 |
45 |
0.837 |
|||||||
Results
from the cued
recalls indicate that both students in the deep and surface
condition reread
essentials in the text. A reason for rereading is that they
did not really
understand essential parts of the text. The motivation to
better understand
these essential parts in the text is only reported by students
in the deep
condition. These results suggest that students in the deep
condition reread
these parts at a deeper level to get a better understanding.
“I am rereading a lot, I read something fast and then
I think whether I
understood it and no I did not, so then I go back again” (R3,
DC)
“I was trying to understand that part better, so that
is why I was
rereading it over and over again” (R2, DC)
Both groups
indicated skimming the
text after reading it for the first time to look back at the
essential parts of
the text.
“What I often do when I finished reading, is
rereading only the
essential parts of the text” (R4, DC)
“I am just scanning quickly to see if I missed
important words in the
text” (R16, SC)
A final
finding from the
cued recall results is that both groups guessed the meaning of
keywords in
context, when they did not understand the word. Overall, cued
recall results
are in line with results from eye tracking, in that no big
differences are
found between both groups when processing essential parts in
the text.
“Here, that was a difficult word, elitist, I tried to
understand the
meaning in the text” (R8, DC)
“Some keywords I do not know, I need to think about
them or see the
context to understand them” (R19, SC)
5.2.
Facts
and details in the text
Table 6
shows the medians
and quartiles for facts and details in the text for students
in both conditions.
Students in the surface condition spent relatively more time
on facts and
details when they looked back at them and also during the
whole experiment.
Next to that, these students read the facts and details with
more depth than
students in deep condition when they look back at them and
during the whole
experiment.
Table 6
First quartile, median
and third quartile
for relative measures and reading depth measures.
|
|
DC |
SC |
|
||||||||||||
|
|
|
|
|
|
|
Mann-Whiney
U |
|||||||||
|
|
Q1 |
Mdn |
Q3 |
Q1 |
Mdn |
Q3 |
U |
p |
|||||||
|
FPFT r |
0.23 |
0.38 |
0.54 |
0.19 |
0.39 |
0.54 |
46 |
0.773 |
|||||||
|
FPFT rd |
66.34 |
78.06 |
102.04 |
38.77 |
82.39 |
112.36 |
44 |
0.902 |
|||||||
|
LBFT r |
0.74 |
1.04 |
1.41 |
1.74 |
2.28 |
2.41 |
17 |
0.036 |
|||||||
|
LBFT rd |
160.56 |
178.28 |
270.98 |
329.09 |
423.92 |
529.68 |
20 |
0.068 |
|||||||
|
TFT r |
1.01 |
1.47 |
1.95 |
2.23 |
2.62 |
2.86 |
15 |
0.022 |
|||||||
|
TFT rd |
241.65 |
276.45 |
351.52 |
440.90 |
479.24 |
611.01 |
18 |
0.045 |
|||||||
Cued
recall results
showed that students in the surface condition repeated facts
and details in the
text, while students in the deep condition did not. Other
coding categories did
not show a link with processing facts and details in the text.
Again we can see
a clear link between the eye movement measures and the results
from the cued
recalls.
“The names of those countries, I really tried to
remember those” (R14,
SC)
“Those four countries, I memorized them” (R19,
SC)
“I tried to remember the name of the author, I
thought that would be
important” (R17, SC)
6.
Conclusion and discussion
This
exploratory study
aimed at extending current research on processing strategies
during learning
from expository text. Research in the SAL domain is mostly
based on students’ self-reports
of processing strategies at a general level in which the
context of learning is
not taken into account (Dinsmore &
Alexander, 2012; Gijbels et al., 2014). By looking at the actual processing behaviour
of students while learning
from expository text, this study makes a first preliminary
contribution to the
field by using a more direct and online measurement tool at a
task specific
level that takes the context explicitly into account. It is
the first experimental
study to explore students’ cognitive processing strategies at
a task specific
level using objective online measures. Most of the research
using online
measures is based on the think aloud method, which can alter
the processing
itself (Veenman, 2005). By using eye tracking methodology followed by a
cued recall this
problem is circumvented, in that this method does not demand
students to manage
cognitive load of the task completion and self-reports of
strategies at the
same time (Samuelstuen &
Braten, 2007).
In our
study we
manipulated the task demands to steer processing strategies.
Results from the
cued recalls indicated that this manipulation was successful
as students in the
deep condition reported a combination of surface and deep
processing
strategies, while students in the surface condition only
reported surface
processing strategies. This is in line with previous research
that showed that demands
on a deep level, to demonstrate a thorough understanding, lead
to more deep
processing strategies whereas demands on a surface level, to
acquire passive
acquisition of facts and details, lead to more surface
processing strategies (Marton et al.,
1975; Richardson, 2000; Scouller,
1998; Scouller & Prosser, 1994; Segers et al., 2006). Results
of the cued recalls indicated that students in both conditions
processed facts
and details and essential parts in the text but they did it in
a different way.
These results are similar to results from think aloud studies
in which
processing strategies were examined without manipulating task
demands (Dinsmore &
Alexander, 2012, 2015; Penttinen et
al., 2012).
Based on
the eye movement
data, we cannot confirm the first and second hypothesis that
stated that
students in the deep condition focus their attention longer on
essentials in
the text compared to students in the surface condition and
that they return
more back to them. Both groups of students spent time on
processing the
essentials in the text. Although we could not find differences
between groups
based on their eye movement data, results from the cued
recalls indicated that
students in the deep condition reread the essentials in the
text to understand
them better. This motivation to better understand these parts
is related to a
deep way of processing (Biggs & Tang,
2007; Entwistle & Ramsden,
1982).
Students in the surface condition did not report this
motivation. These descriptive
findings indicate that students in our sample processed the
text in a different
way but more substantive research is needed to further explore
found
differences in overt eye movement behaviour. In contrast with
research from the
angle of perspective driven text comprehension, these
essential parts do not seem
to be perceived as more relevant by students in the deep
condition (Kaakinen &
Hyönä, 2005, 2007, 2008), they
are just processed in a more deep way. Another interesting
finding from the
cued recall results is that some students in the deep
condition indicated that
they took a pause at some places in the text to integrate the
processed information
instead of actively looking back. Other students in the deep
condition reported
actively looking back at these essential parts in the text.
Also the higher
standard deviations for students in the deep condition may be
an indication of
these differences. It is in line with other research that
shows that building
the necessary links to incorporate text information to the
developing memory
representation can be achieved mentally or can result in overt
behaviour in
which students actively reread essential parts (Hyönä et al.,
2003; Kaakinen & Hyönä, 2008). So, based on these preliminary findings, we
suggest that some deep
processors actively return back to essentials to encode it to
memory, while
others take a pause to integrate the new information without
looking back to
this information. Further research is needed to confirm these
findings.
Regarding
the third and
fourth hypothesis, the results indicated that students in the
surface condition
indeed looked longer at facts and details and returned more
back to them. It
seems that students in the surface condition switch to
strategic processing by
paying more attention to relevant parts, namely facts and
details (Kaakinen &
Hyönä, 2007). Research of Kaakinen and Hyönä
(2005) showed that the extra time spent on relevant
information is used to
rehearse this information in order to encode it to memory.
Results from both
eye tracking and cued recalls indicate that facts and details
are more repeated
in order to encode into memory in the surface condition (Kaakinen &
Hyönä, 2007). Only students in the surface condition reported
repeating facts
and details, while students in the deep condition did not
report learning
activities like that.
Although
our findings
suggest that eye tracking followed by a cued recall is a
fruitful way to investigate
processing strategies, we want to stress the preliminary
nature of this study
because of some limitations. An important limitation of this
study is the small
sample size. Due to equipment failure or problems with eye
tracking
calibration, the sample size decreased at the onset of this
study. Because of
this smaller sample size we decided to use non-parametric
tests and deepened
the results obtained by a cued recall. We also raised the
significance level to
increase power due to the smaller sample size (van Gog et al.,
2005). The findings from this study can serve as a
baseline for further
research in which larger samples can be used to increase power
without
adjusting the significance level. Another limitation of this
study is that we
used a between groups design. Reading times and online
processing strategies
can vary among adult readers (Hyönä et al.,
2002; Kaakinen & Hyönä, 2008). Therefore we suggest for further research to
use a within groups
design in which students use both processing strategies to
take this individual
variability into account. Another way to understand the
significance of individual
variability is to include control variables such as reading
ability, interest
in the topic and prior knowledge about the topic (Fox, 2009; Mason
et al., 2013). By increasing the sample size and using a
within subjects design,
more complex statistical analysis can be conducted to confirm
our preliminary
findings. In this way it will be possible to give more
generalized statements
regarding processing strategies as measured by eye tracking. A
last limitation
is that students needed to process the text on a computer
screen to be able to
use the eye tracking. By doing this it does not reflect the
natural setting in
which students habitually process learning contents.
Despite
the limitations,
this study was able to show that eye tracking followed by a
cued recall is a
promising tool to examine students’ processing strategies. An
important finding
from our study is that it is valuable to combine eye tracking
with a cued
recall, because differences in processing strategies not
always lead to overt
eye movement behaviour (Hyönä et al.,
2003; Kaakinen & Hyönä, 2008). By using a cued recall we were able to uncover
differences in
processing strategies that were not reflected in eye movement
behaviour. Based
on our preliminary findings, the combination of eye tracking
and a cued recall seems
to be a promising tool to further investigate cognitive
processing strategies
when learning from text. Students in the deep condition do not
seem to look
longer at essentials and do not seem to return more back to
them, but processed
them in a more deep way then students in the surface
condition. Results suggest
that students in the surface condition looked longer at facts
and details and
did return more back to them. This first exploratory eye
tracking study in the
SAL domain is an important illustration on how processing
strategies can be
further examined beyond the use of self-report questionnaires.
In our opinion
it would be worthwhile to use this innovative eye tracking
methodology in
multi-method designs to triangulate it with often used
self-report measures to
look for convergent or divergent validity. In our study we
steered students’
processing strategies by task demands. Although research
indicated that it is
possible to influence processing strategies by manipulating
this contextual
variable (Baeten et al.,
2010; Gielen et al., 2003; Marton et al., 1975; Scouller,
1998; Scouller &
Prosser, 1994; Segers et al., 2006), it would be
interesting to combine
it with these self-report measures in order to examine a more
natural way of
processing behaviour. Next to
that, using multiple sources of data is important to develop a
comprehensive
understanding of how we can adequately measure students’
processing strategies.
Eye tracking methodology followed by a cued recall in the SAL
domain can also deepen
the conceptual underpinnings on what constitutes deep and
surface processing of
learning contents.
Keypoints
Eye tracking followed
by a cued recall is a
promising tool to uncover differences in students’ processing
strategies while
learning from expository text.
Students in the deep
condition do not look longer at
the essentials, but they process them in a more deep way by
trying to
understand these parts better.
Students in the
surface condition look longer at
facts and details and try to rehearse these parts.