Figure 2.
Cycle of differentiation. (see pdf)
Organisation
is placed centrally in the cycle, because successful
implementation of differentiation depends on a facilitative
organisational structure
and good classroom management. A key
organisational
characteristic of the model for differentiation agreed upon by
the experts is the assignment of students to subgroups based
on achievement level, allowing teachers to make instructional
adaptations for subgroups of students with similar educational
needs. Only the remaining individual educational needs that
are not met within the subgroup call for individual
accommodations.
The
first step in the cycle of differentiation is the
identification of educational needs. Initially, the teacher
should assign students to subgroups (typically a
low-achieving, an average-achieving and a high-achieving
subgroup) based on their results on standardised tests and
curriculum-based tests. In the course of the schoolyear,
teachers continuously gather new and more detailed information
about students’ educational needs, for example with the
analysis of daily work, informal observations and diagnostic
conversations. The
subgroups should be flexible, i.e. students should be able to
switch groups based on changes in their educational needs.
Based on the
educational needs of the students, differentiated goals should
be set. Overarching objectives (the material that students
should master at the end of primary school) and lesson goals
(goals for a specific lesson) are distinguished. Overarching
objectives should not only be formulated for average-achieving
students but also specifically for low-achieving and
high-achieving students (see also Appendix 1, differentiation
in goals). The overarching objectives should be translated
into concrete lesson goals, which are provided mostly by the
curriculum. However, only some of
the mathematics curricula available in the Netherlands
differentiate lesson goals for three achievement levels.
When the curriculum does not differentiate goals
sufficiently, the teacher should formulate challenging but
realistic lesson goals for all subgroups.
Based
on the educational needs and the goals that have been set, the
teacher differentiates instruction through broad whole-class
instruction, subgroup instruction tailored to the needs of
that subgroup, and individual adaptations. During whole-class
instruction, the teacher should serve a broad range of
educational needs by varying the difficulty level of
questions, stimulating all children to think about the answer
to a question by giving thinking time, teaching at various
levels of abstraction, and using several input modalities
(e.g. visual, verbal, tactile). In subgroup instruction, the
teacher should adapt instruction to the educational needs of
low-achieving and high-achieving students. It is assumed that,
in general, low-achieving students need more guidance (e.g.
explicit instruction) and instruction at lower levels of
abstraction (e.g. using blocks to represent and calculate a
sum) while high-achieving students need more exploratory
instruction about advanced content with a focus on conceptual
understanding (e.g. the relation between multiplication and
division). To the extent possible, the teacher should also
take into account individual differences during subgroup
instruction and while giving individual feedback.
In
the practice phase of the lesson, the subgroups need
quantitatively and qualitatively different tasks. For the
low-achieving subgroup, completing all regular tasks is often
not realistic, so the tasks that are crucial for mastery of
the objectives for low-achieving students should be selected
(the remaining tasks can still be completed when students have
time left). For the high-achieving subgroup, the regular
material should be compacted. For the most popular Dutch
mathematics curricula, guidelines exist to inform the teacher
which tasks can be skipped by high-achieving students. In
other cases, the teacher should remove most of the repetitive
tasks and select the tasks that are crucial steps towards
mastery of the objectives. The time freed up by compacting
should be spent on enrichment. Some curricula provide
enrichment tasks and these tasks can be used in some cases,
but they are often not sufficiently challenging for very
high-achieving students. Therefore, supplemental enrichment curricula
should also be used. Technological applications
such as mathematics websites and instructional computer
programmes can also be valuable tools for individual
differentiation, provided that they are used deliberately for
additional practice in areas the student does not master yet
or for enrichment at an appropriate challenge level.
The
final step in the cycle of differentiation is the evaluation
of progress and learning process. Based on daily work and
achievement tests, the teacher should evaluate whether the
students have met the lesson goals. Regarding process, the
teacher should evaluate whether the applied adaptations of
instruction and practice had the desired effect. For example,
when a teacher has intentionally taught the low-achieving
subgroup at a lower level of abstraction, the teacher should
evaluate whether this was helpful for these students. To gauge
the effectiveness of the accommodations made, the teacher may
supplement achievement results with informal measures such as
observations or diagnostic conversations. The evaluation phase
informs the teacher about students’ current achievement level
and about instructional approaches that work for these
students, completing the cycle and serving as new input for
the identification of educational needs.
2.4 Concluding
summary
Study 1
The
aim of Study 1 was to operationalise the concept of
differentiation by achieving consensus among a consortium of
mathematics experts about a coherent set of strategies for
differentiating primary school mathematics education. A
combination of focus group discussions and the Delphi
technique was used to investigate the experiential knowledge
of eleven experts in mathematics education systematically.
Consensus was reached on all summarising concepts and on the
majority of specific statements from the Delphi questionnaire.
The input from the experts was synthesised into a cycle of
differentiation consisting of the following five steps:
identification of educational needs, differentiated goals,
differentiated instruction, differentiated practice, and
evaluation of learning progress and process. For each step,
strategies were specified, providing teachers with concrete
guidelines for implementing differentiation in primary school
mathematics.
3. Study
2
3.1
Aims Study 2
Study 2
linked the results of Study 1 to teachers’ daily practice by
investigating teachers’ self-assessed use of the strategies
for differentiation recommended by the experts. Therefore, we
developed the Differentiation Self-Assessment Questionnaire
(DSAQ) which covers the recommended strategies in five
subscales corresponding with the five steps of the cycle of
differentiation (see also section 3.2.2). Since the DSAQ was
newly developed, we also aimed to investigate its statistical
properties, including its factor structure and relation with
other scales.
The
development of a new instrument was necessary to ensure
coverage of the broad set of strategies recommended by the
experts in Study 1. Another recently developed instrument to
measure teachers’ self-reported use of differentiation is the
Differentiated Instruction Scale (DIS; Roy et al., 2013). This
instrument was based on a similar theoretical framework and
there is overlap between the content of the items of the DIS
and the DSAQ. However, the DIS has only twelve items and is
not sufficiently specific to measure all strategies
recommended by the experts. Regarding progress monitoring, for example, the DIS
includes the rather general item ‘analyse data about students’
academic progress’ while the DSAQ distinguishes between the
different types of progress monitoring recommended by the
experts, ranging from standardised tests to diagnostic
interviews. Thus, the added value of the DSAQ is that it is a
detailed measure of the specific strategies recommended by the
experts in Study 1.
The first
aim of the current study was to examine the factor structure
of the DSAQ. The literature reviewed in the introduction
indicates that effective differentiation entails two
components: progress monitoring and instructional adaptations.
The steps in the cycle of differentiation reflect these
components: identification of educational needs and evaluation
of progress and process involve progress monitoring, while
differentiated goals, instruction, and practice involve
instructional adaptations. For the DIS, Roy et al. (2013)
found that a model with these two factors provided a better
fit to the data than a model in which all items loaded on one
general differentiation factor. Therefore, we investigated
whether the DSAQ has a similar factor structure by comparing
the fit of a two-factor model with one factor for progress
monitoring and one factor for instructional adaptations to the
fit of a one-factor model.
The second
aim was to examine the convergent and divergent validity of
the DSAQ by investigating its relationship with other teacher
self-report scales. Teacher self-efficacy is a
multidimensional construct which comprises teachers’ perceived
ability to perform various aspects of teaching (Skaalvik &
Skaalvik, 2007; Tschannen-Moran & Woolfolk Hoy, 2001).
Theoretically, self-assessed usage of differentiation as
measured by the DSAQ should be more closely related to aspects
of self-efficacy related to differentiation than to other
aspects of teacher self-efficacy. Specifically, we expected
stronger correlations with scales that measure teachers’
self-efficacy for instruction to students of diverse
achievement levels, for adapting education to individual
students’ needs, and for self-assessed prerequisite knowledge
for differentiation (which would support convergent validity)
than with scales that measure teachers’ self-efficacy for
motivating students, for coping with changes and challenges,
and for classroom management (which would support divergent
validity).
The third
and main aim was to investigate teachers’ self-assessed use of
the strategies for differentiation recommended by the experts.
Besides examining teachers’ overall usage, we
also aimed to identify strategies which were relatively
infrequently used. Such information may provide starting
points for teacher professional development by indicating in
which areas teachers perceive most room for improvement. Based
on the literature reviewed in the theoretical background as well as
on the input from the experts in Study 1,
we hypothesised that average scores would be low to moderate
and that specialised
strategies aimed at individual students’ unique educational
needs as well as strategies targeted specifically at
high-achieving students would be used relatively infrequently.
3.2
Method Study 2
3.2.1
Participants
and procedure
The sample
consisted of 268 primary school teachers working at 31 schools
participating in a large-scale project about differentiation.
Schools were informed about the project through flyers and
advertisements and could register themselves for participation
on a project website. The schools were located in rural and
urban areas spread across the Netherlands and were diverse in
terms of school size, student population, religious
background, and mathematics curriculum used. All 325 teachers
of Grade 1 through 6 of the participating schools were invited
by email to fill out an online questionnaire containing the
DSAQ and related scales. The
questionnaire
was administered at the beginning of the 2012 - 2013 school
year. A total of 268
teachers (83%) completed the questionnaire and gave informed
consent. The remaining teachers did not give informed consent
(n = 3), completed
the questionnaire only partly (n = 7) or did not
respond at all (n =
47). On average, participants had 15.6 years of teaching
experience (range 0 – 40 years). Seventy-one teachers (26%)
taught a
multigrade class. Fifty-four teachers (20%) worked
full-time, whereas most teachers worked two, three or four
days a week (61, 81 and 55 teachers respectively).
3.2.2
Instruments
The DSAQ was
developed to examine how teachers assess their use of the
strategies recommended by the experts in Study 1. Each
subscale represents one step of the cycle of differentiation
and covers core strategies for differentiation
belonging to that step. Subscales
and sample items are provided in Table 2. Organisational
aspects of the model for differentiation were not captured in
a separate scale but were partly covered in the subscales
corresponding to each step of the cycle. In an earlier pilot
study, a pilot version of the DSAQ had been completed by 27
teachers recruited at four schools. Based on the analysis of
the internal consistency of the pilot version, which was
acceptable to good, some adaptations were made in the final
version of the DSAQ. The internal consistency of the final
version, obtained in the current sample (N = 268), is reported
in section 3.3.1.
To
assess the convergent and divergent validity of the DSAQ,
subscales from two well-established multidimensional teacher
self-efficacy scales were selected. The Norwegian Teacher
Self-Efficacy Scale (Skaalvik & Skaalvik, 2007) was
developed with special attention for adapting education to
individual educational needs. It consists of six subscales
with acceptable to good internal consistency which load on six
primary factors, which in turn load on a second-order factor
for general teacher self-efficacy (Skaalvik & Skaalvik,
2007). For the current study, the subscales for Instruction -
which emphasises instruction to students of diverse
achievement levels - and
Adapting
Education to Individual Students’ Needs[1]
were selected to assess the convergent validity of the DSAQ
while the subscales for Motivating Students and for Coping
with Changes and Challenges were selected to assess the
divergent validity. The subscales were translated into Dutch
and the wording was adapted to make the items domain-specific
for mathematics instruction.
To
further examine the divergent validity, the subscale for
classroom management from the well-established Ohio State
Teaching Efficacy Scale (OSTES; Tschannen-Moran & Woolfolk
Hoy, 2001) was administered in Dutch translation (Goei,
Bekebrede, & Bosma, 2011). The OSTES consists of three
subscales which load on a first-order factor and on a general
teaching efficacy factor. The subscales have demonstrated high
internal consistency and can be used independently with
in-service teachers (Tschannen-Moran & Woolfolk Hoy,
2001).
As
a third potential support for convergent validity, a
self-assessment scale about Prerequisite Knowledge for
Differentiation was adapted from an informal scale that had
already been used to assess the level of prior knowledge in
professional development programmes (Nationaal
Expertisecentrum Leerplanontwikkeling, 2010). Teachers
self-assess the extent to which they already possess the
knowledge necessary for implementing differentiated
instruction.
3.2.3
Analyses
Because
we wanted to compare the fit of two specific models based on
theory and previous findings, we used confirmatory factor
analysis (CFA) to investigate
the factor structure of the DSAQ. We first tested a one-factor
model in which all DSAQ subscales loaded on one general
differentiation factor. Second, we tested a two-factor model
with one factor for progress monitoring (subscales
Identification of Educational Needs and Evaluation of Progress
and Process) and one factor for instructional adaptations
(subscales Differentiated Goals, Differentiated Instruction
and Differentiated Practice). Version 7.3 of the Mplus
statistical package (Muthén & Muthén, 1998-2012) was used.
Model fit was evaluated with the chi-square statistic, the
comparative fit index (CFI), the Tucker-Lewis Index (TLI), the
root mean squared error of approximation (RMSEA), and the
standardised root mean square residual (SRMR). Values above
.95 for the CFI and TLI and values below .06 and .08 for the
RMSEA and SRMR, respectively, indicate good model fit (Hu
& Bentler, 1999). The maximum likelihood estimator was
used. In the standardised solution, the variance of the
factors was fixed to 1 so all factor loadings could be
estimated freely.
Correlational
analyses were performed to assess the convergent and divergent
validity of the DSAQ. We expected moderate to strong positive
correlations with Prerequisite Knowledge for Differentiation,
Adapting Education to Individual Students’ Needs, and
Instruction (for students of all achievement levels). For the
latter two scales, we expected that correlations would be
lower for the factor Progress Monitoring than for the factor
Instructional Adaptations, because Progress Monitoring does
not focus on the instructional phase. Regarding divergent
validity, we hypothesised that DSAQ scores would be less
strongly related to self-efficacy for Motivating Students,
Coping with Changes and Challenges, and Classroom Management,
although we still expected positive correlations because these
dimensions of teacher self-efficacy can be helpful when
implementing differentiation.
To identify
areas of relatively low use, we compared the means of all
single items to the mean of their factor. If a mean was more
than one standard deviation below the mean of the factor, it
was classified as relatively low.
Table 2
Sample items and descriptive statistics (N =
268) of the administered scales
|
Scale |
Sample item |
Response options |
No. of items |
α |
M |
SD |
|
DSAQ |
|
|
|
|
|
|
|
Identification
of educational needsa |
I
analyse the answers on curriculum-based tests to
assess a student’s educational needs |
1 = does
not apply to me at all, 5 = fully applies to me |
5 |
.69 |
3.64 |
.55 |
|
Differentiated
goalsa |
I set
extra challenging goals for high-achieving students |
1 – 5 as
above |
6 |
.79 |
3.78 |
.55 |
|
Differentiated
instructiona |
I adapt
the level of abstraction of my instruction to the
educational needs of the students |
1 – 5 as
above |
7 |
.72 |
3.81 |
.42 |
|
Differentiated
practicea |
I select
the most important elaboration activities for very
low-achieving students |
1 – 5 as
above |
8 |
.72 |
3.46 |
.55 |
|
Evaluation
of progress and processa |
I use
diagnostic conversations to evaluate whether specific
students have met the lesson goals |
1 – 5 as
above |
7 |
.86 |
3.56 |
.57 |
|
Additional scales |
|
|
|
|
|
|
|
Instructionb
|
How
certain are you that you can explain central themes in
mathematics so that even the low-achieving students
understand? |
1 = not
certain at all, 4 = absolutely certain |
4 |
.74 |
3.13 |
.37 |
|
Adapting
education to individual students’ needsb |
How
certain are you that you can adapt instruction to the
needs of low-achieving students while you also attend
to the needs of other students in class? |
1 - 4 as
above |
4 |
.78 |
2.91 |
.44 |
|
Coping
with changes and challengesb |
How
certain are you that you can manage instruction
regardless of how it is organised (working with
subgroups, multigrade classes with 3 grades, etc.)? |
1 - 4 as
above |
4 |
.76 |
2.99 |
.43 |
|
Motivating
studentsb |
How
certain are you that you can get students to do their
best even when working with difficult problems? |
1 - 4 as
above |
4 |
.76 |
2.95 |
.42 |
|
Classroom
managementc |
How much
can you do to control disruptive behavior in the
classroom? |
1 =
nothing, 9 = very much |
7 |
.92 |
7.17 |
.77 |
|
Prerequisite
knowledge for differentiationd |
I know
the different solution strategies that are used by
children |
1 = does
not apply to me at all, 5 = fully applies to me |
10 |
.84 |
3.79 |
.41 |
a Newly deloped DSAQ-scales b Adapted
from Skaalvik and Skaalvik (2007) c Taken from Goei, Bekebrede, and
Bosma (2011) d Adapted from Nationaal
Expertisecentrum Leerplanontwikkeling (2010)
3.3
Results Study 2
3.3.1
Properties
of the DSAQ: internal consistency and factor structure
As
reported in Table 2, the internal consistencies of the DSAQ
subscales were acceptable to good (Streiner, 2003).
The
results of the confirmatory factor analysis indicate that the
one-factor model in which all five subscales loaded on a
general differentiation factor did not fit the data well: χ2(5)
= 55.126, p <
.001; RMSEA = .193 (90% CI .149 - .241); CFI = .912; TLI =
.824; SRMR = .050. The two-factor model had a good fit: χ2(4)
= 5.637, p = .228;
RMSEA = .039 (90% CI .000 - .107); CFI = .997; TLI = .993;
SRMR = .017. For the factor Progress Monitoring, standardised
factor loadings were .84 (SE = 0.03, R2 = .71)
for Identification of Educational Needs and .85 (SE = 0.03, R2 = .73)
for Evaluation of Progress and Process. For the factor
Instructional Adaptations, standardised factor loadings were
.77 (SE = .04, R2 = .59)
for Differentiated Goals, .75 (SE = .04, R2 = .56)
for Differentiated Instruction, and .74 (SE = .04, R2 = .54)
for Differentiated Practice (p < .001 for all
factor loadings). The correlation between the factors was .78
(p < .001). Since
the two-factor model provided a better fit, the two factor
scores (average of the subscale scores comprising that factor)
were used in subsequent analyses.
3.3.2
Convergent
and divergent validity: correlations with other scales
The
correlations between the two DSAQ factors and related scales
are reported in Table 3. In support of convergent validity,
the correlation with Prerequisite Knowledge for
Differentiation was strong for both factors. As hypothesised,
correlations with self-efficacy for Instruction and
self-efficacy for Adapting Education to Individual Students’
Needs were moderate to strong for the factor Instructional
Adaptations and somewhat lower for Progress Monitoring.
Regarding divergent validity, the correlations with Motivating
Students and Classroom Management were less strong, although
still in the moderate range. Contrary to expectations, Coping
with Changes and Challenges correlated strongly with the
factor Instructional Adaptations.
Table 3
Correlations (p < .001) between DSAQ factor
scores and related scales
|
Scale |
Progress Monitoring |
Instructional Adaptations |
||
|
|
r |
95% CI |
r |
95% CI |
|
Selected
for convergent validity |
|
|
|
|
|
Prerequisite knowledge for
differentiation |
.62 |
.54 - .68 |
.70 |
.64 - .76 |
|
Instruction (to students of all
achievement levels) |
.40 |
.30 - .49 |
.47 |
.37 - .56 |
|
Adapting education to individual
students’ needs |
.38 |
.28 - .48 |
.56 |
.48 - .64 |
|
Selected
for divergent validity |
|
|
|
|
|
Motivating students |
.30 |
.19 - .40 |
.37 |
.27 - .47 |
|
Classroom management |
.34 |
.23 - .44 |
.40 |
.30 - .50 |
|
Coping with changes and challenges |
.42 |
.31 - .52 |
.58 |
.49 - .65 |
3.3.3
Distribution
of DSAQ scores: mean scores and infrequently reported
strategies
The mean
factor scores were 3.60 (SD
= 0.52) for Progress Monitoring and 3.68 (SD = 0.43) for
Instructional Adaptations. With a range from 1.83 to 4.86 for
Progress Monitoring and from 2.45 to 4.94 for Instructional
Adaptations, the factor scores were normally distributed at
the high end of the scale. Table 2 provides the means and
standard deviations of all subscales. Taken together, the
mean factor and subscale scores reflect moderate to high
self-assessed use of differentiation strategies.
Table 4
provides the means and standard deviations for each item of
the DSAQ. Five items - numbers 3.7, 4.2, 4.4, 4.8, and 5.5 -
had a mean score at least one standard deviation below the
mean of their factor. Two of these - the use of diagnostic
conversations to evaluate whether the learning goals have been
met and the adaptation of type of practice to students’ needs
- reflect specialised strategies
because they involve the refined diagnosis of and adaptation
to individual students’ needs. Other specialised strategies
(items 1.5 and 5.7) also had somewhat lower means, although
these means were within one standard deviation of the factor
mean. The three remaining infrequently reported items
concerned adaptations for high-achieving students, namely additional on-level instruction or
guidance, curriculum compacting, and the use of computer
programmes for additional
challenge. Nevertheless, two other strategies targeted at
high-achieving students (items 2.5 and 4.5) were frequently
reported.
Table 4
Means and standard deviations of DSAQ items
(scale range 1 - 5)
|
DSAQ item |
M |
SD |
|
Subscale
1: Identification of educational needs |
|
|
|
1.1 I analyse the
answers on curriculum-based tests to assess a
student’s educational needs |
4.02 |
0.77 |
|
1.2 I analyse the
answers on standardised tests to assess a student’s
educational needs |
3.49 |
0.91 |
|
1.3 I assess
specific students’ educational needs based on daily
maths work |
3.75 |
0.72 |
|
1.4 I assess specific students’
educational needs based on (informal) observations
during the maths lesson |
3.76 |
0.77 |
|
1.5 If necessary, I conduct diagnostic
conversations to analyse the educational needs of
specific students |
3.20 |
0.90 |
|
Subscale
2: Differentiated goals |
|
|
|
2.1 I set different goals for the
children, dependent on their achievement level |
3.62 |
0.79 |
|
2.2 I set extra challenging goals for
high-achieving students |
3.57 |
0.83 |
|
2.3 I set well-considered minimum
goals for very low-achieving students |
3.75 |
0.76 |
|
2.4 I know the opportunities for
differentiation offered by the curriculum |
4.03 |
0.68 |
|
2.5 I use the opportunities the
curriculum offers for differentiation for
high-achieving students |
3.88 |
0.84 |
|
2.6 I use the opportunities the
curriculum offers for differentiation for
low-achieving students |
3.83 |
0.82 |
|
Subscale
3: Differentiated instruction |
|
|
|
3.1 I adapt the level of abstraction
of instruction to the needs of the students |
3.95 |
0.55 |
|
3.2 I adapt the modality of
instruction (visual, verbal, manipulative) to the
needs of the students |
3.82 |
0.62 |
|
3.3 I adapt the pace of instruction to
the needs of the students |
3.95 |
0.56 |
|
3.4 I deliberately ask open-ended
questions during whole-class instruction |
3.82 |
0.67 |
|
3.5 I deliberately ask questions at
various difficulty levels during whole-class
instruction |
3.69 |
0.73 |
|
3.6 I regularly
provide low-achieving children with additional
instruction (extended instruction, pre-teaching) |
4.25 |
0.64 |
|
3.7 I regularly provide high-achieving
students with additional instruction or guidance at
their level, in a group or individually |
3.20 |
0.92 |
|
|
|
|
|
|
|
|
|
Table 4 (continued) |
|
|
|
DSAQ item |
M |
SD |
|
Subscale
4: Differentiated practice |
|
|
|
4.1 I vary different
types of practice during the maths lesson (e.g.
individual or group work, solution spoken, written or
drawn) |
3.53 |
0.78 |
|
4.2 I adjust
different types of practice to the needs of the
students in the classroom (e.g. having a specific
child complete exercises on the computer because this
child learns more in this way) |
3.04 |
0.83 |
|
4.3 I select the
most important tasks for very low-achieving students |
3.73 |
0.73 |
|
4.4 I use curriculum compacting
for high-achieving students |
3.20 |
1.25 |
|
4.5 I provide high-achieving
students with enrichment tasks |
4.00 |
0.87 |
|
4.6 I also use
computer programmes or maths websites in my maths
lessons |
3.68 |
0.97 |
|
4.7 I use computer programmes
and/or maths websites to offer children focused
practice in a skill that they do not sufficiently
master |
3.32 |
0.96 |
|
4.8 I use computer
programmes and/or maths websites to offer specific
children additional challenge in the maths lesson |
3.15 |
1.05 |
|
Subscale
5: Evaluation of progress and process |
|
|
|
5.1 I use scores on standardised and
curriculum-based tests to evaluate whether the
learning goals have been met |
4.04 |
0.73 |
|
5.2 I analyse the answers on
curriculum-based tests to evaluate whether the
learning goals of that unit have been met |
4.06 |
0.72 |
|
5.3 I regularly evaluate whether all
students have met the learning goals based on their
daily maths work |
3.75 |
0.85 |
|
5.4 I evaluate whether all students have
met the lesson goals based on (informal) observations
during the maths lesson |
3.45 |
0.86 |
|
5.5 I conduct diagnostic conversations to
evaluate whether specific students have met the lesson
goals |
2.85 |
0.87 |
|
5.6 I evaluate whether the type of
instruction and practice chosen by me were effective
for the majority of the students in the class |
3.44 |
0.77 |
|
5.7 I evaluate whether a specific type of
instruction was effective for specific students |
3.32 |
0.80 |
3.4
Concluding summary Study 2
Study 2
investigated teachers’ self-assessed implementation of
differentiation using the DSAQ. The first goal was to examine
the psychometric properties of the DSAQ. The subscales of the
DSAQ were internally consistent and loaded on two correlated
but distinct factors: Progress Monitoring and
Instructional Adaptations.
Confirmatory factor analysis demonstrated that this two-factor
structure provided a better fit than a one-factor model, which
converges with the findings reported by Roy et al. (2013). The
second goal was to examine the convergent and divergent
validity of the DSAQ. The pattern of correlations between the
DSAQ and other scales supported its convergent and divergent
validity. As expected, strong to moderate correlations with
Prerequisite Knowledge for Differentiation, Adapting Education
to Individual Students’ Needs, and Instruction were found. As
hypothesised, the correlations with the scales selected for
testing the divergent validity were lower, except for the
correlation with Coping with Changes and Challenges which was
unexpectedly strong.
The third
and main goal was to examine teachers’ perceived usage of the
strategies recommended by the experts. With factor means in
the moderate to high range, teachers assessed their use of
differentiation strategies more highly than we had expected.
Five items with relatively low means were identified. In
support of our hypothesis, these items concerned specialised
strategies and strategies targeted at high-achieving students.
4. General
discussion
Teachers
are required to implement differentiation for students of
diverse achievement levels. However, the term differentiation
had been used in diverse ways and the literature did not
provide sufficient information regarding the most effective
strategies to provide teachers with general guidelines for
implementing differentiation. To fill this gap, Study 1
operationalised the
concept of differentiation by achieving consensus among a
consortium of experts about a model and strategies for
differentiation in primary school mathematics. Study 2
investigated the degree to which Dutch teachers already
implement the strategies suggested by the experts.
Study
1 resulted in a model for differentiation consisting of five
steps: identification of educational needs, differentiated
goals, differentiated instruction, differentiated practice,
and evaluation of progress and process. These steps reflect
the two core components of differentiated instruction
identified by Roy et al. (2013). Progress monitoring is
captured by the steps of identification of educational needs
and evaluation of progress and process. The component of
instructional adaptations is represented by the steps of
differentiated goals, instruction, and practice. Study 2
demonstrated that a two-factor model in which the subscales of
the DSAQ load on these two factors provides a better fit than
a one-factor model. Our findings converge with the findings
reported by Roy et al. (2013), supporting the idea that
progress monitoring and instructional adaptations are two
distinct but related components of differentiation.
New
in this study is expert consensus on how progress should
be monitored and how
goals, instruction and practice should be adapted to the
learning needs of students with diverse achievement levels.
Regarding progress monitoring, the experts recommended to use
standardised and curriculum-based tests first to divide
students over achievement groups. More refined and informal
measures such as the analysis of daily work should be used
frequently to monitor short-term progress, to diagnose unique
educational needs, and to determine whether a (temporary)
adjustment of the groups is necessary. Compared to
technological applications which tend to make use of one or
two types of assessment to monitor progress (e.g. McDonald
Connor et al., 2009; Ysseldyke & Tardrew, 2007), the
experts recommended a broader range of strategies and
indicated how they can be used together. The strategies have
complementary purposes: while relatively formal and
standardised tests are useful to get an overview of what a
student can do, more informal and qualitative measures such as
diagostic conversations and the analysis of daily work provide
valuable information about why a student struggles with a
certain problem and what the student needs.
The
use of within-class homogeneous achievement groups provides
the opportunity to tailor subgroup instruction to similar
educational needs and has demonstrated positive effects (Kulik
& Kulik, 1992; Lou et al., 1996; Slavin, 1987; Tieso,
2005). In line with Slavin (1987), the experts stressed the
importance of flexibility, i.e. allowing students to switch
between groups based on changes in their educational needs.
The literature indicates that the effects of within-class
ability grouping may depend upon student achievement level,
with smaller or even negative effects for low-achieving
students (Deunk et al., 2015; Lou et al., 1996). Nevertheless,
the experts clearly perceived small-group instruction as a
good way to provide low-achieving students with the
instruction they specifically need. Also, students are only
grouped for part of the lesson and participate in the
whole-class instruction for students of all ability levels as
well. Future research should establish whether these
conditions ensure that low-achieving students also profit from
this type of within-class flexible ability grouping.
Regarding
instructional adaptations, the experts recommended a coherent
set of strategies to differentiate goals, instruction and
practice. This comprehensive approach is somewhat broader than
technology-based interventions which have tended to focus on
differentiation of either instruction (Individualizing Student
Instruction) or practice (Accelerated Math). Many of the
strategies recommended by the experts are supported by
previous research, including the adaptation of practice tasks
to the skill level of the student (Ysseldyke & Tardrew,
2007), the use of explicit instruction and visual
representations for low-achieving students (Gersten et al.,
2009) and the use of compacting, enrichment, and instruction
at challenge level for advanced students (Rogers, 2007). To
use teachers’ time efficiently, the experts recommended to
teach the whole class when possible, to use subgroups when the
diverse educational needs of subgroups require this, and to
serve remaining unique educational needs individually. Thus,
the experts recommended both universal supports (supports for
all students such as varying the difficulty level of questions
in broad whole-class instruction) and targeted supports
(supports specifically for low-achieving and high-achieving
students including small-group instruction and differentiation
in practice tasks). The experts also recommended some
adaptations to individual students’ educational needs (e.g.
the adaptation of type of practice to the preference of
specific students), but they realised that such specialised
adaptations were advanced and primarily suitable for teachers
who already master basic strategies for differentiation.
To link the advice
provided by the experts to teachers’ daily practice, Study 2
investigated teachers’ self-reported usage of the recommended
strategies. Overall, DSAQ scores were moderate to high,
exceeding the expectations we had based on previous studies.
Perhaps, the different context (primary schools in the
Netherlands versus middle schools in the United States) can
explain the discrepancy with the low use of differentiation
strategies reported by Moon et al. (2002). Our findings are
more similar to those of a recent study with Canadian primary
school teachers in which moderate usage was reported (Roy et
al., 2013). Nevertheless, the moderately high self-assessments
in the current study seem discrepant with the finding of the
Dutch Inspectorate of Education that adequate adaptations to
students’ diverse educational needs are only made at about
half of the schools (Van den Broek-d’Obrenan et al., 2012).
Also, the experts in Study 1 clearly perceived a need for
professional development about differentiation. Perhaps, the
inspectors of education and the experts from our consortium
have high standards for the quality of
implementation which are not captured by the DSAQ. Teachers
might also overestimate their implementation. Refined
observational studies are necessary to examine whether
teachers’ high self-assessed usage of differentiation
strategies can be confirmed by external observers.
In
line with previous studies (McLeskey & Waldron, 2002,
2011; Reis et al., 2004; Scott et al., 1998; Westberg et al.,
1993; Westberg & Daoust, 2003), specialised studies and
strategies targeted at high-achieving students were used
relatively infrequently. Two specialised strategies - the use of diagnostic
conversations to evaluate whether the learning goals have been
met and adaptation of the type of practice to specific
students’ needs - were relatively infrequently reported. This
corresponds with the view expressed by the experts that
individual-level differentiation is advanced and primarily
suitable for teachers who already implement group-based
strategies for differentiation successfully (Van Groenestijn,
Borghouts, & Janssen, 2011).
Three
strategies targeted at high-achieving students - curriculum compacting, the use of computer programmes for additional challenge, and targeted
instruction for these students - were used infrequently. The
difference between the use of instruction targeted at
high-achieving students versus low-achieving students is
especially striking. Perhaps, teachers are not aware that high-achieving
students
also need guidance when working on sufficiently challenging
enrichment tasks (VanTassel-Baska & Stambaugh, 2005). Many teachers do implement some
differentiation in practice tasks. However,
there is still a lot of room for improvement, since it seems
that only few teachers use a complete approach including
challenging goals, curriculum compacting, enrichment tasks and
on-level guidance. Low usage of differentiation for
high-achieving students has repeatedly been attributed to a
lack of the specific attitudes, knowledge, and skills this
requires (Latz, Speirs Neumeister, Adams, & Pierce, 2009;
Megay-Nespoli, 2001; VanTassel-Baska & Stambaugh, 2005).
Many teacher educators feel that initial teacher training does
not adequately prepare teachers to differentiate instruction
for high-achieving students (Schram, Van der Meer, & Van
Os, 2013). Thus, it seems important that this topic receives
sufficient attention in teacher training and professional
development programmes.
The
following limitations should be taken into account. First, the
results of consensus procedures are inherently restricted by
the participating experts. The risk that other experts might
have provided different input cannot be eliminated but was
diminished in this study by recruiting experts working for
several different institutions for both pre-service and
in-service teacher training. Second, some experts missed some
components of the procedure (a focus group discussion or a
round of the Delphi questionnaire). This limitation was
compensated for by the repetitive nature of the procedure:
participants who missed one component could still provide
comments and additional input in the subsequent component.
Third, it is possible that teachers provided socially
desirable answers since a self-report questionnaire was used
in Study 2. The variability between the items provides an
indication that teachers did not simply rate themselves highly
on all items to create a favourable impression. Nevertheless,
we state again that observational studies are necessary to
investigate how self-reported use is related to observed use
of differentiation strategies. Fourth, it is unknown whether
non-responders differed from teachers who did respond to the
questionnaire, although the response rate of 83% is quite
good.
a
strong combination of methodologies was used. Study 1 employed
an innovative methodology which combined the advantages of two
methods: focus group discussions are suitable for creating
shared understanding and generating ideas, while the Delphi
procedure gives all participants an equal and anonymous say in
the systematic evaluation of those ideas. This combination was
fruitful and efficient and we recommend it for future
research. Moreover, the collaboration with experts who were
familiar with the daily practice of teaching enhanced the
feasibility of the findings. Based on their experience in
various settings, the experts perceived these strategies as
effective and feasible. In addition to this expert
perspective, Study 2 examined the results of Study 1 from a
teacher perspective. The fact that the teachers in Study 2
reported to use most of the strategies recommended by the
experts in Study 1 shows that teachers acknowledge the need to
differentiate and that the recommended strategies are largely
compatible with teachers’ daily practice. At the same time,
the discrepancy between the teachers’ and experts’ perception
of the degree of application of differentiation opens up an
interesting avenue for future research. Thus, the inclusion of
two complementary sources of information provides a richer
perspective on differentiation. Despite these methodological
advantages, future research is necessary to test empirically
whether the implementation of the strategies recommended by
the experts leads to higher student achievement.
The
results of the current studies contribute to scientific
research as well as to educational practice. Although the
experts in Study 1 departed from a practical rather than a
theoretical perspective, the elements of the cycle of
differentiation overlap with elements of more general
didactical models, such as Van Gelder’s didactical analysis
model (Van Gelder, Oudkerk Pool, Peters, & Sixma, 1973)
and De Corte’s didaxology (De Corte, Geerligs, Lagerweij,
Peters, & Vandenberghe, 1976). Apparently, effective
readiness-based differentiation is consistent with the
principles of general good teaching, with the addition that
each stage of teaching needs to be differentiated. A first theoretical
implication is that, rather than studying specific elements
(i.e. differentiated
practice) in isolation, it seems promising to move towards an
integral view of differentiation which involves all stages of
teaching. Second, the experts clearly endorsed the view that
students of different achievement levels have different
educational needs and need different treatments at least part
of the time, echoing the aptitude-treatment interaction
literature (Cronbach & Snow, 1977). This emphasises the
need to consider the potential variation between students in
the design and analysis of educational intervention studies:
what works for high-achieving students, may not work for
low-achieving students and vice versa.
At
the practical level, the model and strategies for
differentiation recommended by the experts can be used in
teacher training and professional development, for which
purpose they have also been published in a Dutch journal for
practitioners (Van de Weijer-Bergsma & Prast, 2013).
Current educational policies require teachers to implement
differentiation and our results provide teachers with concrete
advice on how to do this. The cycle of differentiation can be
used as a framework to structure professional development
about differentiation. It shows teachers that differentiation
requires attention at all stages of teaching in one coherent
approach. The recommended strategies provide teachers with
practical suggestions for each step (these can be found in
section 2.3, Appendix 1, and also in the DSAQ-items listed in
Table 4). The focus on mathematics promoted the concreteness
of the results, since domain-specific guidelines can be
applied directly without the need to transfer general
principles. For example, the general guideline that advanced
learners should be adequately challenged was made ready for
use by providing achievement criteria for selecting
high-performing students, suggestions for increasing task
difficulty, guidelines for compacting, and a list of
supplemental enrichment curricula. Nonetheless, the principles
behind these concrete recommendations, including the cycle of
differentiation, seem to be applicable in other domains as
well. Future research could examine whether and how our
results extend to other domains.
Study
2 provides researchers and practitioners with a new tool, the
DSAQ. Researchers can use it, for example, as a pre- and
post-assessment in intervention studies or to investigate
teachers’ self-assessed implementation in other countries. In
professional development, the DSAQ can inform
trainers about areas in which teachers perceive most room for
improvement. Theoretically, Study 2 builds on the existing
literature by providing support for the two-dimensional
structure of differentiation. Moreover, this study is the
first to investigate the self-reported use of a broad range of
strategies for differentiation in mathematics in the
Netherlands. The identified areas of low usage have practical
implications, including the need to pay sufficient attention
to differentiation for high-achieving students in teacher
training and professional development.
Keypoints
Effective combination of the Delphi technique and
focus group discussions to achieve consensus among experts
Use of experts’ practical knowledge to enhance
feasibility of the findings
A model and strategies for implementing
differentiation in primary school mathematics, usable in
teacher professional development
A new questionnaire (the DSAQ) to measure
teachers’ self-assessed implementation of differentiation
strategies, usable in future research
Teacher self-assessment indicating moderate to
high usage and identification of relatively infrequently used
strategies
Acknowledgements
This work is part of the research programme ‘Every
child deserves differentiated mathematics education’, which is financed by the Netherlands
Organisation for Scientific Research (NWO), grant number 411-10-753.
The
NWO was not involved in the collection, analysis,
interpretation, or reporting of the data.
We
thank the consortium members for their valuable input during
the consensus procedure. We are also grateful to the teachers
who completed the questionnaire. Finally, we thank the
anonymous reviewers for their useful comments on a previous
version of this article.