Equity-Efficiency-Effectiveness through Assessment Levers - 1-23-15
1. 1
FOSTERING DEEPER LEARNING WITH EQUITY, EFFICIENCY, AND EFFECTIVENESS (E3
): THE
ESSENTIAL ROLE OF ASSESSMENT
Peter Hofman
January 2015
Summary
A cacophony of voices is clamoring for changes to our
assessment and accountability system to address myriad
shortcomings and a growing desire for deeper learning and
college, career, and life readiness for all students. Especially
at a time when complaints of too much testing are
mushrooming, how could anyone seriously suggest that
assessment can play an essential, positive role in achieving
this goal? In fact, compelling evidence demonstrates that four
assessment-related levers are not only essential in these
efforts, but also in improving educational efficiency and
effectiveness: accessibilityto assessment content,improved
assessment literacy, use of formative assessment practices,
and curriculum-embedded performance assessment.
Despite progress in all other areas, we will remain stuck
without fundamental changes in assessment practices – in
classrooms and for accountability. Why? Because without
these changes we cannot constructively change how students
and teachers spend their time, the key determinant of
education outcomes and how efficiently and effectively we
realize them. Such changes cannot occur spontaneously or in
isolation, but rather require strong leadership and a high-
reliability and responsive system.
Accessibility: ensuring all instructional and assessment
materials are accessible to all students is a fundamental,
moral obligation, but also a practical necessity. It
eliminates barriers to learning and levels the playing field
so all students have equal opportunities to learn, to
demonstrate what they know and can do, and to get
needed support.
Assessment literacy: although traditionally ignored or
superficially treated, the knowledge and skills to
effectively gather and use evidence of student learning
has been proven essential to achieving high outcomes for
all students. It also ensures the generation of data useful
for efficiently and effectively evaluating and adjusting
instruction and curricula. In fact, the widespread lack of
assessment literacy likely causes over-testing.
Formative assessment practices: individually and
collectively, these instructionally embedded practices
have been researched more than any other assessment
approaches and have been proven to improve learning
for all students (as much as one-on-one tutoring and
more than reducing class size), with the greatest impact
on lower-performing ones, thereby closing achievement
gaps.
Curriculum-embedded performance assessment: a
growing body of research and other evidence
demonstrates that performance assessment can promote
engagement and deeper learning for all students, build
teacher capacity, and help shrink achievement gaps.
Despite a greater up-front investment of time and effort,
when curriculum-embedded it enriches instruction and
improves efficiency and effectiveness by providing direct
evidence of what students know and can do. Yet, despite
its far-reaching and dramatic benefits, curriculum-
embedded performance assessment is far more likely to
be implemented if accountability measures become more
performance-based, especially if they include curriculum-
embedded tasks.
In light of the power offered by each lever and although
none are quick fixes, examples of successful implementation
of one or more of them exist around the country. However,
these examples generally do not encompass all four levers
and/or tend to be at a small scale and in relative isolation.
Implementing the system as a research-based pilot at the
district level will demonstrate the potential and lay the
foundation for scaling up the system along with other needed
reforms.
Implementing these four levers at the district level should
require no external policy changes, although it will likely call
for internal changes in policy and especially practice. The
greatest investment of time and effort will be in building
educator capacity and supporting continuous collaboration
and improvement. The increased efficiencies and effectiveness
should produce savings that will ultimately more than cover
the transition and ongoing investment. And the benefits to
students and staff will be priceless!
Deeper
Learning with
Equity,
Efficiency,
Effectiveness
(E3)
Assessment
Literacy
Accessibility
Formative
Assessment
Curriculum-
Embedded
Performance
Assessments
(CEPAs)
2. 2
Discussion
Achieving equity in educational opportunities and high
outcomes is a moral, legal/civil rights, and economic
imperative. Success requires multiple education reforms as
well as efforts to address out-of-school factors. Unfortunately,
making fundamental changes in assessment practices is rarely
included in these reforms. Such changes are, however,
essential, not simply because of their effects on equity and
student outcomes in general, but because how significantly
they can improve educational efficiency and effectiveness.
Accessibility: It seems obvious that all students must be
able to access all instructional and assessment materials
to have a full opportunity to learn and to demonstrate
what they know and can do. Sadly, for far too many
students, this is not their reality. And despite provisions
in IDEA covering instructional materials, even here many
are not accessible to all students. The situation is far
worse with assessment content, particularly local
assessments. State accountability tests tend to be more
accessible. But if assessments used during the school
year aren’t accessible, the year-end tests will merely
confirm that these students didn’t learn or grow as
much as they could or should have.
Attention devoted to accessibility typically focuses on
students coded with disabilities under IDEA
and English language learners. But far more
students have a range of accessibility needs
that aren’t been met – they might not even
be on the radar screen. Thus, we fall
egregiously short in terms of serving all
students.
Access to all assessment content is critical for
equity, efficiency, and effectiveness. If
students can’t access instructional materials,
they can’t learn. If they can’t access
assessment content, they can’t accurately
demonstrate what they know and can do,
depriving teachers of the insights needed for
appropriate interventions, regardless of the
number of tests they give. The lack of
accessibility weaves a web in which students
don’t learn, teachers don’t know what they
need to do to help these students, possibly prompting
them to administer even more tests…wasting time (that
can never be recaptured), effort, and money, with
learning languishing and teachers deemed ineffective.
When educational content is accessible, student
frustration decreases, engagement and learning increase,
and assessments more accurately measure what students
know and can do. The results provide teachers withthe
insights they needto support learning and growth…i.e.,
one simple step to increasing efficiencyand effectiveness.
Opportunities to make instructional and assessment
content more accessible than ever now exist thanks to
research, technology advances, and the increased use of
digital delivery. Newly available supports can help far
more students than are typically identified as needing
them (as many as 35-40% of the school population), not
just students with a wide range of disabilities, but also
ELLs, and even students who are color blind!
Assessment literacy: This term encompasses the skills,
knowledge, and beliefs educators need to identify, select,
or create assessments optimally designed for formative
and summative purposes; and, with a sound
understanding of test quality and comparability
considerations, to analyze, evaluate, and use the
quantitative and qualitative evidence generated by these
assessments to improve instructional approaches and
programs to advance student learning.
Often overlooked due to the myopic focus on external
high-stakes testing is the proven role effective classroom
assessment can play in providing rich and accurate
insights to student learning that inform instructional
interventions fostering growth for all students. Gaining
such insights – through various means, from formative
assessment practices to performance assessment
(discussed below), is particularly important for students
in a sub-group or otherwise at risk or low-performing.
Assessment literate educators are in the best position to
create effective classroom assessments and use the results (as
well as results from external tests) to help all students
succeed. Since we can’t – and don’t want to – “teacher-
proof” education, teachers need all the knowledge and skills
required to be effective. Even if teachers are experts in their
content area,if they are not assessment literate, they will fall
short – and their students will suffer. Those who aren’t
assessment literate can’t be truly effective. Yet typical
educator preparation, certification, and evaluation systems
ignore assessment literacy, with the impact falling most
heavily on subgroup populations.
Assessment
Literacy
Types of Measures:
Classroom formative-
summative
External interim and
summative
Results and their
use:
Feedback, informing
instruction, curriculum, etc.
Scores and grading
Reporting, growth, and
comparability
Quality of Measures:
Match to purpose
Breadth, depth, and
alignment
Universal Design
Item, task, and test quality
Validity, reliability
3. 3
In fact, most educators aren’t assessment literate, which
contributes to the over-abundance and misuse of testing.
These educators just don’t know any better. So, they
create or use ineffective tests, generating incomplete or
misleading information that leads to inappropriate
interventions, which in turn don’t have the desired
effects. The inevitable results are lost time and money,
wasted effort, lack-luster student outcomes, then
probably more tests, and the cycle repeats itself.
Assessment literacy can be taught and mastery achieved,
but it requires policy support and a commitment of time
and effort. The investment pays off in more efficient and
more effective assessment practices and instruction and
improved learning for all students.
Formative assessment, as disseminated by the Council of
Chief State School Officers (CCSSO) in 2007, “is a
process used by teachers and students during instruction
that provides feedback to adjust ongoing teaching and
learning to improve students’ achievement of intended
instructional outcomes.” As previously noted, theoverall
process and its components (listed below) have been
researched more than any other form of assessment and
foundto (1) increase learning by as muchas individual
tutoring and morethan reducing class size,and (2)
improve learning outcomes for all students but more so
for low-performing ones (i.e. close achievement gaps).
To dispel a common misperception, formative
assessment is not just frequent testing. It is an
instructionally embedded process that can be a powerful
lever for achieving equity, efficiency, and effectiveness.
How? It helps students understand what success looks
like and how to get there, so success seems achievable
and they are motivated and empowered to take more
ownership of their learning, such as through self- and
peer-assessment, which as a side benefit leads to fewer
formal tests.
Besides transparency, formative assessment promotes
student growth in a non-threatening manner through
qualitative feedback instead of grading during the
learning process. Thus, even “failures” provide positive
insights for growth rather than labels that sap self-
confidence and discourage effort. The descriptive
feedback helps students and teachers figure out what to
do next, including making possible adjustments to
learning and teaching activities, which is far more
efficient and effective during learning than test scores or
grades.
During the process, teachers use myriad means (far
more than traditional tests) to gauge learning during
instruction, catch misconceptions as they occur, and
promptly address them, which again is far more efficient
and effective than waiting days or weeks to administer
formal tests following instruction. The fact that
formative assessment works, especially with low-
performing students, also saves time, effort, and the
need for more tests, increasing opportunities to focus on
learning.
Formative assessment can be used in all subjects and
and grades, for basic knowledge and deeper learning.
The only significant investment to implement it involves
building teacher capacity, which takes time and efforts,
but the results are well worth it.
Curriculum-embedded performance assessment:
Performance assessment asks students to demonstrate
what they know and can do through some form of
demonstration, product, or presentation, as opposed to
selecting among a limited number of answer options. In
the context of deeper learning, performance assessment
commonly refers to substantive activities – either short-
term, on-demand tasks or curriculum-embedded,
project-based tasks – carried out through a potentially
wide range of activities and yielding reliable and valid
scores. All students need exposure to them and
knowledgeable support to gain the most benefit from
them.
A growing body of research and other evidence
demonstrates that performance assessment offers
significant potential benefits, such as increasing student
engagement; providing rich,relevant,and rigorous
opportunities for students to develop higher-order
cognitive and non-cognitive skills; and helping build
educator capacity. Its use is increasing, especially in
competency-based programs and to measure deeper
learning and college/career readiness.
High-stakes accountability testing dominated by
multiple-choice or other items that focus on basic facts
and skills has deprived students of opportunities to
engage in richer, more relevant, and more rigorous
activities that could stimulate deeper learning. These
opportunities have traditionally not been available to all
students. Indeed, such “enrichment” activities were
reserved for the highest-achieving ones. Yet, recently
published research and other evidence show that when
The power of the formative assessment process – and
the role it could play in educational equity, efficiency,
and effectiveness– is best illustrated by its components.
Teachers ensuring students under-stand the learning
targets and the criteria for success
Teachers gathering rich evidence of student learning
by observation, questioning, quizzes, and more
Teachers, knowledgeable about learning
progressions associated with learning targets,
providing descriptive feedback on gaps in student
learning
Teachers and students using the feedback to adjust
instruction and learning activities
Students engaging in self-assessment and meta-
cognitive reflection
Teachers activating other students as resources
4. 4
all students engage in them, they more than rise to the
challenge – increasing their motivation to learn,
improving performance, in fact thriving, with the
benefits persisting years later.
Designed for all students, curriculum-embedded
performance assessments (CEPAs) provide such
opportunities. CEPAs have much in common with
project-based learning and with various models around
the country that employ performance-based instruction
and assessment.
Especially with degree of choice by teachers and
students, CEPAs provide unique opportunities for all
students to engage in activities relevant and meaningful
to them. They also offer all students scaffolded
opportunities to engage in deeper learning. They can
include group activities, in which any student can play a
role and that provide multiple access points to be
meaningfully involved.
By including formative assessment, CEPAs capitalize on
all the benefits described earlier. The extensive use of
performance-based measures enables teachers to see
student work and thereby obtain direct and immediate
evidence of what students know and can do, which is
critical for informing instruction. This benefit reduces
the need for traditional tests – whether classroom or
external – that are less intertwined with instruction and
learning.
The common belief is that performance assessment takes
too much time to develop, administer, and score to use
on a regular basis. Admittedly, it requires a greater up-
front investment of time and effort than do selected-
response tests, which still dominate the world of
assessment because of their supposed efficiency. Yet,
performance assessment is far more efficient and
effective from the perspective of the entire
learning/teaching process, which is explained by the
“pay-me-now-or-pay-me-more-later” principle.
As noted, performance assessment enables teachers to
see actual student work, so they can readily and
accurately identify and address
misunderstandings/misconceptions or build upon clear
strengths. It can also dramatically improve teacher
practice, as teachers are often surprised when they see
student work responding to particular questions or
assignments. Indeed, over the years the most powerful
professional development has typically involved teachers
evaluating student work, a far more insightful endeavor
than looking at bubbles.
On the other hand, with selected-response items,
teachers really do not know why many students select
correct – or incorrect – response options. What role did
luck or a simple mistake play? These items also severely
limit possible student responses, another barrier to
understanding student learning and thinking. These
factors, combined with such issues as item quality and
number, could place follow-up interventions on shaky
ground and perhaps make them off target and
ineffective, leading to more tests and a repeat of the
cycle. This process takes far more time and effort,
possibly more testing, and might not even uncover what
a performance assessment revealed the first time around.
Without getting too technical, from a measurement
perspective, achieving acceptable reliability requires a
handful of performance-based items versus perhaps 50
multiple-choice ones. And, except in highly unusual
instances, it is extremely difficult to measure higher-
order cognitive skills with selected-response items,
whereas effective performance assessments can readily
do this. The comparison is even more skewed with
regard to non-cognitive skills.
Implementing CEPAs takes time, effort, knowledge, and
skill – including assessment literacy – the biggest
investment being in staff development. The incentive to
take this on is far greater if accountability testing is also
performance-based. Yet, the significant and far-reaching
benefits alone have been sufficient for an increasing
number of schools, districts, and networks to do so.
Now What?
Despite the increased interest in one or more of the levers,
only a tiny fraction of students are benefitting from them. To
achieve educational equity and increase efficiency and
effectiveness on a large scale, we must demonstrate how the
complete package can be implemented by one or more
districts, documenting the context, the process, the lessons
learned, and the impacts on students and staff. Positive
results should attract both policy and funding support for
scaling up.
As noted earlier, the four levers are an essential complement
to other needed reforms. Without them, we will fall short.
Students’ lives hang in the balance. We owe it to them to
act…now.
Curriculum-embedded performance assessments
(CEPAs) are instructional units that include learning
and evidence-gathering activities. They have both
formative and summative purposes and elements,
including formative assessment practices and valid
and reliable evaluations of student work products,
presentations, and other demonstrations of student
learning and capabilities. As such, they are ideal for
helping all students develop the high-order cognitive
and non-cognitive skills embedded in deeper learning
and needed for college, career, and life.