White paper -- Using the Learning Delta to Manage Teaching and Learning Performance May, 2003

1. Data-Driven Decision-Making in K-12
Education: Using the Learning Delta to
Manage for Results
A High Delta Learning  White Paper
Gregory D. Luther
Founder and Principal
High Delta Learning, LLC
May, 2003

2. Page 1
quot;…When you can measure what you are speaking about, and express it in numbers, you
know something about it; but when you cannot measure it, when you cannot express it in
numbers, your knowledge is of a meagre and unsatisfactory kind; it may be the beginning
of knowledge, but you have scarcely in your thoughts advanced to the state of Science,
whatever the matter may be.quot;  Lord Kelvin, 1884
sector. Typically, instead of em- In this paper I make a three-part
Introduction
ploying a quot;generally accepted argument:
The emerging market for K-12 accounting principlequot; for learning
education services is placing cities, measurement, there is a call for 1. More can be done to improve
school districts, and parents into the outside research to assess or validate education by managing the ef-
position of education consumers a program's salutary effects, to be fectiveness of teaching than by
who have a growing need for performed using multivariate statis- any other single factor.
accurate, understandable, and timely tical analysis and experimental
2. Using the Learning Delta to
information on which to base their designs taken from social science
measure learning progress is
choices. Similarly, education phi- and clinical drug trials. In short,
fundamental to managing and
lanthropies have choices to make because schools do not measure
improving the effectiveness of
with regard to which reforms or their own performance, somebody
education reform programs,
innovations to back with their else has tousually at considerable
schools, and teaching generally.
financial resources. In the words of expense and with substantial delays
the Thomas B. Fordham Founda- between the point of data capture
3. In order to implement Learning
tion: and the reporting of results.
Delta Management, classroom
workflows must be digitized in
In this paper I propose the Learn-
Private individuals and organizations…
order to enable the instrumen-
ing Delta as a top-level perform-
can spend their dollars exactly where
tation of teaching and learning
ance metric for schools. Learning
they seek to do the most good. If they
in schools.
Delta is a measure of learning
direct their money, energy, and influ-
progress that captures each stu-
ence toward the right targets. . . their
dent's change in mastery for a given
leverage will help move the system Part 1: Value-Added Testing
period of time. In addition to
itself. The philanthropic sector-America's
and Teacher Effectiveness
improving the manageability of
unique blend of private organizations
individual student progress, Learning There is already a substantial body
with public-minded goals-has the
Delta data can be aggregated to of empirical work from the state of
freedom of action to push on the right
arrive at progress measures for Tennessee that substantiates the
pressure points, and it has clout that
classrooms, entire schools, or school
i
importance and utility of collecting
most parents lack .
systems. Learning Deltas can be data on students' learning progress.
But a critical difficulty facing educa- used to minimize the time students Together with the Tennessee Value-
tion philanthropiesas well as cities, spend waiting for learning to hap- Added Assessment System
school districts, and parentsis that pen. Perhaps most importantly, (TVAAS), Dr. William Sanders, at
schools do not adequately account continuously accumulating and the University of Tennessee Value-
for their primary product: learning. analyzing longitudinal Learning Added Research and Assessment
Measuring and analyzing the impact Delta data for all students represents Center, found that of all the factors
of reforms and innovations on an embedded research process that influencing the observed change in
students' learning progress is a can inform and help to improve student test scores from year to year,
problem because schools do not instruction in every classroom, teacher effectiveness was by far the
employ performance measures and school, and school system that uses most powerful.
management processes like enter- it.
prises outside of the education

3. Page 2
The TVAAS has been in use since progress being made in the class- material satisfactorily by the end of
1991. Every student in grades 2-8 is room. the term.
tested each year in math, reading,
language arts, science, and social
Part 2: Using Learning Delta Data Discipline
studies. Testing of high school
(and An Illustration)
students began in 1995. Each The quot;data disciplinequot; associated with
student’s test results are compared Learning Delta is quite simple.
Learning Delta is given as the
to his or her scores from the previ-
change in a student's mastery of an Step 1 - A quot;pre-testquot; is administered
ous year, and the average score
academic subject divided by the on or near the first day of class to
increases of all the students taught
change in time. Where Sanders' establish the baseline learning level
by each teacher are reported to
of each student.
quot;value-addedquot; is a measure of
administrators for use in evaluations
academic distance, Learning Delta is
• Pre-test questions are drawn
and personnel decisions.
a measure of speed, i.e. distance
from the same set of test ques-
tions that would appear on a final
The headlines from Sanders' find-
exam. A pre-test may also in-
ings in Tennessee include the Figure 2
clude questions that represent
followingii:
prerequisite knowledge and
knowledge that is advanced be-
1. The top quintile of teachers-- Learning Delta:
yond the scope of the course.
ranked according to their stu- the Rate of
dents' average increase in test • A student's pre-test score is a
Learning
scores--raised their students’ measure of the subject knowl-
Progress ∆ Mastery
achievement test scores 39 per- edge they brought into the class
Success
centile points more than teachers with them.
Rate
from the bottom quintile.
• Pre-test resultstagged to each
∆ Time
2. Teachers performing in the two student, to the specific subject
units and skills being examined,
lowest quintiles in Tennessee
failed to produce any achieve- and to the teacher and classare
divided by time, as shown in Figure
recorded and/or stored.
ment gains with most of their 2. The quot;distancequot; term can be
students. thought of as a scale score that is • The difference between a stu-
used in the same way as Sanders dent's pre-test score and the
3. The performance difference
uses it. Time is measured in school score required to demonstrate
between students having three
terms. So if a course lasting one mastery in a subject. is their ini-
consecutive teachers in the top
tial Learning Gap, which also
term is given a value of 100 points,
one-fifth of the ranking and
serves as the Planned Learning
and a student masters half of the
three in the bottom one-fifth was
Delta for that student.
units making up that course in one
50 percentile points(!!).
half of the academic term, then the Step 2 - As instruction progresses,
4. The teacher quality effect on student is proceeding at a pace of additional tests are administered on
student performance was found 100 points per term and has a a weekly, bi-weekly, or monthly
to be highly significant and larger Learning Delta of 100 (i.e. 50 points basis. Once again, each test is
in size than any other factor-- ÷ 0.5 terms = 100 points/term). If effectively a final exam, and stu-
including class size and quot;hetero- dents' scores reflect their new level
on the other hand, the student has
geneity.quot; of subject knowledge attained since
only mastered curriculum units
the beginning of the term.
worth 30 points by the mid-term,
In Dr. Sanders view, then, quot;…more that student has a Learning Delta of Step 3 - At the time of each assess-
can be done to improve education 60 (i.e. 30 points ÷ 0.5 terms = 60 ment Learning Delta figures are
by improving the effectiveness of points/term), and unless she/he calculated for each student and the
teachers than by any other single increase their Learning Delta to 140, average is calculated for the class as
factor.quot; And the key to that effort she/he will not complete the course a whole.
is collecting data on the learning

4. Page 3
• As interim Learning Deltas are Thought
Figure 3
measured, the students, teachers, Experiment
and parents can see if adequate % of Number of Students
Out of 7,000 Students in
Population Students Below Grade
progress is being made to the Reform Program
What follows Level
goal of mastery, and if not, they is a quot;thought Group 1 - Reading at Grade
can adjust course while there is Level (at 50th %ile or above)
experimentquot; 78.9% 5,523
still time. and illustra- Group 2 - About One Year
Behind (assume they score
tion of why
• Students who are progressing too around 20%ile) 11.7% 819 819
data on
slowly will have low Learning Group 3 - About Two Years
students'
Deltas that identify them for Behind (assume they score
special assistance or motivating. learning around 10%ile) 9.4% 658 658
progress
• Students who are moving much
Total 7,000 1,577
should be
faster than the rest of the group
used to drive
will have high Learning Deltas
decision-making in education
that identify them for additional The experiment involves applying
philanthropy. The example I will use
quot;enrichmentquot; work or accelera- some of the findings from the work
is based on an actual philanthropy-
tion into a more advanced class. of Dr. William Sanders and TVAAS
funded reading reform program
to this sample education philan-
A key difference between the (with some of the details disguised)
thropy program and posing the
Learning Delta and quot;value-addedquot; in a state where roughly one out of
following questions:
measures is that Learning Delta is
every five students in the target
designed to be a frequent measure
population fail to achieve quot;profi-
that enables the teacher and school • What if we measured the Learning
cientquot; status on the state's reading
leaders to make immediate-term Deltas being produced by teachers
tests, and about one in ten demon-
course corrections in every class- participating in a philanthropy-funded
strate only quot;minimalquot; reading ability.
room. The quot;value-addedquot; measure education reform program by captur-
For the purposes of this example,
as used by TVAAS is a once-a-year ing data from the students on a
the program covers 7,000 students
measure that, as a result, is retro- monthly or quarterly basis?
across 20 schools, and it costs about
spective. Although it can give a
$2 million per year to administer.
reading on how a school performed
• If Sanders' findings about teacher
in the prior year, as any particular Figure 1 shows a simple cost-per-
performance in Tennessee were
school year progresses it provides success calculation for this pro-
duplicated elsewhere, how would this
less and less help with how to gramwhich may involve expendi-
information enable us to increase the
improve learning performance in tures on professional development,
specific classrooms next week or quot;returnquot; of an education reform pro-
specialized curricular materials,
next month. gram or other innovation that is un-
technology innovations, or all these
derway?
elements.
Appendix A shows what Learning
Delta data would look like over a
term and how that allows classrooms Having made this calculation, the The Base Case
and teachers to be grouped into question remaining for the donor to
performance quintiles. Measures of learning progress are
answer is, quot;Do you think $4000 per
important because, as Sanders has
student is a satisfactory num-
shown, highly effective teachers
ber?quot; In the event that the
Figure 1 - Program Cost Per Success
move their students along at a much
donor is a venture-philanthropist
Number of Students in Program 7,000
greater pace than average or below-
with an especially keen interest
Total Program Cost for One Year $2,000,000 average teachers. And the situation
in results, the further question
in any particular school is further
may be, quot;Shall we continue
Number of Students Meeting 500
complicated because the curricular
Reading Standards Who Would- with this program or do
n't Have Otherwise
quot;distancequot; that must be covered in
something else where the
order to meet standards of compe-
Program Cost per Student- $4,000 impact may be greater?quot;
Success
tency can vary greatly from one
student to another.

5. Page 4
school year.
Figure 4 Quintile 3 teachers
accomplish some
Base Case
improvement, but
Q 1+2 Q3 Q4 Q5 Total
not enough to bring
$3,000,000 1500
their students up to
standard (see table
in Appendix B for
$2,000,000 1000
detail). As a result
of these combina-
Write-off Yield = 4.1 Yield = 7.4 Yield = 2.3
tions of teacher
$1,000,000 500
Number of Successes
132
effectiveness and
Program Cost
132 328
student deficit,
164
0 164
$0 0
then, only 40% of
0 0
Q 1+2 Q3 Q4 Q5 Total
Group 2 students
($400,000) ($400,000) ($400,000)
will be brought up
($1,000,000) ($800,000) -500
to grade level, and
only 20% of the
Group 3 students.
($2,000,000) -1000
($2,000,000)
Overall, based on
Sanders' findings in
($3,000,000) -1500
Tennessee, only
Teacher Performance Quintiles
7% of the 7,000
Program Cost Group 2 Students Group 3 Students
students in this
program are likely
to be counted as
Case graphic in Figure 4 shows the
successes at the end of the year
As shown in Figure 3, because 20%
effect of the differing levels of
(i.e. students who met the statewide
of students in the reform program
teacher effectiveness predicted from
reading standard who would not
we are looking at score below
Sanders' findings for students who
have otherwise). The cost per
quot;proficientquot; status on the state's
are one and two years behind in
student-success will, therefore, come
reading tests and about 10% demon-
their reading skills. Specifically,
in at about $4,400 and the return can
strate only quot;minimalquot; reading ability,
teachers in the bottom two quintiles
be thought of as 2.3 successes-per-
we get roughly 1,500 students out of
do not produce any improvement,
$10,000 quot;investedquot; by the philan-
the 7,000 who are functioning below
and there is a 39 percentile point
thropy.
grade-level.
difference in results between quin-
tiles 3 through 5, which is assumed
For the purposes of this illustration However, as shown in the figure,
to be distributed evenly across these
we then assume that the 800+ analyzing results according to
quintiles. So Quintile 5 teachers
Group 2 students that scored performance quintile makes certain
produce gains of 39%ile points in
quot;BASICquot; on the statewide compe- program improvement actions
their students; Quintile 4 teachers
tency test are one year behind, and immediately obvious. For example,
produce gains of 26%ile points; and
the remaining Group 3 students who the yield achieved by Quintile 5
Quintile 3 teachers produce gains of
scored quot;MINIMALquot; are two years teachers is 7.4 successes- per-
behind. To represent a typical 13%ile points in their students.
$10,000, and the Quintile 4 yield is
heterogeneous classroom these
4.1. But over half of the $2 million
student groups are distributed Under these conditions, when
program cost was spent on class-
uniformly across all the teacher combined with the children's varying
rooms that produced no suc-
quintiles. levels of skill deficit, only Quintile 4
cesses. What if actions were taken
and 5 teachers are likely to suc-
The implications of the Sanders to either a) increase the effective-
ceed in bringing students up to
findings for these 1,500 students is ness of lower quintile teachers, or b)
grade level by the end of the
what is most interesting. The Base move more Group 2 and 3 students

6. Page 5
the program's impact and yield. In
into quintile 4 and 5 classrooms? In program's reading interventions into
the absence of such data, though,
addition, there are borderline cases the next grade in order to finish the
proactive management of this type is
(the yellow cells in the Base Case job that was started with the border-
much harder because decisions
table in Appendix B) where some line students. This timeassuming
depend on a chain of personal
improvement was seen, but not that spending increases proportion-
observations, opinions, and judge-
enough to meet the proficiency ally by 4% to $2.08 millionthe
ments, which without hard data to
standard. What if additional funds program's success rate increases by
back them up, more often get
were spent to extend the program 64%; the cost per student-success
translated into politics rather than
and put those kids across the finish drops by 37%; and the program's action.
line? yield has increased from 2.3 to 3.6
successes-per-$10,000. (See table in
Managing with the Learning Delta
Appendix D for detail.)
What-If Scenarios
The point of management informa-
As shown in Figure 5 (and in Ap- tion, like the Learning Delta, is to
The second scenario in this experi-
pendix E), if scenarios two and three identify and target the areas that
ment moves Group 2 and 3 students are both put into effect, the pro- need to be better understood and
out of the bottom quintile classes gram's success rate would increase that are candidates for scrutiny and
and into the top two quintiles. Even by 193%; the cost per student- change. Once exceptions have been
assuming that program money success would drop by 64%; and identified, though, the purpose of
follows the kids and overall spending the program's yield would increase management information is not to
remains at $2 million the pro- from 2.3 to 6.3 successes- per- answer the question quot;what really
gram's success rate almost doubles $10,000, a jump of 176%! happened?quot;but rather, quot;who am I
to 13%; the cost per student-success going to call?quot; As was illustrated in
If, then, we had in our possession
has dropped by 50%; and the the above thought experiment, the
Learning Delta data that allowed us
program's yield has doubled from critical first step is to identify
to rank classrooms/teachers in order
2.3 to 4.6 student-successes per exceptions either based upon
of average actual Learning Delta per
$10,000. (See table in Appendix C specific criteria or using a normative
class, we could modify the allocation
for detail.) approachas in the examples
of students to teachers and/or
above.
enhance the skills of the low per-
In the third scenario we extend the forming teachers in order to increase
Unlike impact studies, the purpose
of performance management
information is not to produce
Figure 5 scientifically precise analyses of
Summary of Scenarios
efficient causes. Instead, perfor-
4500 70.00
mance measures and manage-
ment information, to be of value,
4000
60.00
must support real-time navigation
Number of Successes / $ Cost per Success
in the sense of providing near
3500
immediate feedback on and
50.00
3000
control over speed, course, and
Successes per $10,000
resource consumption. So
40.00
2500
because performance measure-
ment is very much like a steering
2000
30.00
wheel in a car, it needs to be
attached to and a part of a
1500
20.00
school's everyday processes.
1000
Otherwise, teachers and school
10.00
leaders cannot steer around
500
obstacles; they can only drive
into them, and then file an
0 0.00
Base Case Scenario 2 Scenario 3 Scenarios 2 + 3
accident report later.
Student Successes Students Successes per $10,000 Cost per Student Success

7. Page 6
model), which need to be improved, 3. Do our measures lead us to
Learning Delta Supports a Manage-
and which represent performance actions that improve perform-
rial Approach to Education Im-
beyond what we thought we could ance?
provement
produce?
4. What is our track record vis a vis
A managerial approach to education these standards?
At a minimum, a managerial ap-
improvement asks different ques-
proach to education improvement 5. Are we improving?
tions about schools than a curricu-
requires answers to five key ques-
lum-based reform or one based
tions: The key elements of a managerial
upon community outreach, etc. But
model for education improvement is
the answers to the managerial
1. What are the standards that the subject of another paper, but
questions can be critical to the
represent how we expect educa- some of the important considera-
success of these other reforms. For
tional processes to perform? tions are listed in Figure 6.
example, which attributes of a
classroom's, a school's, or a school 2. How do we measure actual
Learning Delta, as one top-level
system's performance are what we performance against these stan- performance measure, is a first-level
expect (under a particular reform dards? response to these questions no
matter what other type of reform
is being pursued. We know, for
example, that a rising fifth grader
Figure 6
who reads at a third grade level
Key Performance
must progress more quickly
Management Questions Selected Specifics than other fifth graders in order
to meet the standard for reading
• Processes include student grouping, curriculum
1. What are the standards
competence by the end of fifth
planning, lesson planning, instruction delivery, class-
that represent how we ex- work/homework assignments, tutoring, testing, stu-
grade. We can express that
dent performance planning, parent communication,
pect educational proc-
requirement as a Learning Delta
etc.
esses to perform?
like quot;150 points per term.quot; At
• What are the reform-related or other standards that
an interim point in the school
correspond to each of the educational processes
above? Are they widely publicized and/or formally
year we can identify students or
adopted?
classrooms who are progressing
too slowly, and we may be able
• Are there data capture mechanisms in place that
2. How do we measure ac-
to apply additional interventions
support performance measures?
tual performance against
to accelerate them to a success-
• Are they fact-based, objective, and timely?
these standards?
ful conclusion. Or we may find
• Are they efficient to use and non-disruptive to
teaching?
that the original interventions
prescribed by the reform model
• in use were never implemented
Are measures captured and reported frequently
3. Do our measures allow us
enough and with adequate turn-around to enable cor-
in certain classrooms. In either
to improve performance? rective action?
case, by capturing and analyzing
• Are measures specific enough to enable positive
Learning Delta data to identify
identification of the people and/or processes needing
improvement?
exceptions, school leaders can
• Are measures specific enough to enable positive
do two things: a) reduce the
identification of people and/or processes demonstrat-
cost of observing (i.e. inspecting)
ing excellence?
classes that are already meeting
performance standards, and b)
• Is there a store of historical measurement data
4. What is our track record
build their fact-base regarding
allowing for trend analysis?
vis a vis these standards?
• what works and what doesn't in
What are the performance trends versus key
educational processes and standards?
practice. Using this fact-base,
schools will be able to make
• Are there clearly articulated improvement goals and
sustainable, repeatable im-
5. Are we improving?
objectives?
provements to classroom and
• Are they being or have they been met?
school-wide processes that raise

8. Page 7
performance for the future. In other
words, we can improve the state of
education if we move beyond the
quot;random acts of progressquot; too often
seen in K-12 schools to data-driven
decision-making that focuses on
actual learning progress and its
underlying drivers.
Maximizing Learning Progress by
Reducing Student Wait-States
Another important reason to adopt
the Learning Delta as an interim
performance measure is that it
provides a better means than annual
standardized tests for ensuring that
all students will make adequate
learning progress.iii Imagine a
school that has as its standard that
every student will achieve a Learning
Delta of 60 points per term. Such a
school would be different from it is almost useless if used in isola-
These examples show how the
conventional schools in several ways: tion. Publicly traded companies get
problem of student quot;wait-statesquot; in
compared on ratios like Return On
• If an advanced student were to conventional classrooms retard
Equity (ROE) and Price-Earnings
score 80 out of 100 on a pre-test, learning progress. Students who are
ratio (P/E), both of which require
that studentand that student's substantially above or below the
an earnings number. But the key
class average in attainment are either
teacher would not be able to drivers of ROE for a nationwide
waiting for the class to catch up to
meet the school's standard of 60
clothing retailer which are them-
them, or waiting for someone to
points per term of learning prog-
selves performance measuresare
help them understand what is going
ress. Because the student's
very different from those for finan-
on. As shown in Figure 7, student
maximum gain in that class is
cial services companies or car
wait-states are the mirror image of
only 20 points, it is up to the
manufacturers. If as investors all we
the distribution of attainments in a
teacher and the school to make a
had to compare the performance of
classroom that was grouped by age.
change to that student's place-
these disparate companies were raw
Simply by using pre-tests and
ment or curriculum, or adequate
earnings numbers, we'd be in trouble
Learning Delta measures to proac-
learning progress will not be
(…more trouble). However, we've
tively manage learning progress,
achieved.
still got to have accurate earnings
student wait-states will be reduced
• Similarly, if a student scored 10 numbers! They are critical, but they
and learning progress improved
are also just the beginning.
out of 100 on a pre-test, it is un- during a school year rather than
likely that student would be able one or two years later. The case for Learning Delta is the
to achieve learning progress of
same: don’t mistake Learning Delta
90 points in one term. In fact,
for the only measure of a school's
Limitations
that student's low score shows
instructional success. Schools, like
that she/he is not in the right Learning Delta, though, is not and companies, are more complicated
classroom to begin with. Some- cannot be the single measure of than that, and there are underlying
thing needs to change if she/he success of a classroom, school, or drivers of Learning Delta that
is to meet the school's learning education reform program. Learning should be measured and managed.
progress standard. And it is up to Delta as a measure behaves a lot like And there are also other important
the teacher and the school to an earnings number for a commer- top-level measures, like graduation
change it. cial company. Earnings is a key rates, that should be attended to.
indicator of a company's health, but

9. Page 8
as teamwork, culture, and be-
But, as with company earnings, we Conventional schools and class-
havior.
need Learning Delta as a measure of rooms deliver instruction, post
the learning progress produced by assignments, collect and correct
• They create new information
teachers and schools because homework, and administer tests in
flows that provide same-day an entirely manual fashion. Manag-
learning progress is the very thing that
feedback on key performance ing the effectiveness of large num-
schools are supposed to produce. Without
attributes. These feedback loops bers of such classrooms eventually
such a measure, it is hard to imagine
support teachers, students, and becomes infeasible as a simple result
how teachers and schools can be
parents in their roles by quickly of increasing numbers of students
systematically managed and im-
identifying exception condi- and classes.
proved. On the other hand, perhaps
tions and bringing responses to
it is because we have been muddling By failing to separate the negative
bear so that the exceptions are
through without such a measure that effects of size from the positive
brought into compliance with
schools have remained essentially effects of good management, the
the school's predefined, pub-
unimproved over the last forty critics miss the opportunity we are
lished, and acknowledged stan-
yearsin spite of the tripling of per presented with: high-performance
dards.
pupil expendituresiv. school management can be scaled-
But high performanceas re- up if schools would only do what
searched in Samuel Casey Carter's the rest of the U.S. economy did
the book No Excusesvis based starting 20 years ago as a response to
Part 3: Putting the Learning
largely on quot;management-by-walking- growth: get digitized.
Delta into Practice
aroundquot;. And it is as a result that
high performance schooling is
We can identify high performance Digitize what?
negatively correlated with school
schools because they are the ones
size.
that produce greater than average Four or five years ago the excite-
Learning Deltas. And as with other ment about quot;cyber-schoolsquot; and the
Critics of the quot;No Excusesquot; research
human enterprises, high perfor- quot;virtual classroomquot; was mostly
often claim that the schools studied
mance schooling is largely a matter focused on distance learning and
are statistical outliers that are irrele-
of good leaders providing sound how it enabled quot;anywhere, anytime
vant to the problems of large
management using effective tech- learning,quot; i.e. the elimination of time
schools because of the special
niques and systems. For example, and space as barriers to taking
conditions involved, small size
KIPP Academy in Houston enrolls courses or earning degrees from off-
among them. They ask, quot;What is the
fifth graders who for the most part campus locations. Subsequently,
lesson of high performance school-
function at a third grade level. In the educational institutions began to
ing for the management of tens of
space of four years KIPP turns out discover the benefits to on-campus
thousands of students rather than
ninth graders operating at a ninth students of on-line courses, and so
hundreds?quot; And they conclude that
grade level--as defined by exclusive the quot;Web-enhanced campusquot;
there is none.
preparatory schools. On average, became an object of enthusiasm in
There are distinct diseconomies of
KIPP's students make 6 year's higher education. Accordingly, the
scale at work and classic quot;locus of
academic progress in 4 year's time. major distributed-learning platforms
controlquot; problems that stand in the like Blackboard, WebCT, and Lotus
KIPPand other high performance way of scaling up quot;No Excusesquot; LearningSpace tried to provide on-
schools like it employ methods, practices. Simply put, attempting to line analogues to the blackboard, the
practices, tools and techniques that duplicate quot;No Excusesquot; results using lectern, the classroom discussion,
are different from those found in quot;No Excusesquot; techniques in large the student's raised hand, and so
ordinary schools. And they are well school settings would require many forth. As a result, the surrounding
managed by leaders who are em- more principals and instructional debate about quot;e-Learningquot; has
powered to do so. For example: leaders quot;managing-by-walking- mostly focused on whether an on-
aroundquot; than school systems are line class discussion is as good as the
• They explicitly define common
able or willing to pay for. face-to-face variety, or whether
roles and standards for teachers,
listening to a lecture is as rich an
students, and parents to play But a well understood lesson from
experience on-line as it would be
within well-defined processes outside the education sector is that
actually sitting in a lecture hall.
covering attendance, classwork, even very effective manual processes
homework, and testing, as well break down under high volumes.

10. Page 9
The point I wish to make here about
quot;digitizingquot; the classroom is differ- Figure 7
ent from the quot;virtual classroomquot; Key Attributes of a Learning Delta Management System
discussion described above. Class-
rooms, schools, and school systems • Curriculum-Neutral  Accommodates whatever curriculum is
1. Least
already in use by a school. Does not impose a new one.
are enterprises, which–like others– intrusive
• Pedagogy-Neutral  Does not require that classroom-based
are comprised of methods, practices, teaching be replaced with computer-based instruction.
tools, techniques, processes, knowl- • Ease-of-Use  Fits well with and supports classroom
edge and people. What needs to be routines. Data capture and feedback mechanisms are user-
digitized in K-12 education are the friendly.
classroom workflowswhich
•
connect the teacher to the students, Allows use of hardware with lowest total cost of ownership:
2. Least
handheld computers like Palm OS or Pocket PC devices.
parents, and the schoolso that costly
• Minimizes requirement for network and/or centralized applica-
basic data on what is getting done tion support though use of application-service-provider (ASP)
and what isn't can be used to man- model.
age classroom performance and
•
improve results. Capturing data like Should automate the testing process from start-to-finish in
3. Most
order to relieve teachers of manual tasks and free up new time
Learning Gap, planned vs. actual efficient for teaching. Should not simply impose a new clerical process.
Learning Delta, and the like is both • Should provide teachers with detailed, useful performance
necessary and desirable because it analyses on demand in order to increase teacher and class-
enables instrumentation of the room effectiveness.
classroom rather than virtualiza-
tion.
first step to making that type of when high-stakes tests are adminis-
Using digitized performance infor- analysis-driven facilitation possible. tered, but by then it is too late to
mation, school and district leaders improve outcomes for the students
can exercise anytime, anywhere,
being tested.
Digitize How?
quot;line-of-sightquot; management of
bricks-and-mortar classrooms in With digital mechanisms for meas- To put into practice data-driven
much the same way as with virtual uring the actual learning progress of performance management based on
classrooms, but without the disrup-
each student, schools could begin to the Learning Delta alone, three key
tion of trying to deliver instruction
correlate learning results to teaching components are required:
in a new way. Digitizing the class-
performance, rank classrooms
room to accomplish instrumenta-
accordingly, and then scrutinize • A testing process producing
tion would enable broad-based
classroom methods, tools and scales (scores) that have a de-
performance improvement through
techniques to identify improvement
the regular use of on-line reporting fined (i.e. mapped) relationship
actions. The current state in our
facilities and quot;drill-downquot; analysis to: to the curriculum, and which
schools, though, is that very little produces measurements that
a) identify the Master Teachers
information comes out of class- extend above and below grade
who are the exemplars of high
rooms once the teachers and stu- level;
performance that should be
dents have gone in. As a result,
emulated, • A database and data capture
instructional leaders, administrators,
tools that track individual stu-
b) identify the classrooms and and parents cannot tell on a daily,
dent scores and maintain links
teachers needing help to im- weekly, or monthly basis which
prove their performance, and to classes, teachers and schools;
classrooms are on schedule and
and
which are not, which curriculum
c) bring the two groups together
elements--new or old--work very
so that expertise and experi- • A reporting and analysis facility
well or very poorly, or which stu-
ence can be shared. that supports the type of data-
dents are functioning at grade-level, driven decision-making illus-
That is a fitting role for school
who have pulled ahead, and who trated above.
leaders and managers, and digitiz-
have fallen behind. Eventually,
ing the classroom is the critical
these questions may get answered

11. Page 10
These components comprise a the data fairly or constructively, the
Change Management and People
minimum infrastructure for data- environment needs to change to
Factors
driven performance improvement in address those fears and establish
schools that would relatively simple common goals.
Most importantly, though, a
to develop and implement. And for Learning Delta Management In the case of schools, control of the
many schools having a system with System can only enable perform- environment is shared mostly
just these components would ance improvement if the real solu- between two groups: the teachers'
represent a logical, high-impact tion is already in place: a concerted unions and school administration. If
starting point for quot;getting digitalquot;
and serious demand for meaningful either one of these groups is op-
and starting to practice data-driven
performance data from schools. That posed to adopting a new system, it
decision-making. But for a school to
will most probably fail. Similarly, if
has to come first, or anyone at-
obtain performance tracking and
it's not in the interest of these
tempting to implement such a
management functionality in the
groups for learning performance
system will likely fail.
educational software market often
management practices to take root
requires that an entire instructional As experienced system implementers and spread, then they will not. So
management system be pur- can all attest, quot;people factorsquot; are unlike smaller class size and in-
often more decisive than a system's
chasedalong with substantial creased professional develop-
design or features in determining the
amounts of quot;course-warequot; and
mentwhich probably benefit kids
success or failure of a project. K-12
embedded curriculum that may not
in some ways and certainly benefit
schools are no exception. Three
be appropriate or useable by class-
the adults performance measure-
dimensions always need to be
room teachers. In addition, such
ment as a means of managing
addressed:
systems require amounts of com-
schools is problematic because it has
puter hardware that many schools
• Ability - Do the intended users the clear potential to create winners
can ill afford. Taken together, these
of a system have the training, and losers. It increases the riskiness
attributes of vendor offerings
time, and resources required to of being a teacher or an administra-
represent substantial barriers to
use it successfully? tor in a school that is not perform-
adoption by schools that compound
ing well, and so it is harder to win
the usual issues of cost, staff • Willingness - Are the users of a votes on the quot;willingnessquot; dimen-
training, and curriculum manage- system willing to change how sion, both among a school's teaching
ment. they go about their jobs in order staff and the groups in charge of the
to use the system successfully?
The broad-based application of environment.
Learning Delta measures in schools • Environment - Do system users Of course, parents and communities
requires that a more focused and believe that doing what it takes have long wanted tools to measure
easier-to-implement Learning to make the system successful our children's absolute and relative
Delta Management System be will be recognized and rewarded educational progress, but they are
made available. To be broadly as a positive achievement by the not the ones being asked to change.
applicable in divers types of schools groups that control the environ- And they do not control the envi-
and communities it should: ment? ronment.
• Accommodate rather than It turns out that the first two dimen-
disrupt the established instruc- sions, ability and willingness, are
tional methods and materials, heavily conditioned by the third:
environment. For example, if a
• Be designed for very low total
school's teaching staff are unable to
cost of ownership, and
implement a new learning manage-
• Provide teachers, students, and ment system because they don't have
the training or the release time to get
parents with the usual and cus-
the training, the environment needs
tomary benefits of automation
to change to accommodate those
rather than creating additional
needs. If a school's teaching staff
work (see Figure 7).
are opposed to a learning manage-
ment system because they don't trust
administrators/management to use

12. Page 11
PRESS RELEASE (Excerpt)
Conclusion
PALO ALTO, Calif., May 12 /PRNewswire/ -- NewSchools Venture Fund announced the launch
of its new Performance Accelerator Fund, targeted at $20 million and designed to invest in ven-
There are signs of positive change, tures that enhance the capacity of school systems to produce high levels of student achieve-
however. Because of the No Child ment. . .
Left Behind Act and other factors, quot;As districts work to increase student achievement, we must ensure that they have the tools they
the secular climate for learning need to be successful,quot; said Kim Smith, co-founder and CEO of NewSchools Venture Fund. . .
performance measurement in State and federal education policies are increasingly pushing for school systems to shift from
classrooms is probably better than it compliance-driven organizations -- governed by rules, regulations and court orders -- to per-
formance-based cultures focused on high student achievement. . .
has ever been. But ideology and
labor relations remain very high . . .The ventures in the Performance Accelerator Fund will focus on developing people who know
how to lead and teach in a performance-based environment; designing tools that give those
barriers to the success of data-driven people the information they need to make good instructional decisions; and supporting practices
learning management within K-12 that promote and reinforce success. The Fund will identify, fund and build ventures that address
education. human capital -- including the recruitment, preparation and support of high-quality teachers and
leaders -- and performance tools -- including information systems and assessment tools
that enable teachers and leaders to make data-based decisions about their students' in-
One bellwether of this positive struction. (emphasis added)
change may be the New Schools
Venture Funda leading venture
philanthropywho in their recent systemic improvement in K-12
One particularly good chance to
press release point to a shift away schools is to transform the class-
demonstrate the value of Learning
from schools being compliance- room into the atomic unit of data-
Delta management is, then, is for the
driven organizations to perform- driven performance management.
quot;managerial modelquot; of school
ance-based cultures focused on The first step is to digitize the
improvement to be incubated inside
achievement. classroom and to start using meas-
a substantial education reform
ures like the Learning Delta to
Venture philanthropy-based educa- program or EMO while the market
manage and improve school per-
tion reforms are in an excellent for Learning Delta management
formance as a regular part of a
position to demonstrate the power software and services develops. To
school's weekly routine.
for positive change that performance be convincing, a demonstration of
measures like Learning Delta repre- the managerial model needs to span
sent. Education management multiple classrooms, schools, and
organizations (EMOs) like Edison even school systems. It needs to
Schools and Chancellor Beacon have shown its effectiveness on a
Academies are also in a position to substantial scale in order to be a
make a positive impact. In fact, candidate solution for the systemic
because they are for-profit and so improvement of education in this
get criticized for diverting money country (something that No Excuses
quot;away from the children,quot; they schools cannot do).
should be strongly motivated to use
Learning Delta measures to en-
sure—and then later prove—that
* * *
they are doing a better job than
traditional schools. And so paying Because improving the effectiveness
profits to shareholders is in some of teachers can improve education
sense justified if it makes possible a more than by any other single factor,
demonstrably better education for our greatest opportunity to achieve
our kids.
For more information contact:
Gregory D. Luther, Founder and Principal ! High Delta Learning, LLC ! P.O. Box 398 ! Litchfield, CT 06759
tel: 860.567.8242 ! e-mail: greg_luther_ab82@post.harvard.edu

13. Page 11
Appendix A
EXAMPLE OF LEARNING DELTA DATA - Classroom A
Mr. Smith
Teacher Name:
80
Mastery Score:
40 points per term
Learning Delta Standard:
Plan vs. Actual
Timeframe: Aug Sep Oct Dec Learning Delta
Learning Gap / Interim Interim Final
Planned Learning Learning Learning Learning
Delta Delta Delta Delta Aug - Dec
Student Pre-Test Test 2 Test 3 Final Test
Student 100 39 41 47 24 61 34 82 43 2
Student 101 21 59 28 21 33 19 55 34 -25
Student 102 24 56 36 37 55 47 80 56 0
Student 103 55 25 57 6 72 25 95 40 15
Student 104 35 45 53 56 58 35 67 33 -13
Class Average
Test Score 35 44 56 76
Class Average
Learning Delta 45 29 64 41
Finished a
Class needs to Going too Speeding
little under
achieve this slowly! to catch
plan
average quot;speedquot; up
41
Teacher's Learning Delta Result for Term
40%
% of Students Meeting Learning Delta Standard
Teacher Inter-ranking
Average Class
Name Rank Learning Delta Quintile
Hume 1 72 Q1
Hobbes 2 68
James 3 66
Locke 4 60 Q2
Descarte 5 58
Smith 6 41
Weber 7 40 Q3
Berkeley 8 39
Russell 9 38
Machievelli 10 32 Q4
Mosca 11 22
Michels 12 18
Rousseau 13 16 Q5
Nozick 14 15
Rawls 15 15