At a time when many schools are choosing to use adaptive math learning programs to support personalized learning approaches, it’s essential that educators focus on collecting and reviewing evidence about the impact these programs have on student understanding and achievement in mathematics. In this webinar, Dr. Tim Hudson, Vice President of Learning at DreamBox Learning, shared the latest research of adaptive math programs. He discussed recent research findings and the methodological considerations that are used for studying the impact of these programs.

1. Personalized and
Adaptive Math Learning
Recent Research and
What It Means for Your
StudentsTim Hudson, PhD
Vice President of Learning
DreamBox Learning
@DocHudsonMath

2. • Define Personalized Learning and
Adaptive Learning
• Present recent third-party research
findings
• Interpret research studies involving digital
programs in which students are learning
along a personalized path and pace
• Share ideas for conducting evaluations of
adaptive or personalized programs in your
own context
• Understand the importance of classroom
teachers in implementing adaptive math
programs, whether in a blended learning
model or not

3. Defining
Personalized
Learning

4. “…one would think that by 2025, age-graded schools and the
familiar teaching and learning that occurs today in K-12 and
universities would have exited the rear door. Not so.
Blended instruction, personalized learning, and flipped
classrooms will reinforce the age-graded school, the 19th
century organizational innovation that is rock-solid in 2015.”
Larry Cuban, December 2015
Stanford University, Professor Emeritus of Education
From “Predictions, Dumb and Otherwise, about Technology in Schools in 2025”
www.larrycuban.wordpress.com
Next Generation Learning & Schooling is at Risk

5. Today’s Lesson: Fraction
Operations

6. I haven’t learned
anything new all
year. This math
class is boring.
This is way too
hard. It doesn’t
make sense.
I give up!
This math
class is just
right for me –
not too hard,
not too easy.
Today’s Lesson: Fraction
Operations

7. Today’s Lesson: Fraction
Operations
Why?

8. Today’s Lesson: Fraction
Operations
Why?
These students were all born during the
same 12-month window about 11 years ago.

9. Seat-time policies, pacing
calendars, standardized testing
schedules, age-based classrooms
and weak competency-based
tools make it difficult for me to
personalize and differentiate.

10. .
Personalized
(Relational)
Impersonal (Industrial)
Schoolin
g
Structures
from
Adults

11. Personalized
Schooling
Personalized
Learning
Industrial
Schooling
Industrial
Learning
Personalized
(Relational)
Impersonal (Industrial)
Schoolin
g
Structures
from
Adults

12. School Policies &
Structures are Designed for
Students as Unique
Individuals.
Strategic & Varied
Schedules, Locations,
Paths & Paces
Empowering Learning
Experiences, Critical
Thinking, Creativity,
Exploration.
Students “Think & Do”
using Their Own Intuitive
Ideas
School Policies &
Structures are Designed for
Efficiency, Economy &
Scale.
Fixed Schedule, Location,
Path & Pace
Age-Based Pacing
Calendars
Traditional Lesson
Paradigm of Mass
Instruction
Teach, Practice, Test
Students “Sit & Get” the
Teacher’s Ideas
Personalized
(Relational)
Impersonal (Industrial)
Schoolin
g
Structures
from
Adults

13. Two Students. Millions of Self-Directed Learning
Paths.

14. The Pacing
Calendar
is Here

15. School Policies &
Structures are Designed for
Students as Unique
Individuals.
Strategic & Varied
Schedules, Locations,
Paths & Paces
Empowering Learning
Experiences, Critical
Thinking, Creativity,
Exploration.
Students “Think & Do”
using Their Own Intuitive
Ideas
School Policies &
Structures are Designed for
Efficiency, Economy &
Scale.
Fixed Schedule, Location,
Path & Pace
Age-Based Pacing
Calendars
Traditional Lesson
Paradigm of Mass
Instruction
Teach, Practice, Test
Students “Sit & Get” the
Teacher’s Ideas
Personalized
(Relational)
Impersonal (Industrial)
Schoolin
g
Structures
from
Adults
Learnin
g
Pedagogy
with
Students

16. Today’s Lesson: Fraction
Operations
Math is boring.
Every math class is
the same: Lecture,
practice, lecture,
practice, quiz, test.
It’s not engaging,
active, or authentic.
I know how to win at
school. I just wait for
the teacher to show
me how to solve
familiar problems.
Math doesn’t make
any sense. I can’t
figure it out or
understand it.

17. School Policies &
Structures are Designed for
Students as Unique
Individuals.
Strategic & Varied
Schedules, Locations,
Paths & Paces
Personalized
Learning
School Policies &
Structures are Designed for
Efficiency, Economy &
Scale.
Fixed Schedule, Location,
Path & Pace
Age-Based Pacing
Calendars
Industrial
Learning
Personalized
(Relational)
Impersonal (Industrial)
Learnin
g
Pedagogy
with
Students
Schoolin
g
Structures
from
Adults

18. School Policies &
Structures are Designed for
Students as Unique
Individuals.
Strategic & Varied
Schedules, Locations,
Paths & Paces
Empowering Learning
Experiences, Critical
Thinking, Creativity,
Exploration.
Students “Think & Do”
using Their Own Intuitive
Ideas
School Policies &
Structures are Designed for
Efficiency, Economy &
Scale.
Fixed Schedule, Location,
Path & Pace
Age-Based Pacing
Calendars
Traditional Lesson
Paradigm of Mass
Instruction
Teach, Practice, Test
Students “Sit & Get” the
Teacher’s Ideas
Personalized
(Relational)
Impersonal (Industrial)
Learnin
g
Pedagogy
with
Students
Schoolin
g
Structures
from
Adults

19. School Policies &
Structures are Designed for
Students as Unique
Individuals.
Strategic & Varied
Schedules, Locations,
Paths & Paces
Empowering Learning
Experiences, Critical
Thinking, Creativity,
Exploration.
Students “Think & Do”
using Their Own Intuitive
Ideas
School Policies &
Structures are Designed for
Efficiency, Economy &
Scale.
Fixed Schedule, Location,
Path & Pace
Age-Based Pacing
Calendars
Traditional Lesson
Paradigm of Mass
Instruction
Teach, Practice, Test
Students “Sit & Get” the
Teacher’s Ideas
Blende
d
Blende
d
Personalized
(Relational)
Impersonal (Industrial)
Learnin
g
Pedagogy
with
Students
Schoolin
g
Structures
from
Adults

20. School Policies &
Structures are Designed for
Students as Unique
Individuals.
Strategic & Varied
Schedules, Locations,
Paths & Paces
Empowering Learning
Experiences, Critical
Thinking, Creativity,
Exploration.
Students “Think & Do”
using Their Own Intuitive
Ideas
School Policies &
Structures are Designed for
Efficiency, Economy &
Scale.
Fixed Schedule, Location,
Path & Pace
Age-Based Pacing
Calendars
Traditional Lesson
Paradigm of Mass
Instruction
Teach, Practice, Test
Students “Sit & Get” the
Teacher’s Ideas
Blende
d
Blende
d
Is there
an app
for this?
Is there
an app
for this?
Is there
an app
for this?
Is there
an app
for this?
Personalized
(Relational)
Impersonal (Industrial)
Learnin
g
Pedagogy
with
Students
Schoolin
g
Structures
from
Adults

21. School Policies &
Structures are Designed for
Students as Unique
Individuals.
Strategic & Varied
Schedules, Locations,
Paths & Paces
Empowering Learning
Experiences, Critical
Thinking, Creativity,
Exploration.
Students “Think & Do”
using Their Own Intuitive
Ideas
School Policies &
Structures are Designed for
Efficiency, Economy &
Scale.
Fixed Schedule, Location,
Path & Pace
Age-Based Pacing
Calendars
Traditional Lesson
Paradigm of Mass
Instruction
Teach, Practice, Test
Students “Sit & Get” the
Teacher’s Ideas
Blende
d
Blende
d
Is there
an app
for this?
Is there
an app
for this?
Is there
an app
for this?
Is there
an app
for this?
Personalized
(Relational)
Impersonal (Industrial)
Learnin
g
Pedagogy
with
Students
Schoolin
g
Structures
from
Adults

22. Defining
Adaptive Learning

23. “Adaptive” as
“Assessment
Diagnostic”
Giving a student slightly harder
or easier problems next
“Adaptive” as
“Unlocking Levels”
Move at your own pace in a
linear fashion upon completion

24. “Adaptive” as
“Recommendation”
Making recommendations for
lessons, videos, readings, or
assignments to do next, often
crowd-sourced and/or using big
data algorithms to choose content.
“Adaptive” as
“Behavior Reinforcement”
Conditioning the mind
to remember
information using tiny,
incremental skill steps
given to students in
repetitive feedback
loops
http://teorije-ucenja.zesoi.fer.hr/doku.php?id=instructional_design:programmed_instruction

25. Seymour Papert, February 29, 1928 – July 31, 2016
“The role of the teacher is to create the conditions for
invention rather than provide ready-made knowledge.”
S. Papert (1993). The children’s machine: Rethinking schools
in the age of the computer. New York: Basic Books.

26. Seymour Papert, February 29, 1928 – July 31, 2016
S. Papert. Constructionism vs. Instructionism. http://www.papert.org/articles/const_inst/const_inst1.html
“But children, what can they make with mathematics?
Not much. They sit in class and they write numbers on
pieces of paper. That's not making anything very
exciting. So we've tried to find ways that children can
use mathematics to make something -- something
interesting, so that the children's relationship to
mathematics is more like the engineer's, or the
scientist's, or the banker's, or all the important people
who use mathematics constructively to construct
something.”

27. What happens in each lesson?

28. “True virtual manipulatives allow more than
simply a viewing of objects on the computer
screen. They allow increased engagement,
forcing the user to interact with dynamic
objects. Through this interaction students
have opportunities to make meaning and
see relationships as a result of their own
actions. And it is this interactive engagement
with a dynamic representation that is the key to
the knowledge construction process.”
From Virtual Manipulatives In The K-12 Classroom, Moyer, Bolyard, Spikell, 2002

29. Truly Adaptive Learning
Technology requires
dynamic content that is built
from the ground up to invite,
analyze and respond to a
learner’s initial conceptions
and support their developing
ideas.

30. Recent Research
about
Personalized
Learning
& Adaptive Learning
Tools for
Interpreting Similar

31. DREAMBOX LEARNING ACHIEVEMENT GROWTH
in the Howard County Public School System and Rocketship
Education
Key Findings Technical
Appendix
(2016). DreamBox Learning Achievement Growth in the Howard County Public School System and Rocketship Education
. Center for Education Policy at Harvard University. http://cepr.harvard.edu/publications/dreambox-learning-achievement-

32. Other Resources
• Lessons Learned From Early
Implementations of Adaptive
Courseware
• College-Level Study found “no discernible
impact on grades” or course completion
• Conducted by SRI. 19,500 students, 280
instructors, 14 universities, April 2016
• https://www.sri.com/sites/default/files/broch
ures/almap_final_report.pdf
• EdSurge Adaptive Learning
Overview
• https://www.edsurge.com/research/special-
reports/adaptive-learning/

33. Conducted by the Center for Education
Policy Research (CEPR) at Harvard
University
3,000 Students in Grades 3-5
2 School Systems
100 Classrooms
2013-14 & 2014-15
NWEA MAP, PARCC, and State
Assessments
7 Key Findings
Quasi-Experimental Analysis

34. 1.How did HCPSS and Rocketship elementary
schools implement DreamBox in their
classrooms over these two years?
2.What was the relationship between DreamBox
usage and achievement gains on interim and
end-of-year assessments for students in these
schools?
3.Was DreamBox adoption causally related to
changes in students’ achievement?
Guiding Research Questions
(2016). DreamBox Learning Achievement Growth in the Howard County Public School System and Rocketship Education
. Center for Education Policy at Harvard University. http://cepr.harvard.edu/publications/dreambox-learning-achievement-

35. “… where 75% or more of the students used
DreamBox during a particular school year. Within
these using-classrooms, we did not impose any
artificial minimum on the amount of usage for a
student to be included in the sample, so we could
capture the full range of usage patterns in
classrooms in which 75% or more of students
had some DreamBox usage.”
Treatment Group: Students in “Using
Classrooms”
(2016). DreamBox Learning Achievement Growth in the Howard County Public School System and Rocketship Education
. Center for Education Policy at Harvard University. http://cepr.harvard.edu/publications/dreambox-learning-achievement-

36. Usage by Site and Year
(2016). DreamBox Learning Achievement Growth in the Howard County Public School System and Rocketship Education
. Center for Education Policy at Harvard University. http://cepr.harvard.edu/publications/dreambox-learning-achievement-

37. Key Finding 1: Below Recommended Usage
(2016). DreamBox Learning Achievement Growth in the Howard County Public School System and Rocketship Education
. Center for Education Policy at Harvard University. http://cepr.harvard.edu/publications/dreambox-learning-achievement-

38. Key Finding 3: Usage was driven by teacher- and
school-level practices instead of student
preferences
(2016). DreamBox Learning Achievement Growth in the Howard County Public School System and Rocketship Education
. Center for Education Policy at Harvard University. http://cepr.harvard.edu/publications/dreambox-learning-achievement-

39. Implication
“More than half of the
variance in usage from one
student to another depended
on the specific teacher
working with them or the
school they attended.”
Teachers and Administrators play a
key role in defining a schedule that
provides time for personalized learning
and access to devices for the digital
components of blended learning.
(2016). DreamBox Learning Achievement Growth in the Howard County Public School System and Rocketship Education
. Center for Education Policy at Harvard University. http://cepr.harvard.edu/publications/dreambox-learning-achievement-

40. Key Finding 2: Usage in some schools was to
support low-achieving students and after-school
learning.
(2016). DreamBox Learning Achievement Growth in the Howard County Public School System and Rocketship Education
. Center for Education Policy at Harvard University. http://cepr.harvard.edu/publications/dreambox-learning-achievement-
Your
Pacing
Calendar
is Here

41. Key Findings 4 & 5More usage resulted in
larger achievement
gains
Completing recommended
lessons resulted in faster
achievement gains
(2016). DreamBox Learning Achievement Growth in the Howard County Public School System and Rocketship Education
. Center for Education Policy at Harvard University. http://cepr.harvard.edu/publications/dreambox-learning-achievement-

42. Key Findings 4 & 5More usage resulted in
larger achievement
gains
Completing recommended
lessons resulted in faster
achievement gains
(2016). DreamBox Learning Achievement Growth in the Howard County Public School System and Rocketship Education
. Center for Education Policy at Harvard University. http://cepr.harvard.edu/publications/dreambox-learning-achievement-
Rocketship
Howard
County
Year 2013-14 2014-15
Average Time
Using DreamBox
6.3 hours 7.1 hours
Starting
Percentile
50 50
Ending
Percentile
54 54-55
Assessment CST (California) PARCC

43. Key Findings 4 & 5More usage resulted in
larger achievement
gains
Completing recommended
lessons resulted in faster
achievement gains
(2016). DreamBox Learning Achievement Growth in the Howard County Public School System and Rocketship Education
. Center for Education Policy at Harvard University. http://cepr.harvard.edu/publications/dreambox-learning-achievement-
Rocketship
Howard
County
Year 2013-14 2014-15
Average Time
Using DreamBox
6.3 hours 7.1 hours
Starting
Percentile
50 50
Ending
Percentile
54 54-55
Assessment CST (California) PARCC
Rocketship
Howard
County
Year 2013-14 2014-15
Time in DreamBox
Recommended Lessons
All All
Average Percentile
Point Gain
5.8 9.8
Assessment CST (California) PARCC

44. Key Question:
Validation“Educators often wonder
whether “progress” as
measured by educational
software translates into
student progress on interim
and state assessments.
As a result, we examined
whether the amount of progress
students made through the
DreamBox curriculum was
related to achievement gains on
state tests and interim
assessments.”(2016). DreamBox Learning Achievement Growth in the Howard County Public School System and Rocketship Education
. Center for Education Policy at Harvard University. http://cepr.harvard.edu/publications/dreambox-learning-achievement-

45. Key Finding 6: DreamBox’s progress
measurement was positively associated with
achievement gains on state tests and interim
assessments
(2016). DreamBox Learning Achievement Growth in the Howard County Public School System and Rocketship Education
. Center for Education Policy at Harvard University. http://cepr.harvard.edu/publications/dreambox-learning-achievement-
DreamBox data about a 3rd Grade Student

46. Key Finding 6: DreamBox’s progress
measurement was positively associated with
achievement gains on state tests and interim
assessments
(2016). DreamBox Learning Achievement Growth in the Howard County Public School System and Rocketship Education
. Center for Education Policy at Harvard University. http://cepr.harvard.edu/publications/dreambox-learning-achievement-
Rocketship Howard County
Year 2014-15 2014-15
DreamBox Curriculum Completed at the
Student’s Grade Level
12.5% 10%
NWEA MAP Average Percentile Point Increase 3 1.5
PARCC Average Percentile Point Increase -- 3.5
“Importantly, we
also found that
progress below
grade level was
related to increased
performance on
both assessments
in HCPSS.”

47. Implication
Prior knowledge is a key driver of
future learning.
Personalized Learning and
Adaptive Learning technologies can
support on-grade level
achievement even when students
don’t reach far into their grade-level
curriculum.
Real-time differentiation with
technology honors students’ prior
knowledge as they move at their
own pace.

48. Key Finding 7: Evidence for
causal impact is encouraging
but mixedOther Variables
• Student motivation
• Teacher effectiveness
• Student study habits
Control via a “Natural Experiment”
• In HCPSS, “compare the gains of
students in classrooms using DreamBox
to the gains of students in similar
classrooms in schools that did not use
DreamBox.”
• Note: All students in Rocketship used
DreamBox(2016). DreamBox Learning Achievement Growth in the Howard County Public School System and Rocketship Education
. Center for Education Policy at Harvard University. http://cepr.harvard.edu/publications/dreambox-learning-achievement-

49. http://cepr.harvard.edu/publications/dreambox-learning-achievement-growth, Technical Appendix, page 13

50. Considerations for Designing
& Conducting Your Own
Studies• Randomized Controlled Trials are difficult and often
inadvisable in many education contexts
• Quasi-Experimental Studies need valid comparison
groups (quasi-control and quasi-treatment groups)
• Case Studies often have no comparison group, which
must be considered when interpreting any potentially
causal impact
• Create unbiased treatment and control groups (i.e., no
minimum restrictions on the amount of curriculum
completed or only considering students who are “on
grade level” in the program)
• Use same-student comparisons, not averages of a
classroom, grade level, or school, nor different cohort
comparisons

51. Questions
Tim Hudson, PhD
Vice President of Learning
DreamBox Learning
@DocHudsonMath

52. Thank You!

53. DreamBox Learning® K–8 Math
Transformative Learning Experiences available in English & Spanish

54. DreamBox Lessons & Virtual Manipulatives
Intelligently adapt & individualize to:
• Students’ own intuitive strategies
• Kinds of mistakes
• Efficiency of strategy
• Scaffolding needed
• Response time
“I’ve tried teaching this topic
with pencil, paper, and
plastic manipulatives for
twenty years and students
have never really gotten it. I
can immediately see that
DreamBox has invented
new, digital manipulatives
and games that are far
more powerful for student
learning.”
– Urban District Math Leader,
CA

55. AssignFocus™
Differentiated assignments for every student through your
Insight Dashboard
To accelerate learning, offer remediation, and adjust classroom instruction

56. Prepare
for the
Next Unit
Homework
Assignme
nt
Review for
Assessme
nt
Empowered Teaching with
Data.
Inspired Learning with
Technology.

57. iNACOL is right around the corner…
Catch us at @DreamBox_Learn
Learn more and see how it works:
www.DreamBox.com/request-a-demo
Efficacy: Independent Validation from SRI
and from CEPR at Harvard University

58. We value your feedback!
Let us know how we’re doing:
www.surveymonkey.com/r/GC6ZCM7