1. Sustaining and Scaling Educational
Innovations:
Sharing Experiences and Challenges
from Singapore
Chee-Kit Looi
National Institute of Education (NIE)
Nanyang Technological University
Singapore
(Talk at CITERS 2012, Hong Kong)
1

2. First a joke/teaser
Change Knowledge (Michael Fullan)
“4 frogs are
sitting on a log,
and one decides
to jump off.
How many frogs
are left?”
2

3. Outline of this talk
• Share some experiences in understanding
scale-up from the research perspectives
• Share some thoughts on the challenges of
translation, sustainability, scaling
– Pose some questions and try to answer them
3

4. • People like to ask (at least in Singapore):
– What is scaling? What is to be scaled up? Which?
– Why scaling? Who to scale?
– How to scale?
– When to scale?
4

5. When you think of sustaining and scaling,
what innovations do you think of?
A good example?
A bad example?
5

6. Let’s think in terms of specific
interventions I am familiar with …
6

7. First innovation: Small group
collaboration in the classroom …
To give you some idea about GS, see this
video
7

8. What is to be sustained and scaled up?
• Rapid Collaborative Knowledge Improvement
(RCKI) practices
• RCKI principles (lesson design principles)
• Teacher Facilitation/Orchestration skills
• Lesson Co-design Practices (collaborative-
apprenticeship learning (CAL) model)
• PD model (principle-based instead of
procedure-based)
• …
8

9. Scale from what to what?
• Achieving greater depth (Coburn, 2003)
• Shift of ownership (Coburn, 2003)
• From the pilot duration to a longer duration
• From a few teachers to a community of
teachers
• From one subject to more subjects
• From one grade to more grades
• From one class (pilot) to more classes
• From one school to more schools
9

10. PD for 2 P4 sci PD for Same classes from 3
teachers in Mayflower scaled to
The 2 New cohort of more classes & schools continued;
Mayflower Pri science Pri 4 & 5 They started their
continued help other
teachers+ classes use own scalability
Derived schools;
2 Chinese GS; journey on GS;
July – Oct 10 RCKI
principles teachers Fuhua Sec used GS for
2007 P4 MFPS helped CL, Science and Math Researchers and the
Science other schools (2 classes); teachers from 3
10 principles adopt GS schools conducted
P5 Science,  6 principles; innovation; two workshops for
Whitley Sec used GS
math & CL Logic models teachers from 20
Jan – Nov for CL and Science (2
schools.
2008 classes).
Deeper understanding of
principles and Deep intervention in
P6 Science Teachers from 3
Jan – Mar SST
pedagogical patterns
More pedagogical patterns Schools did action
2009 experimented research on GS and
ETD worked with 6
shared experiences
April – Oct schools using GS
Sharpen understanding of the
Intensive GS
2009 principles & conditions for their
Sec 1 & 2 Sci, Math, CL interventions in 2
use via classroom data
schools in 2011.
Jan 2010
Future School-SST – CL & EL; Principles and pedagogical patterns
– now
GS interventions in 6 other schools , for language learning
more curricular products (Pri Sci & derived and deployed
Math. Sec Bio, Chem, & D & T)
10

11. Why scaling?
When do you decide what to scale?
• Research evaluation of GS in classes showed
– learning gains in subject matter,
– better communication,
– collaboration and critical thinking skills,
– ICT literacy,
– positive attitudes to subject learning;
– collaborative learning community and culture
building;
– teachers developed constructivist beliefs and
practices
11

12. How to scale?
• Diffusion
– Teacher CoPs
• Sharing RCKI principles, lesson plans,
materials, exemplar lesson videos & lesson
studies
• Supporting schools for GS installation and
running; software feature enhancements
12

13. New tools, materials & principles that
constitute the core of the innovation
• GS website (http://gs.lsl.nie.edu.sg/)
– RCKI principles & models
– Sample lesson plans
– Sample classroom videos
– Enactment strategies/tips
– Software download
13

14. The Wooden Barrel Metaphor:
木桶原理
Technology innovation
Researcher-Practitioner
Policies Tensions
Pedagogy Teacher Beliefs
Curriculum Assessment
Practices
Skills of Teachers
15

15. Who to scale?
• Top-down Or Bottom-up?
• MOE/School Administrators/Teachers
• A commercial or subsidized entity that will do
ongoing improvement of GS and provide
services for schools
16

16. Second innovation:
Mobile learning in a primary school
To give you some ideas, see this video
17

17. Mobile Learning
• What is to be scaled up?
– Mobilized curricula
• Leads to bridging informal learning spaces
• Leads to self-directed learning
– Lesson design principles
– Facilitation skills
– Teacher readiness
– Student readiness incl. hardware and software
training
– Technology infrastructure, e.g. WiFi, 3G
Connectivity
18

18. Mobile Learning
• Scale from what to what?
– From two science teachers to all science teachers in
level
• Why scaling?
– Research study of efficacies showed learning gains in
subject matter, positive attitudes to subject learning,
new media literacy, good learning habit – self-directed
learning
– More holistic learning with mobile device as a learning
hub to support seamless learning in classroom and
outside of classroom
– Teachers developed constructivist practices
19

19. Comparing our 2 experiences …
• Think about it
– Scaling ML/SL within a school seems more
challenging than scaling GS across schools
– Why?
• Within a school means spreading to all (and the
average) teacher
• Spreading the seeds of innovation to other schools
allows more flowers to bloom
• This is a simplistic analysis …. can’t generalize
…. More research can enlighten us
21

20. What is the problem or need?
22

21. 23

22. Crossing the chasm
(Moore, 1991)
24

23. Crossing the chasm in education
“Most research projects fail
to cross the chasm, and
though they may yield
valuable knowledge about
the nature of cognition,
teaching & learning, and
therefore inform the
research community, they
do not have a broad or
lasting impact on
mainstream K-12 education”
(Fishman, 2005)
25

24. Scaling seen as the solution to
addressing systemic change
• Define the problem of achieving systemic
change primarily as
• a problem of how to scale up effective
interventions (Mehan, Datnow & Hubbard,
2010)
26

25. Reconsidering scaling
• Adaptation of an innovation successful in
some local setting to effective usage in a wide
range of contexts (Dede, Honan & Peters,
2005)
• Different perspectives from practitioners,
policy makers & researchers
27

26. Question: I view scaling as a way to
extend reach of (good) innovations
to benefit more people, reduce
instances of re-inventing the wheel
and wastage of resources (e.g. time).
In your opinion, are there negative
consequences of scaling?
28

27. • Intervention-studies + Can’t scale
• Good intervention + Good scaling
– Leads to depth, sustainability and spread
• Good intervention + Bad or Ugly scaling
– Bad implementation fails to exploit potential of good
intervention
• Undeveloped or not-so-good-intervention + Good
scaling
– Work of practitioners in adapting interventions is
crucial for building generative capacity for sustainable
improvements 29

28. When innovations are complex …
• “Implementing programs with fidelity”
– Assumes a delivery standard to which local agents are
held accountable by external agents
– May work well with artifact-centric innovations
• More complex innovations require individual and
organization-wide learning and change
• A more organic conception of travel than simply
“build effective tools and then market them”
– Require developing human and social capacity for
their spread

29. How can we differentiate
"scaling" from "translation"?
31

30. Stokes (1997) Pasteur’s Quadrant
32

31. Translational research is BIG in
medicine
• In the US, NIH translational
research: 60 Clinical and
Translational Science Award
(CTSA) centers funded by 2012
with $500Mil per year
33

32. Type-1 and Type-2 translational research
• Institute of Medicine’s Clinical Research Roundtable
• T1:
– “the transfer of new understandings of disease
mechanisms gained in the laboratory into the
development of new methods for diagnosis, therapy, and
prevention and their first testing in humans.”
• T2:
– “the translation of results from clinical studies into
everyday clinical practice and health decision making.”
34

33. Type I and Type II Matrix: Translational Research in
the Learning Sciences (Penuel, 2009)
35

34. Different Styles of Classroom Research
(Sabelli, 2008)
What the Study The “System”
Entails Intention
Descriptor Definition Question
A new curriculum, May include
Innovation technology, material, etc. isolated classroom Does this work? Classroom only
and associated pedagogy studies
The use of that innovation Is this ready for
in one or more classrooms May include outside adoption and
Intervention Excluded
evaluation dissemination?
What are conditions for
Intervention Systematic study of an Includes extensive successful May be
Study intervention evaluations implementation? considered
Implemen-
tation Do sites learn from the
Research (or Ongoing work by the
The iterative study of work? Can they sustain Local structure
Design site and the
adaptation or localization and grow the considered
Research) researchers
intervention?
The aggregation of multiple What is the range of
Clinical implementation studies A common applicability of the Must be
Research framework intervention? considered 36

35. The direction matters!
• Approach scaling from micro-level (innovation
or intervention)
• Or approach scaling from macro-level (MOE
policy)
• What is the purpose of scaling?
– Address specific problem
– Address general problem
37

36. Why are university researchers ill-
equipped to do scaling studies?
38

37. Current State of Affairs of Ed R&D
(adapted from Bryk, 2007)
• They want their own identity
– Build or stick to something
– New theory dev is more valued than practical
solutions to real problems
• They don’t think it is their business to do scaling
• Tenure & promotion requires quick publishing
pace which are not consistent with long-term
perspective to scaling up work
• School districts operate in a short-tem reactive
env vis-à-vis innovation
39

38. Are there research components to
be tied to a scaling project? What
research questions do we
investigate/track/monitor?
Can scaling involve:
(1) Meta-studies?
(2) Intervention projects?
(3) Implementation project without
research or research lite (i.e. lesson
study, teacher reflections)?
40

39. We can take another perspective:
design-based implementation
research
41

40. Design-based Implementation
Research (Penuel, 2010)
• In the sister disciplines of medicine and public
health, DBIR has a robust infrastructure and a
clear focus on the interdisciplinary challenge
of bringing about large-scale improvements to
complex systems (Fixsen, Naoom, Blase, &
Friedman, 2005).
• Other names for this area of research include
– improvement research (Bryk, 2009),
– formative interventions (Engeström, 2008), and
– social design experiments (Gutiérrez & Vossoughi,
2010). 42

41. How is DBIR different from other R?
1. Adopt an intentional stance with respect to
anticipating and addressing recurring
problems of systemic change at multiple
levels and settings.
2. Pay attention both to theories of learning
and theories of implementation or change.
3. Attempt to prefigure new relationships
between research and practice through how
they organize their research efforts.
43

42. Forward-looking Implementation
Research
• Take both curricular interventions and
contexts of implementation as objects of
design and study
• Inform the process of system-level changes in
education in ways that improve
implementation of curricular interventions
44

43. APPLYING WHAT WE KNOW
(SABELLI, 2010)
• First, that we know enough about improving the
teaching and learning to engage in large-scale
implementation of the education we envision.
• Second, that we do not yet know enough about
expanding, disseminating, accessing, and
sustaining what we already know.
Based on these premises, we must engage in the
second:
• Implementation research — to achieve long
term, sustainable, improvements in education.
45

44. The terms 'ready-to-scale' &
'scalable' are often used to
characterise the state of an
innovation. How does one define or
measure the state of readiness to
scale or that something is scalable?
46

45. Coburn’s dimensions of scale (2003)
• Deep and consequential changes in
Depth practice
Sustainability • Maintaining these changes over
substantial periods of time
Spread • Diffusion
Shift in reform • Districts, schools and teachers to assume
ownership ownership
47

46. • Go back to Coburn’s dimensions
– Does it have depth?
– Does it have some form of established efficacy?
• It is relative
– We do not think that there is an exact
measurement for when to scale.
– The stakeholders decide
• DBIR takes a different perspective
48

47. Traps to avoid
(Sources: Dede, 2007 & Coburn, 2003)
Depth Sustainability Spread Shift
• Trap of • Trap of Mutation • Trap of • Trap of
Perfection • When adapting Optimality Origination
• Do not seek the innovation, • A somewhat less • Researchers
unattainable goal do not undercut powerful should not
of perfection its core innovation that attempt to
• The great should conditions for reaches much control the
not be the success greater numbers original
enemy of the of users in a step innovation that
good! forward! deter adaptation
& further
innovation by
users
49

48. What directions can we take to
improve sustainability and
scaling of innovations?
Ponder our next few steps …..
51

49. A New Vision
• Instead of defining the problem of achieving
systemic change as a problem how to scale up
effective interventions, adopt more collaborative
approaches to improvement (Mehan, Datnow &
Hubbard, 2010)
• Mutual Adaptation
– Long-term collaboration between researchers and
practitioners
• Use-inspired research that address broad-based
problems critical for ed improvement
52

50. Our Translation and Scaling Model

51. New Centre for Scaling, Translation
and Commercialization (STaC)
• devoted to
– the scholarship of the processes of translation and
scaling in education
– the incubation of potentials towards commercialization
• studies the design, enactment and scale-up of
interventions in various situations that provide
compelling evidence of what works and what does
not
• documents the benefits and trade-offs of
balancing fidelity of implementation with
adaptation to dynamic local contexts.
54

52. Steps …
• To understand scaling, do scaling
– Do a few scale-up/implementation projects
• Meta-studies of intervention projects
• Distill the conditions for successful scaling of
an intervention
– Target not system-wide
– But a school or a cluster of schools
– Scale adaptively based on school profiles
55

53. Back to this:
Change Knowledge (Michael Fullan)
“4 frogs are sitting on a log,
and one decides to jump off.
How many frogs are left?”
56

54. The End
Contact me at: cheekit.looi@nie.edu.sg