Leverage points are simply identifying effective ways where small changes may have great impacts. Often, these points are simple actions that may fall outside the complex analysis patterns commonly considered in evaluation. The diagram suggests that quality teaching/learning innovations may move the inertia of school systems.
The ITSI-SU project, Innovative Technology in Science Inquiry Scale-Up, helps teachers develop inquiry-based science projects that use probes and models in grades 2-12 classrooms across four states. Teacher professional development is conducted via a one-week in-person summer workshop and two five-week online courses throughout the year, a situation that fosters the building and sustaining of an online learning community. Approximately 100 new teachers join each year.
My fuzzy logic model changes the typically static directional arrows into project verbs: the main arrows (red) are professional development activities, subsequent flow arrows are experiences, all of which ultimately lead toward contributions, which are cycled back to the model’s start point.
Donella Meadows outlines 12 systems levers to identify and understand. Here I will focus on four, highlighted in red: Delays, Reinforcing Feedback Loops, Information Flows, and Paradigms.
Projects have a natural time flow and optimal sequence of events. Delays, arguably expected in the best of circumstances, will impact how the project responds to the give and take of information and inputs.
If a delay is too short, that is, an unnaturally accelerated response to an intervention, the results (and findings) will be inaccurate. In this example, if all test scores immediately rise after using probes, the students might be re-tested at a later date to ascertain longer-term gains.
If, on the other hand, a delay is too long, the results might be irrelevant. Here, student standardized test scores are reported much too late to have any bearing on curriculum change or diagnostic purpose.
A classroom delay in ITSI occurred as a result of student data from online assessments completed as part of student projects was not reported back to teachers until months after the classroom activities, at which point students and teachers were on to different topics, rendering the findings irrelevant. The leverage point from this lesson would be to provide next-day analysis of student assessments so teachers might use the data immediately. Small shift = large gain.
A regional delay might prove a barrier to widespread adoption of the ITSI model, which, after the funded project period, will switch to a pay-for-service model. The time frame of districts’ abilities to respond to quality work may be too long for ramping up the project independently; by the time connections are made between student test results and ITSI practices, the district may be thinking about other opportunities.
Another lever I examine is that of the reinforcing feedback loop. Looking at the examples above, the more something works, the more it gains power to work some more, both positively and negatively: the more people have the flu, the more will get infected, which will produce more flu and infections; the more money you have, the more you’ll make, etc.
A startling realization occurred to me after careful analysis of the teacher online coursework and related participation and contributions. Course participation requires a certain amount of online blogging, usually 2 posts per week; teachers are partnered with others from out-of-district to foster a community of national scope. What I discovered was that those teachers who were doing the highest quality projects in their classes were often the teachers who blogged a lot; in other words, if you posted a lot, your class did better work.
In this analysis, you see a reasonable correlation among frequency of blog posts (blue) and three dimensions of classroom projects frequency of enacting ITSI activities, extent of student engagement, and likelihood to continue ITSI after the support year. This shows scores for ten teachers (across bottom) using a 0-5 point quality scale.
This chart shows the blog post frequencies (blue) compared with the means of the other three dimensions
This analysis leads to the the simple suggestion that teachers’ overall performance may very well improve if they post more frequently; they will experience increased support, increased sense of community, which will lead toward increased motivation, leading to improved classroom experiences, which will predict increasing likelihood to continue.
A significant lever in school work is Information Flows. We see this a lot when implementing technology projects; the teacher is dependent on an IT expert who may not be the most willing to offer timely support, especially if the technology is somewhat new or unusual (as it always is). Often a project is halted mid-stream because of a simple gap in the information flow. Trying to identify and address this often has “big bang-for-buck” results.
Information Flow issues in ITSI I discovered included simple access to the school network, getting through firewalls that are designed specifically to block out the kind of data ITSI needs to support its software (send/receive student personal project data), teacher comfort with technology (everything from logging in to setting up probeware), and many computers logging onto the local network simultaneously, taxing its capacity. Addressing these findings had more positive impact on how the project ran than design, curriculum, or content issues.
Finally, the leverage points of Paradigms: how to change them, have not yet been realized in ITSI. We are seeing evidence of trace movement along some strands (classroom use of technology and data, inquiry-based learning practices, teacher community-building), but nothing of grain size big enough to measure; something we hope to see by the end of the project.
Leverage Points: Places to Intervene in aSystem David Reider, Principal Partner Education Design, INC October, 2012
Leveraging Power or IdeasPositional advantage; power to act effectively T/L INNOVATIONSSCHOOL SYSTEMS
ITSI-SU Project Innovative Technology in Science Inquiry Scale Up ITEST (NSF), Concord Consortium Alaska, Iowa, Virginia, Kansas state cohorts >250 teachers Year 3/5 Web-based science simulations, probeware, modeling for gr. K-12 students Teacher online community Online assessment systems
Systems LeversSmall Movement: Large Effects 12 Systems Levers: Donella Meadows • Numbers •Information Flows • Buffers • Rules, Incentives • Stock and Flow Structures • Self-Organization •Delays • Goals/Purpose of system • Balancing Feedback Loops •Paradigms •Reinforcing Feedback Loops • Transcending paradigms
Lever: Delays Delays are critical relative to the rate of change in the stocks the feedback loop is controlling: You’re looking for oscillations, as any system will have Too short a delay: overreaction Too long a delay: sustained or exploding oscillations
Delays: Too Short Too short a delay: overreaction Example: all the kids are doing online science and using probes immediately, their test scores rise immediately) what’s wrong with this data?
Delays: Too Long Too long a delay: sustained or exploding oscillations Feedback that is irrelevant or useless Example: typically school system feedback occurs only as a function of standardized test results; is there anything the intervention can do to shift this focus?
ITSI Delay: Classroom Student content knowledge: Teachers have students take online assessments pre/post before each unit. For the research purposes of the grant, sent back to CC. Not analyzed until after several months, then reported to PI, or following year. Leverage point: If feedback were immediate (within days), teachers would refine instruction and implementation designs the next time (e.g. INK-12 immediate laptop response 4th grade math)
ITSI Delay: Regional Local/regional adoption: In a district, the response and feedback of implementation e.g. Anchorage: (how many teacher Yr 1, Yr 2, Yr 3) grows; as a function of hearing about earlier cohorts Hypothetical: district adoption into a pay-service model; they want to see results in their language: standardized science exam scores, but by the time these will be analyzed, reported, and tied to ITSI classroom activities (2- 10 contact days/year), will be irrelevant, too late after project funding. Leverage point: difficult to enact b/c funding timeline, needs several years of repeated practice
Lever: Reinforcing Feedback Loops A reinforcing feedback loop is self-reinforcing: the more it works, the more it gains power to work some more, driving system behavior in one direction, either positively or negatively more people have flu, the more will get infected, more will get the flu and infect more the more money you have, the more you’ll make, you invest more, you make more more the polar ice cap melts, the less reflected albedo decreases, less radiant energy reflected back, more heat retained, more ice melts
ITSI: Reinforcing Feedback Loop ITSI: teachers online course: 5 consecutive weeks Fall & Spring Required participation, includes assignments, readings, peer review of lessons. Lesson plan posting and blog area for discussion/response Required 2 posts/week, some participate more than others More they blog, the more they’ll receive a peer response, the more they’ll blog/contribute, inevitably positioning them to implement better; the less they participate, the least likely they are to engage fully, build community, they are then left out, implement poorly
ITSI: Reinforcing Feedback LoopITSI: teachers online course: 5 consecutive weeks Fall & SpringRelationship of Frequency of blog posts to: Student engagement Frequency of ITSI activities likelihood to continue
ITSI: Reinforcing Feedback Loop Blog frequency: class dimension means
ITSI: Reinforcing Feedback Loop How is this useful? By encouraging teachers simply to post (anything, even trivial), their overall performance might improve as a result of increased support sense of communityincreased motivationimproved classroom experiences with program increased likelihood to continue
Lever: Information Flows The structure of who does and does not have access to information Missing information flows are one of the biggest causes of system malfunction
ITSI: Information Flows Technical—online access, getting computer lab to work (IT/tech director or school firewall, etc.) Fundamental computer comfort; this knowledge or lack of will considerably impede project advance Case of school with poor access, took months, until CC visited, got online between programmer at CC and IT person; things fixed in ½ hour; Understanding this information flow helps design issues
The Tough Levers: Paradigms Highly valued lever of Paradigms: how not realized or observed yet. What that might look like Why deep change Why not seen yet What it might look like Is it ok as an evaluator to expect to see this?