Karen Horsch Evaluating Change V.2


Published on

From the Harvard Pilgrim Growing Up Healthy conference (11/4/09)

Published in: Technology
  • Be the first to comment

  • Be the first to like this

No Downloads
Total views
On SlideShare
From Embeds
Number of Embeds
Embeds 0
No embeds

No notes for slide

Karen Horsch Evaluating Change V.2

  1. 1. Evaluating Change<br />Karen Horsch<br />Evaluation Consultant<br />
  2. 2. We are experiencing epidemic rates of childhood obesity that have long-term consequences for health and well-being of individuals and society<br />It’s getting worse, not better<br />What we know:<br />
  3. 3. Reversing this trend in a sustained way is going to take fundamental changes to the way we think and act: we cannot program our way out of this<br />Reversing this trend in a sustained way is going to require that we build a movement<br />What we know:<br />
  4. 4. We need to shift perceptions such that physical activity and healthy eating are the social norm<br />We need to establish environments that make individuals’ default decisions healthy<br />What needs to happen:<br />
  5. 5. They raise issues of urgency (identify problems)<br />They build a base of interested and motivated individuals and coalitions<br />They educate and raise awareness<br />They advocate for positive changes in policies and institutions<br />They identify and promote promising interventions and programs (show what’s possible)<br />What movements do….<br />
  6. 6. How does evaluation fit in?<br />
  7. 7. Data about the movement (is the movement moving?)<br />Data to build the movement<br />Two levels of evaluation when building a movement:<br />
  8. 8. So how do we evaluate whether we are building a movement?<br />
  9. 9. NOT: What did we accomplish?<br />BUT: Did we accomplish what we set out to <br /> accomplish? Why or why not?<br />Start with the right evaluation question:<br />
  10. 10. The “theory of change” about how to build a movement<br />Laying out what we hope to accomplish:<br />The Strategies and Interventions <br />The Results we hope to achieve:<br /><ul><li> Short-term
  11. 11. Intermediate term
  12. 12. Long-term</li></li></ul><li>Strategies<br /><ul><li>Coalition building
  13. 13. Grassroots organizing/mobiliz-ation
  14. 14. Leadership development
  15. 15. Advocacy training
  16. 16. Media advocacy
  17. 17. Identification & replication of proven programs</li></ul>Short-Term Outcomes<br /><ul><li>Increased awareness about and prioritization of the issue
  18. 18. Strong and diverse network of advocates
  19. 19. Strengthened alliances
  20. 20. Increased media attention to the issue
  21. 21. Established relationships with key decision-makers</li></ul>.<br />Intermediate-Term Outcomes<br /><ul><li>Increased public and policymaker support for policies and actions that promote positive changes in the issue
  22. 22. Actions supporting physical activity and healthy eating:</li></ul>-- more funding<br />-- more programs<br />-- policy change<br />-- changes in institutions<br />Long-Term Outcomes<br /><ul><li>Behavior changes: increased physical activity and healthy eating and reduced screen time
  23. 23. Reductions in obesity rates
  24. 24. Improved health</li></li></ul><li><ul><li>Who the decision makers are
  25. 25. Who the public is
  26. 26. Who partners might be</li></ul>Being clear about: <br />
  27. 27. Increased awareness about and prioritization of the issue:<br /><ul><li>Information sent out and overall message framing
  28. 28. Number and type of people who receive information
  29. 29. Attendance at meetings/events
  30. 30. Website hits
  31. 31. Public opinion surveys</li></li></ul><li>Strong and diverse network of advocates:<br /><ul><li>Number and type of people trained in advocacy
  32. 32. Number and type of people taking on leadership roles in advocacy
  33. 33. Number of new advocates
  34. 34. High profile people who adopt the issue and publicly advocate for it
  35. 35. Actions taken by advocates</li></li></ul><li>Number and type of coalitions/partnerships established<br />Number of active members of coalition<br />Health of coalitions<br />Number and type of cross-organization and cross-sector partnerships/collaborations<br />Network analysis<br />Strengthened alliances: <br />
  36. 36. Number of press releases<br />Number of op-eds/letters to the editor published<br />Number of articles covering the issue<br />Relationships with the press<br />How the issue is framed in media<br />Increased media attention to the issue: <br />
  37. 37. Visits to decision makers<br />Number of decision makers attending events related to issue<br />% of phone calls returned by decision makers<br />Established relationships with decision makers: <br />
  38. 38. Number of decision makers and/or opinion leaders who express support for the issue<br />Policy makers’ press releases and comments in legislative hearings<br />Policy maker attendance at hearings related to the issue<br />Increasing support for:<br />
  39. 39. New policy proposals<br />New policies passed<br />Policies expanded/or not reversed<br />Implementation of policies<br />Improving Policies: <br />
  40. 40. Balancing the data collection effort with the utility of the information collected<br />It’s not all countable: the importance of qualitative data<br />Data Considerations<br />
  41. 41. So what data do we need to build the movement?<br />
  42. 42. Obesity as connected to environments and not solely individual behavior<br />Role of government in this<br />Importance of prevention rather than long-term health consequences<br />Data that helps to reframe the issue:<br />
  43. 43. Population-level data collection<br />Community mapping and other visuals<br />The cost of doing nothing<br />Data that points to the urgency of the issue and its causes:<br />
  44. 44. What programs work and how well<br />How many they serve<br />The outcomes they achieve<br />What programs cost or save<br />Data about what works (Program Evaluation)<br />
  45. 45. “Policy change occurs when community leaders receive credible and reliable data and research AND community members provide personal stories and advocate for change.”<br />-- Kay Monaco, New Mexico Voices for Children<br />
  46. 46. Policymakers receive a lot of information<br />Easy visuals: mapping, metrics, summaries<br />Personal stories and testimonials matter<br />Timeliness and credibility are critical<br />Policymakers as Audience<br />
  47. 47. Climate ready<br />For change<br />Source: PolicyLink<br />