This document discusses evaluation practices and challenges in violence prevention. It provides examples of evaluations conducted on child sexual abuse prevention programs in Massachusetts and shaken baby syndrome prevention programs. It also discusses using evaluation to promote sustainability, dissemination, and teaching evaluation practices to social workers. Key challenges discussed include understanding stakeholder culture and complexity in real-world settings.

1. Evaluation Practice and Violence Prevention David S. Robinson, Ed.D. August 22, 2006 Add Corporate Logo Here Connecting Families - MSPCC Child Sexual Abuse Prevention Project

2. TOPICS Violence prevention projects & evaluation challenges in the field Evaluation for sustainability Evaluation for dissemination The flattening of the world (ala Thomas Freedman) Final Notes and Implications

3. Violence Prevention Projects Child Sexual Abuse Prevention Program Connecting Families – Child Abuse Prevention Evaluation Shaken Baby Syndrome Prevention Program Evaluation Children’s Emotional Health Link Evaluation Teaching practice and program evaluation for prevention science

4. Massachusetts Child Sexual Abuse Prevention Initiative Evaluation 4 Sites in Massachusetts using Enough Abuse Campaign resources and concepts Community participatory process designed to prevent child sexual abuse perpetration Workshops focus on child safety, how adults can take action to keep children safe, identifying risky situations and “red flags” for concerning behaviors, and understanding and responding to children’s sexual behaviors

5. Evaluation of the Massachusetts Child Sexual Abuse Prevention Partnership Initiative

6. METRICS - I MCSAPP Evaluation Partnership has increased knowledge of CSA prevention services and gaps in services. Identification of #, type, and location of CSA prevention programs through inventories Conduct statewide inventories of CSA prevention programs 2.4. Identify existing CSA perpetration prevention services and gaps in services over time. IMMEDIATE OUTCOMES OUTPUTS ACTIVITIES OBJECTIVES

7. METRICS - II MCSAPP Evaluation Survey of members indicating % received inventory, % recognize prevention service gaps Partnership has increased knowledge of CSA prevention services and gaps in services. Inventory results published and distributed to members Identification of #, type, and location of CSA prevention programs through inventories MEASURES IMMEDIATE OUTCOMES MEASURES OUTPUTS

8. METRICS - III GOAL 4. ASSIST LOCAL COLLABORATIVES IN APPLYING THE PUBLIC HEALTH MODEL TO UNDERSTAND THE NATURE AND SCOPE OF CHILD SEXUAL ABUSE, CONDITIONS THAT ALLOW IT TO OCCUR, AND STRATEGIES TO PREVENT IT IN THEIR COMMUNITIES. MCSAPP Evaluation Increased belief in importance of comprehensive response to CSA by stakeholders; Community specific understanding of nature and scope of CSA; Strong local collaboratives formed. # Training sessions (training agenda). # Meetings/forums on community scan. # Participants. # CSA reported & substantiated cases # Court adjudicated cases. Training on surveillance methods, data collection, review data, developed plan. 4.1. Assist each local collaborative in conducting injury surveillance of CSA in its community. IMMEDIATE OUTCOMES OUTPUTS ACTIVITIES OBJECTIVES

9. METRIC - IV MCSAPP Evaluation Surveys (three points in time) of beliefs, attitudes and knowledge. Review of meeting notices, agenda, minutes. Observations of meetings. Interview with chairs & sub-committee leaders. Increased belief in importance of comprehensive response to CSA by stakeholders; Community specific understanding of nature and scope of CSA; Strong local collaboratives formed. Review training agenda, sign in sheets, notices and agenda for community scans. Review documented CSA cases by type, conditions, results. Review plan derived from surveillance activities. # Training sessions (training agenda); # Meetings/forums on community scan. # Participants. # CSA reported & substantiated cases # Court adjudicated cases. MEASURES IMMEDIATE OUTCOMES MEASURES OUTPUTS

10. Connecting Families: MSPCC/MDSS/Brandeis Problem – research suggests that many children with unsupported/not confirmed child abuse investigations are re-reported within 12 months. Beginning in 2003, phased in two studies – randomize control group and matched control 600 families referred in 2005 Half of caregivers & children with positive mental health screening Over 1/3 of families have previous supported or unsupported investigation for child abuse.

11. Connecting Families – MSPCC/MDSS/Brandeis

12. Figure 1. Outcome Evaluation Design Difference in Fall River and Other Pilot Sites FALL RIVER EVALUATION DESIGN Families offered CF by DSS office (E) Families in matched DSS area offices without CF program (C) 2000 2001 2002 2003 2004 2005 2006 2007 Compare (Matched Control) Compare (Historical Control)

13. EVALUATION DESIGN FOR OTHER FIVE PILOT SITES All eligible families referred to CF by DSS staff Randomize Families offered CF (E) Families not offered CF – usual DSS response (C) Compare E and C

14. Shaken Baby Syndrome In-Hospital Prevention Program Evaluation Massachusetts Citizens for Children developed the Shaken Baby Syndrome Prevention curriculum which was approved by the Massachusetts Association of Registered Nurses in 2005 Nurses receive 1.2 Continuing Education hours upon completion of the SBS Prevention education program Nurses at the Worcester County birthing hospitals invite parents to watch a video on SBS and give parents a pamphlet about the dangers of shaking infants before they leave the hospital Nurses are encouraged to talk to each caregiver or parent to answer questions about the SBS prevention information By December 2005, over 262 nurses and 3076 parents and caregivers of newborns in four hospitals in Worcester County have been trained.

15. 2005 Evaluation Design Historical comparison using SBS reports and ICD coded injuries 5 years prior to 2006 (full implementation year), and prospectively through 2008 Comparison groups made up of comparable hospital sites without SBS prevention program Injuries annually reported prior to SBS prevention implementation Injuries annually reported after SBS prevention implementation Reports & Index conditions in comparable sites without SBS prevention program SBS Intervention 2000 2006 2008

16. Children’s Emotional Health Link www.cehl.org This web site is intended to serve a dual purpose. It contains articles to empower parents, advocates and providers in caring for the psychosocial needs of children. It also serves as a coordinating tool for a pilot project for health care providers. Resources for physicians, advocates and parents Focus on helping physicians initiate difficult conversations with parents about family violence, emotional problems, traumatic health conditions. Evaluation of unique pilot study to improve identification and response to family violence and emotional problems in children

21. Teaching Prevention Evaluation to Graduate Social Workers Principles of prevention and evidence-based programming Participatory and empowerment evaluation principles Combination of case study and placement evaluation planning & presentation Focus on program theory, logic model, research design, measurement, understanding stakeholder culture and setting complexity

22. Framework for Understanding Practice & Program Evaluation

23. http://cecp.air.org/guide/websites.asp

25. Evaluation for Sustainability The Devolution Initiative – WKKF ( http://www.gse.harvard.edu/hfrp/pubs/onlinepubs/sustainability/planning.html ) Advanced Tools for Sustainability Assessment ( http://ivm5.ivm.vu.nl/sat/ ) Center for Civic Partnerships ( http://www.civicpartnerships.org/docs/tools_resources/sustainability.htm ) 10 Steps to Sustainability.

26. Evaluation for Dissemination What if Google and CDC DVP created “Google Violence Prevention” with database of local VP projects and case studies of dissemination activities? How would Syndemics and syndemic approach change implications for evaluation, dissemination and information management? What are other countries doing to disseminate information about VP and what problems are they working on and solving?

27. Flattening of the World Thomas Friedman’s thesis: it is now possible for more people than ever to collaborate and compete in real time with more other people on more different kinds of work from more different corners of the planet and on a more equal footing than at any previous time in the history of the world – using computers, e-mail, networks, teleconferencing, and dynamic new software. Open-sourcing: self-organizing collaborative communities; Outsourcing Offshoring Supply-chaining Insourcing In-forming Increasing speeding of Computational capability, storage capability, and input/output capability

28. Final Notes & Implications Opportunities for strengthening evaluation designs in real world settings Connecting learning, implementation, infrastructure and dissemination Strengthening appearance and reality Understanding and using the resistance Using the new connectivity as a tool

29. David S. Robinson, Ed.D. Faculty, Simmons College School of Social Work [email_address] President, DSRobinson & Associates 903 Providence Place, #155 Providence, RI 02903 O: 401-383-7953 M: 617-733-5979 [email_address] www.evaluationhelp.com