This document discusses evaluation practices and challenges in violence prevention. It provides examples of evaluations conducted on child sexual abuse prevention programs in Massachusetts and shaken baby syndrome prevention programs. It also discusses using evaluation to promote sustainability, dissemination, and teaching evaluation practices to social workers. Key challenges discussed include understanding stakeholder culture and complexity in real-world settings.
2 Integrating Child Protection and Community Engagement, Sierra Leone, Kathle...
DSRA
1. Evaluation Practice and Violence Prevention David S. Robinson, Ed.D. August 22, 2006 Add Corporate Logo Here Connecting Families - MSPCC Child Sexual Abuse Prevention Project
2.
3.
4.
5. Evaluation of the Massachusetts Child Sexual Abuse Prevention Partnership Initiative
6. METRICS - I MCSAPP Evaluation Partnership has increased knowledge of CSA prevention services and gaps in services. Identification of #, type, and location of CSA prevention programs through inventories Conduct statewide inventories of CSA prevention programs 2.4. Identify existing CSA perpetration prevention services and gaps in services over time. IMMEDIATE OUTCOMES OUTPUTS ACTIVITIES OBJECTIVES
7. METRICS - II MCSAPP Evaluation Survey of members indicating % received inventory, % recognize prevention service gaps Partnership has increased knowledge of CSA prevention services and gaps in services. Inventory results published and distributed to members Identification of #, type, and location of CSA prevention programs through inventories MEASURES IMMEDIATE OUTCOMES MEASURES OUTPUTS
12. Figure 1. Outcome Evaluation Design Difference in Fall River and Other Pilot Sites FALL RIVER EVALUATION DESIGN Families offered CF by DSS office (E) Families in matched DSS area offices without CF program (C) 2000 2001 2002 2003 2004 2005 2006 2007 Compare (Matched Control) Compare (Historical Control)
13. EVALUATION DESIGN FOR OTHER FIVE PILOT SITES All eligible families referred to CF by DSS staff Randomize Families offered CF (E) Families not offered CF – usual DSS response (C) Compare E and C
29. David S. Robinson, Ed.D. Faculty, Simmons College School of Social Work [email_address] President, DSRobinson & Associates 903 Providence Place, #155 Providence, RI 02903 O: 401-383-7953 M: 617-733-5979 [email_address] www.evaluationhelp.com
Editor's Notes
Strong and Permanent State-Level Collaboration. Infrastructure: Broad, diverse, multidisciplinary cadre representing public and private groups A comprehensive State-Level Perpetration Prevention Response. Program Implementation: (1) develop action agenda (logic model); (2) Survey current attitudes (statewide & local); (3) Approve standards (treatment, referral, accountability); (4) Improve CSA knowledge (statewide conference and training of trainers); (5) Identify program services & gaps (statewide inventories). A Network of Three Permanent Local Collaboratives. Local Infrastructure: 3 broad, diverse collaboratives committed to implementing comprehensive CSA prevention & evaluation. Apply Public Health Model in Local Pilot Sites. Program Planning: (1) Injury surveillance (data collection & review); (2) Risk protective factors (individual & community); (3) Program development (community action plan). Comprehensive perpetration prevention responses in three local collaboratives. Program Implementation: (1) Public education about CSA )prevention booklet, media messages); (2) Increasing knowledge & skills for parents/adults (training with standardized curricula, pre- & post surveys); (3) Increasing knowledge and skills for professionals (training of targeted professionals, e.g., childcare providers, educators).
Statewide inventory using self-selection, snowball sampling, advertisements, telephone survey, committee review and decisions about inclusion. Replicate annually.
Dr. King had little experience with logic models and theory of change models. Introduced him to logic of program and possible outcomes. Revised three times based on his comments and reflections. Developed measures for practitioners and parents based on his extended list of outcomes and pilot tested in 2004-2005 Recently awarded grant to refine and replicate for 2006-2007. New seminar trainer with doctor-patient communication research and teaching leading. Modifying outcomes, indicators, measures for 2007 delivery.
We discuss with them the principles of prevention and what we know about evidence-based prevention programs.
From SAMHSA’s CSAP
6.4 Dissemination Dissemination is the active, purposeful process of knowledge transfer. Like evaluation processes, dissemination requires resources, infrastructure and planning and is essential in the feedback link to informing future planning (see Figure 3, Section 4.1). Reviews of the dissemination processes for findings from health promotion practice indicate that these processes are complex, easily underestimated and often devoid of deliberate and systematic approaches. It is also noted that many health promotion programs in the past were not disseminated widely or findings were disseminated prematurely, limiting the full evidence of effectiveness being recognised or shared.100 Key findings and learnings can be disseminated via a range of strategies, such as training through workshops, train-the-trainer and continuing professional education; communication through print; communication through video and computer technologies such as databases of good practice stories, library search systems and websites; personal face-to-face contacts; consultancies; policies, administrative arrangements and funding incentives; committees and other decision-making structures; and collaborative applied research programs. The stages of dissemination can be summarised as: 1. Providing and seeking information. 2. Persuasion about the relevance and applicability of the innovation or findings. 3. Making a decision to adopt the findings or try the innovation. 4. Changing practices and using the innovation. 5. Sustaining the changed practice.101 Toolkit: For further information on dissemination see two key Australian references called: King, Hawe and Wise (1996) From research into practice in health promotion: a review of the literature on dissemination, Sydney. ISBN: 1 86451 228 8 Oldenburg B et al (1997) The dissemination effort in Australia: strengthening the links between health promotion research and practice, School of Public Health, Queensland University of Technology. Publication Identification No. 2182 Members of the Australian Health Promotion Association will be able to download these references from the Association’s website at http://www.healthpromotion.org.au The Department plays an active role in disseminating integrated health promotion practice examples, evidence and tools. See http://www.dhs.vic.gov.au/phkb under Health Promotion for further information. This guide also features a range of good practice examples from PCP funded health promotion programs. It is planned that more examples will be developed and disseminated as updates to the guide. In 2003–04, the Department will also support the development and dissemination of good practice case studies. These will be disseminated in partnership with VCHA as part of the QIPPS initiative.
Thomas L. Friedman, 2005, The World is Flat: A Brief History of the Twenty-first Century, Farrar, Straus and Giroux, NY “ It is this triple convergence – of new players, on a new playing field, developing new processes and habits for horizontal collaboration – that I believe is the most important force shaping global economics and politics in the early twenty-first century. Giving so many people access to all these tools of collaboration, along with the ability through search engines and the Web to access billions of pages of raw information, ensures that the next generation of innovations will come from all over Planet Flat. The scale of the global community that is soon going to be able to participate in all sorts of discovery and innovation is something the world has simply never seen before.” p 181-2