+Doing DevelopmentalEvaluation at theSystem Level:Lessons learned froman addiction systemchange initiativeTuesday, June 11...
+Agenda An overview of the DTFP Evaluation DTFP Evaluation Methods DTFP Evaluation Results Developmental Evaluation at...
+Drug Treatment Funding Program Addiction treatment system change Health Canada-funded project, delivered through the On...
+DTFPFundingSelection ofProjects &OrganizationsProject ImplementationTesting &validationDevelopmentof Tools andresourcesDe...
+Developmental Evaluation of DTFP Incomplete knowledge 1) Spread processes within the system need to be developed 2) Re...
+Developmental Evaluation of DTFP Purpose To bring evaluative thinking to DTFP and to help DTFP to evolveand create chan...
+Results Enhancements seen in evaluation and performancemeasurement, evidence-informed practice, knowledgeexchange projec...
+Results Themes Define the system Pay attention to dynamics Lack of ownership Tools themselves will not create change...
+Developmental Evaluation Challenges Multiple simultaneous impacts Projects having limited sphere of direct control Tim...
+Lessons Learned Focus on learning Developingrelationships Rapid feedback Redundancy in agood thing
+Lingering questions
+Thank youQuestions? Comments?Jenna van Draanenvandraanenj@smh.cawww.torontoevaluation.ca
Upcoming SlideShare
Loading in …5
×

Ces 2013 - Doing Developmental Evaluation at the System Level

304 views
241 views

Published on

from Jennifer von Draanen

Published in: Business, Education
0 Comments
0 Likes
Statistics
Notes
  • Be the first to comment

  • Be the first to like this

No Downloads
Views
Total views
304
On SlideShare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
0
Comments
0
Likes
0
Embeds 0
No embeds

No notes for slide
  • Clarifying the nature of the addiction treatment systemAchieving change in a climate of fragmentationBranding DTFP as Quality Improvement InitiativeChanging the system from an output to outcomes culture Needing to better understand incentives and clarify capacitiesPaying attention to supply and demandNeed to clarify ownership of the projectUncertainty around key partner roles and expectationsReluctance to change roles over timePlanning for use - tools in themselves will not bring about change.Need to embed spread processes into Accountability agreementsUtilize or develop bottom-up models of spreadNeed to clarify the role of various actors in spread efforts Need to fund spread and sustain activities for long enough to see change
  • How can evaluators assess readiness for developmental evaluation in a system of actors?Comfort with nonlinearity, emergence, unpredictability, and atypical evaluationWhat is the role of evaluation in disrupting systems and encouraging change?How can evaluators foster co-evolution with partners who are not ready for developmental approaches?
  • Ces 2013 - Doing Developmental Evaluation at the System Level

    1. 1. +Doing DevelopmentalEvaluation at theSystem Level:Lessons learned froman addiction systemchange initiativeTuesday, June 11th, 2013Jenna van Draanen, Kaileah McKellar, Sanjeev Sridharan
    2. 2. +Agenda An overview of the DTFP Evaluation DTFP Evaluation Methods DTFP Evaluation Results Developmental Evaluation at the System Level Lessons learned Lingering questions for the field of evaluation
    3. 3. +Drug Treatment Funding Program Addiction treatment system change Health Canada-funded project, delivered through the OntarioMinistry of Health 11 system change projects (3 streams) Multiple project leads and organizations 5 year timeline compressed to 18 months
    4. 4. +DTFPFundingSelection ofProjects &OrganizationsProject ImplementationTesting &validationDevelopmentof Tools andresourcesDevelopment ofInter-organizationalNetworksSystem-level evaluation& Evaluation capacitybuildingUptake&SpreadSystem Changeand practiceimprovement• Increased accessand use ofevidence• EnhancedCollaborativeProblem-SolvingCapacity• EnhancedEvaluative CulturePlanning forsustainingnetworksCollaboration & Co-DevelopmentKnowledge ExchangeSphere of DirectControlCollaborativeActivitiesEvaluationActivities4Project-levelactivitiesSphere of DirectInfluenceSphere of IndirectInfluence
    5. 5. +Developmental Evaluation of DTFP Incomplete knowledge 1) Spread processes within the system need to be developed 2) Relationships between different stakeholder groups need to bestrengthened.
    6. 6. +Developmental Evaluation of DTFP Purpose To bring evaluative thinking to DTFP and to help DTFP to evolveand create change Multiple and flexible methods Key-informant interviews, case studies, surveys, capacity-buildingworkshops, demand mapping, observation, videography,document review, realist synthesis, and timeline of impactexercises Heavy engagement What needs to be developed?
    7. 7. +Results Enhancements seen in evaluation and performancemeasurement, evidence-informed practice, knowledgeexchange projects At the project level Case studies giving insight into the system performance
    8. 8. +Results Themes Define the system Pay attention to dynamics Lack of ownership Tools themselves will not create change Planning for spread8
    9. 9. +Developmental Evaluation Challenges Multiple simultaneous impacts Projects having limited sphere of direct control Timeline of impact Providing feedback in an environment of uncertainty
    10. 10. +Lessons Learned Focus on learning Developingrelationships Rapid feedback Redundancy in agood thing
    11. 11. +Lingering questions
    12. 12. +Thank youQuestions? Comments?Jenna van Draanenvandraanenj@smh.cawww.torontoevaluation.ca

    ×