Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
+Doing DevelopmentalEvaluation at theSystem Level:Lessons learned froman addiction systemchange initiativeTuesday, June 11...
+Agenda An overview of the DTFP Evaluation DTFP Evaluation Methods DTFP Evaluation Results Developmental Evaluation at...
+Drug Treatment Funding Program Addiction treatment system change Health Canada-funded project, delivered through the On...
+DTFPFundingSelection ofProjects &OrganizationsProject ImplementationTesting &validationDevelopmentof Tools andresourcesDe...
+Developmental Evaluation of DTFP Incomplete knowledge 1) Spread processes within the system need to be developed 2) Re...
+Developmental Evaluation of DTFP Purpose To bring evaluative thinking to DTFP and to help DTFP to evolveand create chan...
+Results Enhancements seen in evaluation and performancemeasurement, evidence-informed practice, knowledgeexchange projec...
+Results Themes Define the system Pay attention to dynamics Lack of ownership Tools themselves will not create change...
+Developmental Evaluation Challenges Multiple simultaneous impacts Projects having limited sphere of direct control Tim...
+Lessons Learned Focus on learning Developingrelationships Rapid feedback Redundancy in agood thing
+Lingering questions
+Thank youQuestions? Comments?Jenna van Draanenvandraanenj@smh.cawww.torontoevaluation.ca
Upcoming SlideShare
Loading in …5
×

Ces 2013 - Doing Developmental Evaluation at the System Level

369 views

Published on

from Jennifer von Draanen

Published in: Business, Education
  • Be the first to comment

  • Be the first to like this

Ces 2013 - Doing Developmental Evaluation at the System Level

  1. 1. +Doing DevelopmentalEvaluation at theSystem Level:Lessons learned froman addiction systemchange initiativeTuesday, June 11th, 2013Jenna van Draanen, Kaileah McKellar, Sanjeev Sridharan
  2. 2. +Agenda An overview of the DTFP Evaluation DTFP Evaluation Methods DTFP Evaluation Results Developmental Evaluation at the System Level Lessons learned Lingering questions for the field of evaluation
  3. 3. +Drug Treatment Funding Program Addiction treatment system change Health Canada-funded project, delivered through the OntarioMinistry of Health 11 system change projects (3 streams) Multiple project leads and organizations 5 year timeline compressed to 18 months
  4. 4. +DTFPFundingSelection ofProjects &OrganizationsProject ImplementationTesting &validationDevelopmentof Tools andresourcesDevelopment ofInter-organizationalNetworksSystem-level evaluation& Evaluation capacitybuildingUptake&SpreadSystem Changeand practiceimprovement• Increased accessand use ofevidence• EnhancedCollaborativeProblem-SolvingCapacity• EnhancedEvaluative CulturePlanning forsustainingnetworksCollaboration & Co-DevelopmentKnowledge ExchangeSphere of DirectControlCollaborativeActivitiesEvaluationActivities4Project-levelactivitiesSphere of DirectInfluenceSphere of IndirectInfluence
  5. 5. +Developmental Evaluation of DTFP Incomplete knowledge 1) Spread processes within the system need to be developed 2) Relationships between different stakeholder groups need to bestrengthened.
  6. 6. +Developmental Evaluation of DTFP Purpose To bring evaluative thinking to DTFP and to help DTFP to evolveand create change Multiple and flexible methods Key-informant interviews, case studies, surveys, capacity-buildingworkshops, demand mapping, observation, videography,document review, realist synthesis, and timeline of impactexercises Heavy engagement What needs to be developed?
  7. 7. +Results Enhancements seen in evaluation and performancemeasurement, evidence-informed practice, knowledgeexchange projects At the project level Case studies giving insight into the system performance
  8. 8. +Results Themes Define the system Pay attention to dynamics Lack of ownership Tools themselves will not create change Planning for spread8
  9. 9. +Developmental Evaluation Challenges Multiple simultaneous impacts Projects having limited sphere of direct control Timeline of impact Providing feedback in an environment of uncertainty
  10. 10. +Lessons Learned Focus on learning Developingrelationships Rapid feedback Redundancy in agood thing
  11. 11. +Lingering questions
  12. 12. +Thank youQuestions? Comments?Jenna van Draanenvandraanenj@smh.cawww.torontoevaluation.ca

×