Patton suggests five potential uses of developmental evaluation Describe the context – why we choose a developmental approach for our evals. Kathleen focused on #3 for grapes Sarah focused on #5 for network literacy as their goal is to truly change how people work and one of the overal questions is how this is impacting the system.
Proposed model for developmental evaluation. Kathleen can discuss overall model.
Sarah – this is where the rubber hits the road and my challenges really emerged – CoP leadership wanted and went looking for a different kind of evaluation but once we got the project started it was difficulty to get folks to fully participate. Work closely with the leadership and constantly bringing data / information to them which they are receptive of but getting some action is challenging. We are still in formative phase. Kathleen discuss her process.
Each discuss our data collection processes. Sarah with netlit See as a constant looping rather than a linear process – social media data is critical and regularly monitored and fed back to the leadership team. To bein to get a sense of what is being developed held reflective practice sessions to leading to questions for the membership so conducted focus groups. Information analyzed, shared and turned into recommended actions. Team is working on next steps
Using a Developmental Approach to Evaluating Innovative Extension Programs
Using a DevelopmentalUsing a DevelopmentalApproach to EvaluatingApproach to EvaluatingInnovative ExtensionInnovative ExtensionProgramsProgramsKathleen Kelsey, Grape CoPOklahoma State UniversitySarah Baughman, Network Literacy CoPeXtension
1. Ongoing development oradapting an intervention to newconditions2. Adapting effective generalprinciples to a new context3. Developing a rapid response toa major change4. Pre-formative development of apotentially scalable innovation orgetting an intervention ready fora summative evaluation5. Major systems change andcross-scale evaluation toprovide feedback about how theintervention is unfolding andhow it may need to be adaptedfor broader application.(p. 21-22)
PlanningPlanningWhy Developmental Approach?Why Developmental Approach?Grapes• Collaborative approach ofcommunity of practice• Exploration of what is beingdeveloped and what isemerging• Sustainability or what’s nextNetwork Literacy• CoP leaders looking for amore “flexible” evaluationprocess that allowed foroutcomes to emerge andchange• Ultimately looking for systemchange• Social media significant partof programming requiringrapid response
Finding an EvaluatorFinding an EvaluatorWhy Us?Why Us?Grapes Network Literacy• Staff evaluator for largerMFLN• Willingness to try differentapproach
Working with the evaluatorsWorking with the evaluators
ReportingReportingGrapes• Informal• Continuous• Included formal annualreports for grantNetwork Literacy• Combination of informalreporting throughconversations withleadership team• Formal reporting to MFLNand Department of Defenseon monthly basis
Cycling Lessons LearnedCycling Lessons Learned• Critical friend roleNetwork Literacy• Incorporates the informaland formal reporting intoleadership team work• Challenge to getmembership and leadershipof CoP to take action• Heavy investment of timeand effort
Where we are now and what weWhere we are now and what wemight changemight change• Year 3 for Grape CoP, year 2 forNetwork Literacy• NetLit still emerging based partly onchallenges reaching target DoDaudience• Grapes ready to begin formativeevaluation