Toward impact: the co-produced pathway to impact evaluation framework and indicators
•Download as PPTX, PDF•
1 like•296 views
Presented at the May 13-15 Canadian Knowledge Mobilization Forum conference, "sustainability" was the theme. This presentation describes the Co-Produced Pathway to Impact evaluation framework, the database designed for KBHN's KT Core to track services for management decisions and progress reporting, and factors for sustainability with reference to database design.
Recommended
More Related Content
What's hot
What's hot (20)
Similar to Toward impact: the co-produced pathway to impact evaluation framework and indicators
Similar to Toward impact: the co-produced pathway to impact evaluation framework and indicators (20)
More from KBHN KT
More from KBHN KT (20)
Recently uploaded
Recently uploaded (20)
Toward impact: the co-produced pathway to impact evaluation framework and indicators
- 1. www.kidsbrainhealth.ca Indicators for Measurement at each Stage of Knowledge Translation: from Research to Impact Anneliese Poetz, PhD – Manager, KT Core David Phipps, PhD, MBA – Lead, KT Core Canadian Knowledge Mobilization Conference Monday June 9, 3:30 – 4:00p (Room Cedar) Saskatoon, Saskatchewan
- 2. What is KBHN? • Federally funded • NCE • FASD, CP, ASD • NI, NE, KT • KTEE • HQP
- 3. KT Core “KT maximizes the impact of research and training in neurodevelopmental disorders” Services for researchers and trainees in KBHN 1. Knowledge Brokering 2. KT Events 3. KT Products 4. Evaluation 5. KT Planning
- 4. Role of Evaluation How are the KT Core services helping KBHN researchers & trainees? How have we made a difference for children with NDDs and their families, in Canada?
- 5. Evaluation Framework Phipps, D. (2013)
- 7. What’s out there already?
- 8. Indicator Definitions Research Dissemination Uptake Implementation Impact
- 9. Anatomy of indicator Definition components 1) Name/title of indicator 2) Definition (including N/D if applicable) 3) Type 4) Rationale 5) Strengths/Limitations 6) Data Source(s) 7) Stage in CPPI 8) Date 9) Responsibility for collecting data Example 1) [CB] % increase in knowledge 2) Sum of scores/#attendees x 100 3) Impact 4) Measure knowledge gained 5) S: measures change/impact, L: bias & missing data 6) Pre-Post questionnaires 7) All 8) May 28, 2014 9) KT Core
- 10. So what’s it used for? Reporting (value/impact) of: • services, events, products, etc. Comparison – Across services, events, products, programs, etc. – Across NCEs (KTEE standards?) Inform decision-making Succession planning Future (finish defn’s, Access)
- 11. www.kidsbrainhealth.ca Questions? Anneliese Poetz, PhD Manager, Knowledge Translation Core KBHN apoetz@yorku.ca
- 14. Data Collection
Editor's Notes
- What is NeuroDevNet? What we do, why we’re here (high level) Federally funded NCE ~80 researchers/trainees (FASD, ASD, CP) Knowledge and Technology Exchange and Exploitation “…seek to shed light on the causes of neurological disorders, and to share this knowledge to health care professionals, policy makers, and communities of interest… NeuroDevNet works across traditional disciplinary boundaries and sectors to ensure our findings are translated into tangible diagnostic, preventative, therapeutic, social, economic, and health benefits for all.”
- What is the role of the KT Core within NeuroDevNet? We (KT Core) support NeuroDevNet’s HQP mandate through workshops like the KT/Social Media workshop.
- What is the role of evaluation? To measure the impact the network has had on children with NDDs (and their families) in Canada. Explain why do we do evaluation, why is it important.
- Explain CPPI, what it is and why we are using it (important planning tool, also for monitoring progress toward impact – CPPI is best suited for a system of research or ‘set of projects’ rather than a single project that may never reach ‘impact’).
- Re: the indicator that is circled – just drawing attention to how this indicator does not really measure impact of capacity building efforts. Still good to measure how many (for resource allocation decisions, or just to know how many events have been done) but doesn’t let you know what the impact of these events are.
- Indicator Type (input, process, output, outcome, impact, performance, behaviour change, etc.) Talk about an interesting thing about the indicator (value of a good indicator) such as the story about % increase in knowledge at the KT/Social Media workshop (e.g. 28% increase in knowledge – infographic on handout) For Rationale – I actually got rid of some indicators because I couldn’t think of a good reason why we should collect data/measure it.
- Why is thinking about indicators in this way, important? So you can report measures of impact or value-for-the-money to your funder (NCE, CIHR, etc.) Allow comparison across a) indicators for each service within NDN and b) across NCEs (a number of NCEs have self-identified as having KTEE within their NCEs, would be good to have a conversation about establishing standards for KTEE in NCEs. Having good clear definitions of indicators, this allows us to start to create some standards - does this form the basis for a standard for other NCEs, so comparisons can be made across Networks?) **tell story about comparison between needle exchange programs (numerator/denominator) inform decision-making (for KT Core services) – in other words, we are creating our own evidence-base for making evidence-informed decisions (**that the right indicators are informing the right decisions – we apply them in the right way to the right decision – we can be confident we are applying evidence in the right way because the definition tells us what it is defined as and used for). As a management tool (e.g. for succession planning purposes) to make the transition easier for the next person (established data collection processes, data to learn from with regard to service provision) Future: Finish the indicator definitions (~130) and continue to collect data on our services. These are ‘grouped’ according to the way they are mapped onto the CPPI. Some are other people’s responsibility to collect (through Valorization Committee, through established routes such as progress reports for each project/core etc.), evaluation data collected by projects/cores that do events and collect this data. Migrate the system from Excel to Access, to make input of data more efficient.
- Have this evaluation framework, and this set of services, needed to map the services onto the evaluation framework.
- Have this evaluation framework, and this set of services, needed to map the services onto the evaluation framework.