Mythbusters Evaluation

1,186 views

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total views
1,186
On SlideShare
0
From Embeds
0
Number of Embeds
4
Actions
Shares
0
Downloads
5
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Mythbusters Evaluation

  1. 1. May 31, 2012KNOWLEDGE UPTAKE TRENDS:The Case of CHSRF’sMythbustersJonathan SachsSelma DidicStephen PetersenKaye PhillipsCanadian Health Services Research Foundation
  2. 2. Background CHSRF is an independent organization dedicated to accelerating healthcare improvement and transformation for Canadians Mythbusters summarizes the best available evidence to challenge a widely held belief about healthcare in Canada 2
  3. 3. Motivation Mythbusters as a KT Tool CHSRF is interested in the uptake of the its knowledge products  Beyond self-perception Objective  Determine the uptake of the Mythbusters series and its role in contributing to evidence-informed decision- making in Canada  In-depth case study of a research summary 3
  4. 4. Defining and Measuring ‘Uptake’ Uptake Acquisition Application Effects of Application (Policy Change) Indicators• Mythbusters are readily • Mythbusters are • Mythbusters contribute toaccessed by key audiences referenced by key audiences evidence-informed policy •Mythbusters are applied to changes that affect health inform healthcare policy and services decision-making •Mythbusters contribute to • Mythbusters are used in strengthened capacity in an educational context research evidence use MethodsWebsite Analytics Citation Analysis Key Informant InterviewsNewsletter Subscription Key Informant InterviewsAnalysis 4
  5. 5. Defining and Measuring ‘Uptake’ Methods  Google Analytics & e-Newsletter Subscription Analysis  Citation analysis  Peer-reviewed journals, grey literature  11 Key informant interviews  Identified from 3 key audiences of Mythbusters:  Educators  Applied Research & Policy Community  Government Policymakers  Inclusion Criteria: Citation or Self-Identification of Use 5
  6. 6. Findings - Acquisition The Mythbusters series has evolved into one of CHSRF’s flagship products  Web analytics  40-50,000 Unique page views annually  Represents ~10% of all CHSRF web traffic 6
  7. 7. Findings - Acquisition Mythbuster Click-throughs By Broad Institutitional Category 1% 8% Non_Govt 11% Govt 14% Academic_Research 38% Delivery 28% Industry Unknown 7
  8. 8. Findings – Acquisition Mythbuster ‘Click Throughs’ by Province 17% Ontario 11% British Columbia25% Quebec Alberta 5% Nova Scotia Manitoba 4% Saskatchewan 3% New Brunswick 2% Newfoundland 2% PEI 1% 1% 29% Yukon 8
  9. 9. Findings – Application Overall, Mythbusters have been cited in 239 documents since 2000. 80 75 70 70 59 60 46 #of yearly citations 50 44 40 36 30 26 23 20 17 9 7 10 0 1 0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 9
  10. 10. Findings – Application Five Mythbusters made up 36% of all citations.  Myth: A parallel private system would reduce waiting times in the public system (46)  Myth: The aging population is to blame for uncontrollable healthcare costs (38)  Myth: User fees would stop waste and ensure better use of the healthcare system (26)  Myth: Seeing a nurse practitioner instead of a doctor is second- class care (24)  Myth: An ounce of prevention buys a pound of cure (22) 10
  11. 11. Findings – Application Media(Newspapers, Magazi Mythbuster Citations - Source Types nes, Wire Feeds) Journal 12% Meeting paper/publication Resolutions/Court 23% decisions 1% Trade literature Conference (trade papers/announceme journals/online nt or presentations brochures) 2% 14% Online/full paper Course syllabus 4% 10%Thesis/student paper 5% Book/book Reports chapter/textbook 24% 5% 11
  12. 12. Findings – Application Factors that encourage ‘Application’  Summative/Short  Evidence-Based  Conversational/Accessible Language  Framing/Style/Presentation  Issue-Focused Factors that would increase likelihood of ‘Application’  Increase confidence in Mythbusters  Improve Dissemination/Communication  Feedback/Communication Mechanisms for CHSRF 12
  13. 13. Findings – ApplicationEducational Context Course curriculum  Improves understanding of contemporary health issues  Positive student reactions  Capacity building 13
  14. 14. Findings – Effects of ApplicationPolicy Context Mythbusters applied as a decision-support to inform policy direction both directly and indirectly  Health Canada (2010-2012)  User Fees  Doctor Emigration to the USA  Nova Scotia Dept. of Health & Wellness (2010)  ER Care  Winnipeg Regional Health Authority (2008)  Chronic Disease  CIHI – End of Life Report (2007) 14
  15. 15. Findings – Effects of Application Evidence that Mythbusters have played a role in shifting the ‘culture of research use*’ over the last 12 years  EvidenceNetwork.ca  2 Major CIHR grants  Published 170 Op-Eds  Applied Researchers keep them ‘top of mind’  Speaking engagements  ‘Service-oriented’ work*Lomas, 1997 15
  16. 16. Limitations Challenges of measuring policy change  Complexity, Contribution Limited to uptake of a single product Website Statistics & Newsletter  Limited sample from 2008-2012 Citation Analysis  Databases Key Informant Interviews  Purposeful Sample 16
  17. 17. ImplicationsFor CHSRF Reaffirms value of our knowledge products Allows for improvements to target key audiences/effectivenessFor Wider Community Case study provides specific evidence of how research ‘uptake’ can occur Findings can inform future efforts to encourage research utilization 17
  18. 18. Thank You! 18

×