'Demystifying Knowledge Transfer- an introduction to Implementation Science Master Class (Newcastle 28 May 2014)
Upcoming SlideShare
Loading in...5
×
 

Like this? Share it with your network

Share

'Demystifying Knowledge Transfer- an introduction to Implementation Science Master Class (Newcastle 28 May 2014)

on

  • 261 views

Powerpoint presentation from 'Demystifying Knowledge Transfer: an introduction to Implementation Science' - 28th May 2014.

Powerpoint presentation from 'Demystifying Knowledge Transfer: an introduction to Implementation Science' - 28th May 2014.
Facilitated by Professor Jeremy Grimshaw and Dr Justin Presseau

Statistics

Views

Total Views
261
Views on SlideShare
261
Embed Views
0

Actions

Likes
0
Downloads
4
Comments
0

0 Embeds 0

No embeds

Accessibility

Upload Details

Uploaded via as Adobe PDF

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

'Demystifying Knowledge Transfer- an introduction to Implementation Science Master Class (Newcastle 28 May 2014) Presentation Transcript

  • 1. Collaborating for Better Care Partnership Master Class: ‘Demystifying Knowledge Transfer’: Implementing Evidence Based Guidance An introduction to Implementation Science 28th May 2014 International Centre for Life @AHSN_NENC @JPresseau
  • 2. Welcome and Introduction Professor Paula Whitty Director of NEQOS & Acting NENC AHSN Knowledge & Information Programme lead
  • 3. Programme 10.10 Session 1: Implementing evidence based guidance 11.00 Session 2: Case studies (working in pairs-followed by group feedback) 11.30 Coffee & biscuits 11.45 Session 3a: Behavioural approaches to implementing evidence based guidance Identifying barriers and modifiable determinants 12.15 Sessions 3b: Identifying barriers and modifiable determinants of implementation Neighbour discussion (15 mins) plus some feedback time (15 mins). Jeremy/Justin barrier assessment in case studies 13.15 Lunch 14.00 Session 4: Behavioural approaches to implementing evidence based guidance Designing implementation programmes (Justin and Jeremy) Case studies Neighbour discussion (15 mins) plus some feedback time (15 mins) 15.15 Coffee 15.30 Session 5: Implementation design in case studies (Justin and Jeremy) 16.00 Summary, conclusions and group discussion - Jeremy 16.20 Concluding remarks - Paula/Jackie 16.30 Close
  • 4. Greetings from Ottawa
  • 5. Greetings from Newcastle
  • 6. Session 1: Implementing Evidence Based Guidance Prof Jeremy Grimshaw Dr Justin Presseau
  • 7. Session 1 Core concepts Knowledge creation funnel
  • 8. Background ‘All breakthrough, no follow through’ Woolf (2006) Washington Post op ed Much of the US $100 billion/year worldwide investment in biomedical and health research is wasted because of dissemination and implementation failures
  • 9. Background Institute of Medicine; Clinical Research Roundtable, Sung et al. JAMA 289:1278,2003
  • 10. Background Why do we need to think about implementation? • Consistent evidence of failure to translate research findings into clinical practice – 30-40% patients do not get treatments of proven effectiveness – 20–25% patients get care that is not needed or potentially harmful Schuster, McGlynn, Brook (1998). Milbank Memorial Quarterly Grol R (2001). Med Care • Suggests that implementation of evidence based care is fundamental challenge for healthcare systems to optimise care, outcomes and costs
  • 11. How do healthcare organisations currently address quality challenges?
  • 12. Issue guidance
  • 13. Internal solutions ISLAGIATT principle Martin P Eccles ‘It Seemed Like A Good Idea At The Time’
  • 14. Designing interventions If you have a hammer, everything looks like nail
  • 15. External solutions
  • 16. External solutions
  • 17. External solutions
  • 18. Throw everything at the problem! 1628466356N = Absolute effect size Number of interventions in treatment group >44321 80% 60% 40% 20% 0% -20% -40% -60% -80% Grimshaw et al (2004) Health Technology Assessment
  • 19. To date, many organisational responses to poor implementation have failed to achieve optimal care despite considerable investments Most approaches to changing clinical practice are more often based on beliefs than on scientific evidence ‘Evidence based medicine should be complemented by evidence based implementation’ Grol (1997). British Medical Journal
  • 20. Could we do better? Undoubtedly
  • 21. Implementing evidence based practices • Implementation is about ensuring that stakeholders are aware of and use research evidence to inform their decision making and actions to improve processes and outcomes of care
  • 22. Implementing evidence based practices • Successful implementation depends upon: – Internal knowledge (eg performance data, tacit knowledge of how organisation (and individuals) work) – External knowledge (eg clinical and implementation science) – Behaviour and organisational change expertise
  • 23. Implementing evidence based practices Quality by any means necessary suggests need to use all tools and levers at your disposal
  • 24. Implementation Science • Implementation is a human enterprise that can be studied to understand and improve implementation approaches • Implementation science is the scientific study of the determinants, processes and outcomes of implementation. • Goal is to develop a generalisable empirical and theoretical basis to optimise implementation activities
  • 25. Implementation Science applied health research capacity building co-optation - cooperation - competing diffusion* dissemination* getting knowledge into practice impact Implementation* knowledge communication knowledge cycle knowledge exchange knowledge management knowledge translation knowledge mobilization knowledge transfer linkage and exchange popularization of research, research into practice research mediation research transfer research translation science communication teaching “third mission” translational research transmission utilization *cited most frequently
  • 26. Implementation Science
  • 27. Implementation Science
  • 28. Implementation Science
  • 29. Implementation science • Implementation science is a research relatively new field in health research • Inherently interdisciplinary • Wide range of disciplines need to be engaged – Clinical – Health services research – Social sciences – Design and engineering – Informatics – Methodologists • Broad range of forms of enquiry needed
  • 30. Implementation science • Knowledge synthesis (what care should we be providing, what do we know about the effectiveness of different implementation approaches); • Research into the evolution of and critical discourse around research evidence; • Research into knowledge retrieval, evaluation and knowledge management infrastructure • Identification of implementation failures; • Development of methods to assess barriers and facilitators to implementation; • Development of the methods for optimising implementation programs; • Evaluations of the effectiveness and efficiency of implementation programs; • Sustainability and scalability of implementation programs; • Development of implementation science theory; and • Development of implementation science research methods.
  • 31. Knowledge to action cycle Knowledge to action cycle Graham et al (2006). Lost in Knowledge Translation. Time for a Map? Journal of Continuing Education for Health Professionals
  • 32. Knowledge creation funnel Potential barriers to evidence based practice – knowledge management • Over 20,000 health journals published per year – Average time professionals have available to read = <1 hour/week • Published research of variable quality and relevance – Research users (consumers, health care professionals, policy makers, researchers) often poorly trained in critical appraisal skills • Individual studies rarely by themselves provide sufficient evidence for policy or practice changes – Individual studies are often misleading
  • 33. Don’t believe the hype: early highly positive results often contradicted
  • 34. • Analyzed 115 articles published in 1990-2003 in the 3 major general medical journals (NEJM, JAMA, Lancet) and specialty journals that had received over 1000 citations each by August 2004 • 49 reported evaluations of health care interventions; 45 claimed that the interventions were effective. • By 2004 5/6 non randomised studies and 9/39 randomised trials were already contradicted or found to be exaggerated Don’t believe the hype: early highly positive results often contradicted Ioannidis JP. JAMA 2005
  • 35. Don’t generate the hype
  • 36. • AHSC release average of 49 press releases annually • 44% promoted animal or laboratory research – 74% of these explicitly claimed relevance to human health. • 47.5% were about primary human research – 23% omitted study size – 34% failed to quantify results – 17% promoted studies with the strongest designs (randomized trials or meta-analyses) – 40% reported results of weak designs (uncontrolled studies, small samples (30 participants), surrogate primary outcomes, or unpublished data) but 58% lacked relevant caution Don’t generate the hype
  • 37. Knowledge creation funnel Systematic reviews are a generic methodology used to synthesise evidence from a broad range of research methods addressing different questions
  • 38. Knowledge creation funnel The steps involved in undertaking a systematic review include – stating the objectives of the research – defining eligibility criteria for studies to be included – identifying (all) potentially eligible studies – applying eligibility criteria – assembling the most complete dataset feasible – analysing this dataset, using statistical synthesis and sensitivity analyses, if appropriate and possible – preparing a structured report of the research.
  • 39. Systematic reviews are a generic methodology used to synthesise evidence from a broad range of research methods addressing different questions – Effectiveness of health care interventions – Diagnostic and screening tests – Determinants of health – Aetiological epidemiological studies – Genetic epidemiological studies – Health system issues (eg quality of discharge coding) – Qualitative methods – consumers’ experiences of health care Knowledge creation funnel
  • 40. Knowledge creation funnel Clarke MJ, Hopewell S, Juszczak E, Eisinga A, Kjeldstrøm M. Compression stockings for preventing deep vein thrombosis in airline passengers. Cochrane Database of Systematic Reviews 2006, Issue 2. Art. No.: CD004002.
  • 41. Knowledge creation funnel
  • 42. Knowledge creation funnel
  • 43. Cochrane Effective Practice and Organisation of Care (EPOC) Group • Cochrane Effective Practice and Organisation of Care (EPOC) group undertakes systematic reviews of interventions to improve health care systems and health care delivery including: – Professional interventions (e.g. continuing medical education, audit and feedback) – Financial interventions (e.g. professional incentives) – Organisational interventions (e.g. the expanded role of pharmacists) – Regulatory interventions
  • 44. Cochrane Effective Practice and Organisation of Care (EPOC) Group EPOC Progress to date • 96 reviews, 1 overview, 36 protocols • Professional interventions – Audit and feedback: effects on professional practice and health care outcomes – The effects of on-screen, point of care computer reminders on processes and outcomes of care • Organisational interventions – The effectiveness of strategies to change organisational culture to improve healthcare performance – Lay health workers in primary and community health care for maternal and child health and the management of infectious diseases
  • 45. Cochrane Effective Practice and Organisation of Care (EPOC) Group EPOC Progress to date • Financial interventions – The impact of user fees on access to health services in low- and middle-income countries – Capitation, salary, fee-for-service and mixed systems of payment: effects on the behaviour of primary care physicians • Regulatory interventions – Effects of changes in the pre-licensure education of health workers on health-worker supply – Pharmaceutical policies: effects of cap and co-payment on rational drug use
  • 46. Cochrane Effective Practice and Organisation of Care (EPOC) Group Intervention # of trials Median absolute effect Interquartile range Audit and feedback (Ivers 2011) 140 +4.3% +0.5% - +16% Educational meetings (Forsetlund 2009) 81 +6% +3 – +15% Financial incentives (Scott 2011) 3 NA NA Hand hygiene (Gould 2010) 1 NA NA
  • 47. Factors influencing effectiveness of audit and feedback Ivers N et al. Audit and feedback: effects on professional practice and health care outcomes. Cochrane Library 2012 – Larger effects were seen if: • baseline compliance was low. • the source was a supervisor or colleague • it was provided more than once • it was delivered in both verbal and written formats • it included both explicit targets and an action plan
  • 48. Summary Knowledge creation funnel • The results of individual studies need to be interpreted alongside the totality of evidence (ie systematic reviews) • Emphasis on KT of individual studies may distract the stakeholder group (increasing the noise to signal) – ‘Don’t believe the hype’ – ‘Don’t generate the hype’ • Substantial evidence of effectiveness of implementation interventions • Average effects modest but considerable variation of observed effects suggesting that intervention design features and contextual factors likely effect modifiers • Key (research and service) challenge is how to optimise interventions and tailor intervention to context
  • 49. Knowledge infrastructure • Knowledge management is fundamental challenge for health care organisations wishing to use evidence • There is a need to develop knowledge infrastructure (services and processes) – Knowledge intelligence services – Rapid synthesis services – Requirements for statement about evidence considered in high level policy documents (eg senior management team submissions) – ….
  • 50. Is research working for you? http://www.cfhi- fcass.ca/PublicationsAnd Resources/ResourcesAnd Tools/SelfAssessmentTool .aspx
  • 51. Is research working for you? 1. Acquire 1.1 Are we able to acquire research? 1.2 Are we looking for research in the right places? 2. Assess 2.1 Can we tell if the research is valid and of high quality? 2.2 Can we tell if the research is relevant and applicable? 3. Adapt 3.1 Can we summarize results in a user-friendly way 4. Apply 4.1 Do we lead by example and show that we value research use? 4.2 Do our decision making processes have a place for research?
  • 52. Is research working for you?
  • 53. Knowledge to action cycle Knowledge to Action loop From: Graham ID et al. Lost in Knowledge Translation: Time for a Map? Journal of Continuing Education in the Health Professions, 2006
  • 54. Adapting knowledge to local context • May require additional data collection to assess applicability of knowledge to local context • May require modification of recommended actions based upon applicability, resources and contextual issues
  • 55. Summary • Implementation is about ensuring that stakeholders are aware of and use research evidence to inform their decision making and actions to improve processes and outcomes of care • Implementation science is the scientific study of the determinants, processes and outcomes of implementation.
  • 56. Summary • Successful implementation depends upon: – Internal knowledge (eg performance data, tacit knowledge of how organisation (and individuals) work) – External knowledge (eg clinical and implementation science) – Behaviour and organisational change expertise • Adopting a systematic (theoretically informed) approach will likely enhance likelihood of successful implementation • The knowledge to action cycle is a useful planning framework.
  • 57. Session 2: Case Studies
  • 58. How would you tackle this? • Two scenarios: choose one scenario, then work in pairs at your tables – Hand hygiene – Diabetes care • Spend 15 minutes in pairs • Feedback to your table for 5 minutes • General thoughts from tables 10 minutes
  • 59. Scenario 1 Hand hygiene in hospital staff • Healthcare-associated infections are among top 10 causes of hospital deaths worldwide • Hand hygiene (washing or disinfecting hands) is most effective and cost-effective prevention method • Adherence to hand hygiene recommendations consistently below 50% Scenario 2 Diabetes care in primary care 2011-2012 National diabetes audit showed: - 66% of patients meet guideline-recommended treatment targets HbA1c (<=58mmol/mol) - 47% had blood pressure < 140/80mmHg - 41% reaching cholesterol target of <4mmol/L - 22% meeting all three targets - Care process completion has plateaued 2011 National study of 99 practices showed: • 73% of patients received general education • 51% with BMI>30 received weight advice • 68% received self-management advice • 59% prescribed for HbA1c when above target • 40% prescribed when BP above target How would you improve the implementation of hand hygiene practices in hospital? How would you improve the quality of diabetes care in primary care? Spend 15 minutes in pairs Feedback to your table for 5 minutes General thoughts from tables 10 minutes
  • 60. Feedback How would you improve the implementation of hand hygiene practices in hospital? How would you improve the quality of diabetes care in primary care?
  • 61. Session 3a: Behavioural approaches to implementing evidence based guidance Prof Jeremy Grimshaw Dr Justin Presseau
  • 62. Knowledge to action cycle Knowledge to Action loop From: Graham ID et al. Lost in Knowledge Translation: Time for a Map? Journal of Continuing Education in the Health Professions, 2006
  • 63. Barriers to implementation • Structural (e.g. financial disincentives) • Organisational (e.g. inappropriate skill mix, lack of facilities or equipment) • Peer group (e.g. local standards of care not in line with desired practice) • Individual (e.g. knowledge, attitudes, skills) • Professional - patient interaction (e.g. problems with information processing)
  • 64. A behaviour change approach to implementation science • Behaviour change approaches apply to any level: from individuals to groups to organisations – Diagnosis: • Who needs to do what, differently? • What is preventing them from doing so – Intervention: • Help them change what they do to promote implementation
  • 65. Identifying behaviours of interest • What is the behavior (or series of linked behaviors) that you are trying to change? • Who performs the behavior(s)? (potential adopter) • When and where does the potential adopter perform the behavior? • Are there obvious practical barriers to performing the behavior? • Is the behavior usually performed in stressful circumstances? (potential for acts of omission)
  • 66. Identifying whose behaviour(s) need to change • Often useful to specify target behaviours in terms of: – Actor performing the behaviour – Action being performed – Target at which the action is directed – Context in which action is performed – Time during which the action is performed. • Provides clarity regarding what to change
  • 67. Why use theory? • An organized, heuristic, coherent, and systematic articulation of a set of statements related to significant questions that are communicated in a meaningful whole for the purpose of providing a generalisable form of understanding. Meleis AI: Theoretical nursing. Development and progress • It describes observations, summarizes current evidence, proposes explanations, and yields testable hypotheses. • It represents aspects of reality that are discovered or invented for describing, explaining, predicting and controlling a phenomenon The Improved Clinical Effectiveness through Behavioural Research Group(ICEBeRG). (2006) Imp Sci Assessing barriers to implementation
  • 68. Why use theory? • Interventions are likely to be more effective if they target determinants of behaviour • Theoretical frameworks facilitate accumulation and integration of evidence – across context, population and behaviour – of effects and of causal mechanisms • Allows refinement and development of theory and, hence, more effective interventions Assessing barriers to implementation
  • 69. What levels of theory? • Ferlie and Shortell suggest four levels of interventions to improve the quality of health care: – the individual health professional; – health care groups or teams; – organisations providing health care (e.g., NHS trusts); – the larger health care system or environment in which individual organisations are embedded. • Different types of theory will be relevant to interventions at different levels Ferlie, Shortell (2001). Milbank Quarterly Assessing barriers to implementation
  • 70. Making sense of theory • Multiple theories and frameworks of individual and organizational behaviour change, often with conceptually overlapping constructs • Two recent attempts to make theory more accessible – Theoretical domains framework – Behaviour change wheel
  • 71. Theoretical domains framework
  • 72. Theoretical domains framework • Conceptual mapping of 128 explanatory constructs drawing on 33 psychological theories • Identified 14 domains covering main known factors influencing behaviour and behaviour change
  • 73. Theoretical domains framework • Knowledge – Aware of guidelines and evidence? • Skills – Sufficient training in techniques required? • Social/professional role and identity – Is the action part of what the actor sees as ‘typical’ of their profession? • Beliefs about capabilities – Confident in capacity to do the behaviour? What makes it easier or difficult? • Optimism – Is the actor generally optimistic that doing the behaviour will make a difference in the grand scheme of things? • Beliefs about consequences – What the the benefits and negative aspects of doing the behaviour? • Reinforcement – Does the behaviour lead to any personal or external reward when it is performed? • Intentions – How motivated is the actor to do this? • Goals – How much of a priority is this action compared to other competing demands? • Memory, attention and decision processes – Does the actor ever forget? Are there reminders in place? • Environmental context and resources – Are there sufficient resources to do the behaviour? If not, what is missing? • Social influences – Who influences the decision to perform the behaviour? • Emotion – Is performing the behaviour stressful? • Behavioural regulation – What does the actor personally do to ensure that they perform the behaviour? Cane et al 2012 (Impl.Sci.)
  • 74. Behaviour Change Wheel
  • 75. From the TDF to COM-B Michie, van Stralen, West (2011) Impl.Sci. Ability • Physical • Psychological Environmental factors • Physical • Social Conscious and automatic decision processes
  • 76. Physical: physical skills Psychological: Knowledge, cognitive and interpersonal skills, memory/ attention/ decisions processes, behavioural regulation Reflective: intention, goals, social/professional role and identity, beliefs about capabilities, beliefs about consequences, optimism Automatic: reinforcement, emotions Physical: Environmental context and resources Social: Social influences Linking the TDF to the COM-B Model Michie, Atkins, West (2014)
  • 77. Should we use the TDF or COM-B? • COM-B highlights higher-level factors • TDF provides a fine-grained analysis that can be aggregated to the COM-B level
  • 78. Summary so far Whatever the level of granularity of the assessment, theory provides a way to assess barriers to implementation that provides… – Common language for building cumulative knowledge- base to learn from past successes (and failures) – Move beyond trial and error and ISLAGIATT – Provides a basis for designing targeted interventions optimised to address identified barriers to improve care
  • 79. Sessions 3b: Identifying barriers and modifiable determinants of implementation Professor Jeremy Grimshaw Dr Justin Presseau • Neighbour discussion (15 min) Feedback time (15 min) • Barrier assessment in case studies
  • 80. Small group exercise • Diagnosing the implementation problem 1. Whose behaviour needs to change? 2. Which behaviour(s)/actions do they need to change? 3. What are the barriers stopping them? • Using COM-B or TDF as your framework for assessing barriers
  • 81. Scenario 1 Hand hygiene in hospital staff Scenario 2 Diabetes care in primary care In pairs, discuss the following 1. What is the specific behaviour in terms of: ACTION: the specific behaviour(s) ACTOR(s): the person(s) whose behaviour needs to change TARGET: details of the recipient of the action CONTEXT: where is the action performed? TIME: When is the action performed 2. Using the TDF or COM-B, identify which barriers may stopping them Spend 15 minutes in pairs Feedback to your table for 5 minutes General thoughts from tables 10 minutes With your neighbour: Choose a scenario: OR
  • 82. Small group exercise • Feedback • What are advantages and disadvantages of using the theoretical domains framework
  • 83. Case study – physician hand hygiene
  • 84. Theoretical domains framework – physician hand hygiene example Determinants of behaviour • Knowledge • Skills • Social/professional role and identity • Beliefs about capabilities • Optimism • Beliefs about consequences • Reinforcement • Intentions • Goals • Memory, attention and decision processes • Environmental context and resources • Social influences • Emotion • Behavioural regulation Cane et al (2012) Implementation Science
  • 85. Knowledge • I am (not) aware of hand hygiene guidelines and have (not) heard of the 4 moments of hand hygiene • I am (not) aware of evidence linking hand hygiene to health care associated infections • Education about hand hygiene ensures that I practice it consistently Theoretical domains framework – physician hand hygiene example
  • 86. Beliefs about consequences • Practicing hand hygiene reduces the transmission of infection • While improper hand hygiene can contribute to infection, it is not the only factor that can do so • Practicing hand hygiene gives patients confidence in their physician Theoretical domains framework – physician hand hygiene example
  • 87. Beliefs about Capabilities • Hand hygiene is easy to practice • I am not confident that I am following hand hygiene guidelines when practicing hand hygiene Theoretical domains framework – physician hand hygiene example
  • 88. Social influence • Patients expectations do (not) influence me to perform hand hygiene • If I see someone practicing hand hygiene, it influences me to do the same • Team culture influences others hand hygiene practice Theoretical domains framework – physician hand hygiene example
  • 89. Goals • Hand hygiene is always a necessity • Hand hygiene is not my highest priority in patient emergency situations • Hand hygiene is one of many priorities that I have to balance every day Theoretical domains framework – physician hand hygiene example
  • 90. Skills • I do (not) consider hand hygiene a skill • I have (not) had training in hand hygiene practice • With repetition, hand hygiene practice becomes automatic Theoretical domains framework – physician hand hygiene example
  • 91. Memory, attention, decision processes • Hand hygiene is (not) an automatic process for me • When not touching the patient or patient environment, hand hygiene is unnecessary • Reminders are useful for my hand hygiene practice • Easily visible hand hygiene stations make it easier to remember hand hygiene Theoretical domains framework – physician hand hygiene example
  • 92. Social professional role and identity • Hand hygiene is a standard part of my patient consultations • My hand hygiene is in line with my peers • Physician hand hygiene compliance is suboptimal • It is my job to be a hand hygiene role model to the members of my team Theoretical domains framework – physician hand hygiene example
  • 93. Environment • Easy access to hand hygiene stations makes it easier to practice hand hygiene • The location of hand hygiene stations is important in facilitating hand hygiene practice • Practicing hand hygiene takes time • When I am busy, I am less likely to comply with hand hygiene guidelines Theoretical domains framework – physician hand hygiene example
  • 94. Environment - Nonparticipant Observation • Observations made while on a Surgery and Medicine Unit confirmed what was said in the physician interviews: – Alcohol dispensers are sometimes empty – Alcohol dispensers blend in with the wall – Beside alcohol bottle baskets are empty Theoretical domains framework – physician hand hygiene example
  • 95. Case study: the iQuaD example • Three dominant theories and approaches in implementation science: – “If you build it they will come”: the structural approach to behaviour change – “There is no ‘I’ in team”: change involves exchanges and shared processes between individuals working in teams within organisations – “Between the ears” : individuals’ perceptions, cognitions beliefs, schemas, cognitive associations about their behaviour • Rarely ever considered alongside each other. Need for empirical comparison of theory for utility in implementation science
  • 96. The improving Quality in Diabetes care (iQuaD) study1,2 Aim: investigate how effectively and consistently factors from predominant organisational and behaviour theories predict - multiple evidence-based clinical behaviors promoted in guidelines - same sample of clinicians, primary care diabetes management in the UK Design: Predictive. Questionnaires sent at baseline and 12 months later to GPs and nurses in 99 practices across the UK National study of primary care in the UK 1 Eccles et al (2011, Impl Sci); 2 Presseau et al (in press, Journal of Behavioral Medicine)
  • 97. National study • Outcomes: – Clinician-level: clinicians’ self-reported behaviour at 12 months follow-up – Practice level: patient report of care received and patient medical records • Recruitment and response rates1 – 12 months follow-up • 427 (289 GPs, 138 nurses) returned questionnaire (51% response rate). • Mean of 41 patients/practice responded to questionnaire • Main Findings • Gaps in quality of care across the behaviours1 • Theory-based factors that predicted high quality care2 1 Eccles et al (2011, Impl Sci); 2 Presseau et al (in press, Journal of Behavioral Medicine)
  • 98. Prescribing ... 1. ...additional antihypertensive drugs for people with type 2 diabetes whose blood pressure (BP) is above a target of 140 mm Hg for Systolic BP or 80 mm Hg for Diastolic 2. ...additional therapy for the management of glycaemic control (HbA1c) for the management of HbA1c in people whose HbA1c is higher than 8.0%, despite maximum dosage of 2 oral hypoglycaemic drugs. Providing advice about... 1. ... weight management to people with type 2 diabetes whose BMI is above a target of 30kg/m2, even following previous management. 2. ... self-management to people with type 2 diabetes. 3. ...general education about diabetes for people with type 2 diabetes. Examining... 1. ...foot circulation & sensation in the feet of people with type 2 diabetes. Health professionals: 63% Patient Records: 40% Health professionals: 69% Patient Records: 59% Health professionals: 78% People with diabetes: 51% Health professionals: 77% People with diabetes: 68% Health professionals: 78% People with diabetes: 73% Health professionals: 70% People with diabetes: 91% Eccles et al (2011, Impl Sci) National study: gaps in quality of care
  • 99. Staffing ratios Meetings Appointment length Admin support Recall system Insulin initiation Dedicated diabetes clinic Structured education Access to specialist care List size IMD National study: testing structural correlates
  • 100. Procedural Justice Relational Justice Implementation Behaviour Participative Safety Support for Innovation Implementation Behaviour Vision Task Orientation Altruism, Courtesy, Sportsmanship, Conscientiousness, Civic Virtue Implementation Behaviour Organizational Citizenship Behaviours (Moorman, 1991) Team Climate (Anderson & West, 1994) Organizational Justice (Greenberg 1990) Elovainio, Steen, Presseau, Francis et al. (2012) Family Practice. R2 adj = 0.01 (0.00, 0.03) R2 adj = 0.00 (0.00, 0.03) Predicting 12m self- report (median, range): R2 adj = 0.00 (0.00, 0.00) National study: testing team theories
  • 101. Presseau, Johnston, Francis, Hrisos, Stamp, Steen, Hawthorne, Grimshaw, Elovainio, Hunter, Eccles (in press) Journal of Behavioral Medicine Outcome Expectations Self-efficacy Proximal Goals Implementation Behaviour Attitude Subjective Norm PBC Intention Implementation Behaviour Anticipated Consequences Evidence of habit Implementation Behaviour Action Planning Coping Planning Implementation Behaviour Social Cognitive Theory TPB Learning Theory Planning R2 adj = 0.15 (0.09, 0.50) R2 adj = 0.14 (0.09, 0.48) Predicting 12m self- report (median, range): R2 adj = 0.15 (0.09, 0.50) R2 adj = 0.15 (0.07, 0.43) National study: testing behaviour theories
  • 102. • Constructs from Organizational Theories did not predict implementation-related behaviours • Constructs from Behaviour Theories consistently predicted multiple behaviours and scores showed room for improvement: – Social cognitive theory in particular, along with habit and post-intentional factors • Testing different theories in the same sample across multiple behaviours provides empirical theory selection through internal replication – Can be used to design intervention to improve care by targeting modifiable factors shown to consistently predict clinicians behaviour National study: testing multiple theories
  • 103.  Analytical  Effortful  Resource intensive  Slow, Low capacity  Conscious, deliberate2  Perceptual and cued  Minimal effort, resources  Fast, High capacity  Unconscious  Automatic  Default process  Operates in parallel2 Clinician Behaviour Reflective process1 Impulsive process1 1Strack & Deutch, 2004; 2Evans 2008 • Dual process approach provides an opportunity to jointly • Skilled decision-making involving behaviours with highly salient consequences (reflective process) • Automatic responses to environmental cues in stable contexts (impulsive process) • Dual process models suggest that behaviour is determined by two interacting process1 Towards a dual process model of clinician behaviour
  • 104. Motivational Phase Volitional Phase Clinician Behaviour Intention Action Planning Coping Planning Towards a dual process model of clinician behaviour Automaticity 1Presseau, Johnston, Heponiemi, Elovainio, Francis, Eccles, et al (in press) Annals of Behavioral Medicine Tested a dual process model predicting • …six clinical behaviours in same sample • Hypothesising differences relative importance of reflective and impulsive system depending on the behaviour
  • 105. • Motivational process remain a key direct and indirect predictor of clinician behaviour • Volitional process help to explain how intentions are translated into behaviour for advising behaviours but not examining behaviours (unclear for prescribing) • Automatic processes are involved in prescribing, examining and advising behaviours, though not without the input of the reflective process  both reflective and automatic processes involved in predicting clinician behaviours  both could be targeted to promote the implementation of healthcare interventions Summary so far
  • 106. Michie, van Stralen, West (2011) Impl.Sci. • Physical • Psychological • Physical • Social • Conscious • Automatic Interpreting iQuaD findings according to COM-B       Predictive
  • 107. Session 4: Behavioural approaches to implementing evidence based guidance Designing implementation programmes Dr Justin Presseau Prof Jeremy Grimshaw
  • 108. Knowledge to action cycle Knowledge to Action loop From: Graham ID et al. Lost in Knowledge Translation: Time for a Map? Journal of Continuing Education in the Health Professions, 2006
  • 109. • Choice of implementation intervention should be based upon: – ‘Diagnostic’ assessment of barriers – Understanding of mechanism of action of interventions – Empirical evidence about effects of interventions – Available resources – Practicalities, logistics etc Designing interventions
  • 110. Designing interventions
  • 111. Who needs to do what differently? Using a theoretical framework, which barriers and enablers need to be addressed? Which intervention components could overcome the modifiable barriers and enhance the enablers? How will we measure behaviour change? Designing interventions
  • 112. Designing interventions Who needs to do what differently? Using a theoretical framework, which barriers and enablers need to be addressed? Which intervention components could overcome the modifiable barriers and enhance the enablers? How will we measure behaviour change?
  • 113. • What is the behavior (or series of linked behaviors) that you are trying to change? • Who performs the behavior(s)? (potential adopter) • When and where does the potential adopter perform the behavior? • Are there obvious practical barriers to performing the behavior? • Is the behavior usually performed in stressful circumstances? (potential for acts of omission) Designing interventions
  • 114. Designing interventions Who needs to do what differently? Using a theoretical framework, which barriers and enablers need to be addressed? Which intervention components could overcome the modifiable barriers and enhance the enablers? How will we measure behaviour change? • COM-B • TDF • Behaviour change theory
  • 115. Designing interventions Who needs to do what differently? Using a theoretical framework, which barriers and enablers need to be addressed? Which intervention components could overcome the modifiable barriers and enhance the enablers? How will we measure behaviour change?
  • 116. Designing interventions We have found it useful to distinguish… – What we are trying to change – Why are we trying to change it? (constructs: barriers and enablers) – How are we going to change it, including • Intervention functions and Behaviour change techniques • Context: the mode of delivery (eg group meeting, DVD) • Content: how the technique will be operationalised
  • 117. Behaviour change wheel Michie, van Stralen, West (2012) Central: COM-B model of behaviour • Intervention functions surround the COM-B • Policy categories to support change
  • 118. Behaviour change wheel: intervention functions Increase knowledge and understanding Use communication tools to provoke positive or negative emotions or behaviour Develop an expectation that a reward will be provided for performance Develop an expectation that performance will result in cost or punishment Developing physical, cognitive or social skills Reduce performance opportunities through rule-setting Make a change to the external social or physical context Exposure to someone that one identifies with to imitate Facilitation beyond education, training and environmental restructuring
  • 119. COM-B TDF Intervention functions Physical capability Physical skills Training Psychological capability Knowledge Education Cognitive and interpersonal skills Training Memory, attention and decision processes Training; Environmental restructuring; Enablement Behavioural regulation Education; Training; Modelling; Enablement Capability Michie, Atkins, West (2014), p113-114 From TDF, to COM-B to Intervention Functions
  • 120. COM-B TDF Intervention functions Physical opportunity Environmental context and resources Training; Restriction; Environmental restructuring; Enablement Social opportunity Social influences Restriction; Environmental restructuring; Modelling; Enablement Michie, Atkins, West (2014), p113-114 Opportunity From TDF, to COM-B to Intervention Functions
  • 121. From TDF, to COM-B to Intervention Functions COM-B TDF Intervention functions Reflective motivation Professional/social role and identity Education; Persuasion; Modelling Beliefs about capabilities Education; Persuasion; Modelling; Enablement Optimism Education; Persuasion; Modelling; Enablement Beliefs about consequences Education; Persuasion; Modelling Intentions Education; Persuasion; Incentivisation; Coercion; Modelling Goals Education; Persuasion; Incentivisation; Coercion; Modelling; Enablement Automatic motivation Reinforcement Training; Incentivisation; Coercion; Environmental restructuring Emotion Persuasion; Incentivisation; Coercion; Modelling; Enablement Michie, Atkins, West (2014), p113-114 Motivation
  • 122. Links between COM-B and intervention functions in the Behaviour Change Wheel COM-B Intervention functions Education Persuasion Incentivisation Coercion Training Restriction Environmentalrestructuring Modelling Enablement Physical capability Psychological capability Physical opportunity Social opportunity Automatic motivation Reflective motivation Michie, Atkins, West (2014, p116)
  • 123. Designing interventions – from functions to behaviour change techniques Need greater clarity re: specific content of interventions to change behaviour - What does an ‘educational session’ involve? What does providing a new piece of guidance involve? What does ‘we sent our GPs on a training day’ actually involve? What are the active ingredients of change? - If we want to replicate and generalise efforts in implementation science, we need a shared understanding of the content of our interventions
  • 124. Goals and Planning Goal setting (behavior) OR Goal setting (outcome) Problem solving Action planning Review behavior goal(s) OR Review outcome goal(s) Discrepancy between current behavior and goal Behavioral contract Commitment Feedback and monitoring Monitoring of behaviour by others without feedback Feedback on behaviour/outcomes of behaviour Feedback on outcomes of behaviour Self-monitoring of behaviour Self-monitoring of outcomes of behaviour Monitoring of outcome(s) of behaviour without feedback Biofeedback Social Support Social support (unspecified) Social support (practical) Social support (emotional) Shaping Knowledge Instruction on how to perform the behaviour Information about Antecedents Re-attribution Behavioural experiments Natural Consequences Info about health consequences Info about emotional consequences Info re social and environment consequences Salience of consequences Monitoring of emotional consequences Anticipated regret Comparison of behaviour Demonstration of the behaviour Social comparison Information about others’ approval Associations Prompts/cues Cue signalling reward Reduce prompts/cues Remove access to the reward Remove aversive stimulus Satiation Exposure Associative learning Repetition and substitution Behavioural practice/rehearsal Behaviour substitution Habit formation Habit reversal Overcorrection Generalisation of target behaviour Graded tasks Comparison of outcomes Credible source Pros and cons Comparative imagining of future outcomes Reward and threat Incentive (outcome Material incentive (behaviour) Social incentive Non-specific incentive Self-incentive Self-reward Reward (outcome) Material reward (behaviour) Non-specific reward Social reward Future punishment Regulation Conserving mental resources Pharmacological support Reduce negative emotions Paradoxical instructions Antecedents Adding objects to the environment Restructuring the physical environment Restructuring the social environment Avoidance/reducing exposure to cues for behaviour Distraction Body changes Identity Identification of self as role model Framing/reframing Incompatible beliefs Valued self-identify Identity associated with changed behaviour Scheduled consequences Behaviour cost Punishment Remove reward Reward approximation Rewarding completion Situation-specific reward Reward incompatible behaviour Reward alternative behaviour Reduce reward frequency Remove punishment Self-belief Verbal persuasion about capability Mental rehearsal of successful perform Focus on past success Self-talk Covert learning Imaginary punishment Imaginary reward Vicarious consequences V1 Behaviour change techniques taxonomy (Michie et al 2013)
  • 125. Examples of techniques w/ definitions • Graded tasks: “Set easy-to-perform tasks, making them increasingly difficult, but achievable, until behaviour is performed” – Capability (Psychological) in COM-B – Beliefs about Capabilities in TDF • Habit formation: “Prompt rehearsal and repetition of the behaviour in the same context repeatedly so that the context elicits the behaviour” – Motivation (automatic) in COM-B – Behavioural Regulation and Reinforcement in TDF • Feedback on behaviour: “Monitor and provide informative or evaluatve feedback on performance of the behaviour (e.g. form, frequency, duration, intensity)” – Motivation (reflective) in COM-B – Behaviour regulation in TDF Not all techniques are useful, and many techniques are designed to address specific types of barriers
  • 126. From behaviour change techniques to theory-informed barriers • Behaviour change techniques can be mapped onto the theory-based barriers and facilitators from the models covered – Behaviour change theories – TDF – COM-B • Behaviour change wheel (intervention functions)
  • 127. Supporting change through policy Michie, van Stralen, West (2012) Policy initiatives can facilitate intervention functions impact on COM-B components
  • 128. Policy categories Intervention functions Education Persuasion Incentivisation Coercion Training Restriction Environmentalrestructuring Modelling Enablement Communication/marketing Guidelines Fiscal measures Regulation Legislation Environmental and social planning Service provision Links between policy categories and intervention functions in the Behaviour Change Wheel Michie, Atkins, West (2014, p138)
  • 129. Optimising interventions Usability studies • Develop prototype intervention • Test prototype in 5 to 8 subjects to review content and format using ‘think aloud’ methodology. These sessions will be audio recorded and the results transcribed and analysed. • In general a modest number of subjects are required for usability testing (e.g. 8-9 subjects), and often 4 to 5 are necessary to identify 80% of the usability problems. • Cycles of design, development and testing will be completed until no further major revisions are needed.
  • 130. Case studies • Neighbour discussion(15 min) Feedback time (10 min) • Implementation design in case studies
  • 131. Scenario 1 Hand hygiene in hospital staff Scenario 2 Diabetes care in primary care In pairs, discuss the following Based on the barriers you identified using the TDF or COM-B, select… 1. Potential intervention functions to target those barriers 2. Potential policy categories that would support that intervention function Spend 15 minutes in pairs Feedback to your table for 5 minutes General thoughts from tables 5 minutes With your neighbour: Choose the same scenario OR
  • 132. Session 5: Behavioural approaches to implementing evidence based guidance Implementation design in case studies Prof Jeremy Grimshaw Dr Justin Presseau
  • 133. Designing interventions Case study of physician hand hygiene
  • 134. Designing interventions physician hand hygiene Physician need to practice hand hygiene routinely Beliefs about consequences – failure to practice hand hygiene not necessarily associated with adverse event Persuasion/social influence – information on hospital associated infections and negative associated consequences, emphasis on hand hygiene as a team level responsibility delivered to team session by social influential Who needs to do what differently? Using a theoretical framework, which barriers and enablers need to be addressed? Which intervention components could overcome the modifiable barriers and enhance the enablers? How will we measure behaviour change?
  • 135. Designing interventions physician hand hygiene Choice of implementation intervention should be based upon: – ‘Diagnostic’ assessment of barriers – Understanding of mechanism of action of interventions – Empirical evidence about effects of interventions – Available resources – Practicalities, logistics etc
  • 136. Designing interventions physician hand hygiene 1. Initial sensitisation (residents) Intervention content: Refresher about: – the 4 moments of hand hygiene (knowledge) – what is the patient environment (knowledge) – TOH hand hygiene compliance and infection rates (beliefs about consequences, social influences (priority for chief resident and hospital)) Proposed delivery for Medicine: – When: During Resident Orientation -1st day of block – What: 1-2 slides on hand hygiene to be developed by team and given to Chief Resident – Who will deliver: Chief Resident at the beginning of the block
  • 137. Designing interventions physician hand hygiene 2. Reinforcement (residents, attending physicians) Intervention Content: Knowledge about: – Infection rates, the 4 moments, the patient environment (exact content to be developed and will be clinically relevant) (knowledge) – Add Glo Germ demonstration in one of these sessions to illustrate technique (booth after session for all to try if interested) (skills) Proposed delivery for Medicine: – When: During Antibiotic Stewardship Rounds – a weekly pause of rounds that lasts a few minutes (already in practice) (social influence) – What: A hand hygiene curriculum delivered weekly (~2min/session) X 4 (for one block) – Who will deliver: Local experts/opinion leaders
  • 138. 3. Address environmental barriers (unit staff) Intervention Content: – Ensure that hand hygiene resources are easily accessible and noticeable (including systems to ensure hand hygiene resources are routinely replaced) Proposed delivery for Medicine: – How: Will walk through the chosen unit(s) – Who will deliver: Members of the study team – Accountability – unit Designing interventions physician hand hygiene
  • 139. -4 6 26 26 3 11 50 44 -10 0 10 20 30 40 50 60 Medicine M1 Medicine M4 Surgery M1 Surgery M4 Resident Audit Scores % Change Between Pre and Post Control Group Intervention Group Designing interventions physician hand hygiene
  • 140. Aim: Conduct a cluster-RCT to evaluate the effectiveness and costs of a theory-based multiple behaviour change intervention targeting general practitioners (GPs) and nurses, to support improvement in the provision of high quality care for people with type 2 diabetes in the North East of England AdvisingPrescribing Examining Local example:
  • 141. Prescribing ... 1. ...additional antihypertensive drugs for people with type 2 diabetes whose blood pressure (BP) is above a target of 140 mm Hg for Systolic BP or 80 mm Hg for Diastolic 2. ...additional therapy for the management of glycaemic control (HbA1c) for the management of HbA1c in people whose HbA1c is higher than 8.0%, despite maximum dosage of 2 oral hypoglycaemic drugs. Providing advice about... 1. ... weight management to people with type 2 diabetes whose BMI is above a target of 30kg/m2, even following previous management. 2. ... self-management to people with type 2 diabetes. 3. ...general education about diabetes for people with type 2 diabetes. Examining... 1. ...foot circulation & sensation in the feet of people with type 2 diabetes. Health professionals: 63% Patient Records: 40% Health professionals: 69% Patient Records: 59% Health professionals: 78% People with diabetes: 51% Health professionals: 77% People with diabetes: 68% Health professionals: 78% People with diabetes: 73% Health professionals: 70% People with diabetes: 91% Eccles et al (2011, Impl Sci) Evidence from our previous national study: gaps in quality of care
  • 142. Design: Cluster randomised controlled trial (stratified by practice size) - Theory-based process evaluation - Interview based process evaluation - Fidelity of delivery - Cost analysis Recipients: GPs, nurses, healthcare assistants delivering care to people with type 2 diabetes Timeline: – Intervention development from Sept 2012 to start of Sept 2013 – Pilot May/June 2013 – Recruitment began in March 2013 – Intervention delivery started mid September 2013 – Follow-up 12 months later The IDEA trial
  • 143. Recruit GPs, nurses, healthcare assistants in 44 Practices Baseline Questionnaire Randomisation (stratified by practice size) Intervention Practices (22) Control Practices (22) Deliver Intervention Interviews (4) Follow-up questionnaire Outcomes (12 months later) - Random 100 patients per practice (anonymous postal questionnaire) - Patient computer records The IDEA trial: flow chart
  • 144. Outcome expectations Self-efficacy Proximal Goals Automaticity Goal conflict Goal Facilitation Goal Priority Action Planning Coping Planning - Based on findings from iQuaD1,2,3 - Social Cognitive Theory4 + volitional constructs5 + dual process model3,6 - Reciprocal determinism1 operationalised to involve environment factors: - Automaticity (automatic response to cues)3 - Competing behaviours (conflict, facilitation and priority)4,5 Behaviour Eccles et al (2011); 2,3 Presseau et al (in pressa; in pressb) 4 Bandura (1998); 5 Sniehotta (2009), 6 Strack & Deutsch (2004); 7,8 Presseau et al (2009, 2011); Logic model
  • 145. Intervention content 1 BCTs from Michie et al (2013). ABM Target Construct Behaviour Change Techniques1 Self-efficacy - Demonstration of the behaviour (beliefs about capabilities) - Social comparison - Verbal persuasion of capability - Behavioural practice/rehearsal - Graded tasks Outcome expectations - Information about health consequences (beliefs about consequences) - Credible source Proximal goals (Reflective motivation) - Goal setting (behaviour) - Discrepancy between current behaviour and goal - Commitment Action planning - Action planning Coping planning - Problem solving - Adding objects to the environment Habit/Automaticity - Behavioural practice/rehearsal (Automatic motivation) - Habit formation - Action planning and problem solving
  • 146. • Self-administered pre- intervention questionnaire • Pre-reading, website and PDF- based • Group-based workshop to each practice – PowerPoint slides – Participant Workbooks – Small group tasks – Video case studies • DVD of materials during evaluation • Self-administered post- intervention questionnaire • DVD of materials after evaluation Intervention Control ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Format
  • 147. clinical expert behaviour change expert Intervention providers
  • 148. 44 GP practices in the North-east of England Setting
  • 149.  Audio recorded sessions - Transcribed/coded for delivery of BCTs, by whom  Facilitator debrief questionnaires - Independently completed; reported delivery of BCTs; coverage across 6 behaviours; intensity  Participant feedback - Write plans on feedback forms Training sessions based on BCTs 1) Facilitator handbook 2) Within facilitator team: observe, coping planning 3) Within research team: practice/rehearsal 4) Feedback on to facilitator team after delivery 1 Bellg et al 2002 Fidelity of delivery
  • 150. – Intervention may or may not be effective – Process evaluation to understand mechanism of change – Theory-based process evaluation1,2: • Pre/post theory-based questionnaires • Test for change in targeted constructs between intervention and control 1,2 Grimshaw et al (2007; submitted) Implementation Science; Outcome expectations Self-efficacy Proximal Goals Automaticity Goal conflict Goal Facilitation Goal Priority Action Planning Coping Planning Behaviour Process evaluation (quantitative)
  • 151. • Four practices randomly selected for follow-up interviews – TDF based barriers and facilitators to engaging in the intervention sessions – Participants: clinicians participating in the intervention, practice manager Process evaluation (qualitative)
  • 152. • Cost of delivering the intervention • Staff training (facilitators) • Primary care costs • Increases in standard materials used (e.g., leaflets) • Time use in consultation • Average cost per patient to the NHS for medication prescribed • Costs of service usage by people with Type 2 diabetes Cost analysis
  • 153. Summary • Designing interventions involves assessing barriers to change and identifying interventions that potentially address these • Behavioural theories may be helpful to inform barrier assessment and intervention choice • Intervention mapping is a technique for systematically considering barriers and potential interventions
  • 154. Developing the field of implementation science • Implementation science is a relatively new field - few health researchers have been engaged in the field for more than 10 years • Substantive level of research activity – Cochrane Effective Practice and Organisation of Care (EPOC) group register includes over 6,000 RCTs and quasi experiments of interventions to improve health care delivery and health care services • Increasing funding and reporting opportunities for knowledge translation research • Move towards research programs and laboratories
  • 155. Implementation Research Laboratories • Research teams integrated into healthcare systems undertaking program(s) of research directly relevant to healthcare systems’ priorities • Reduces problems relating to convening de novo research teams, seeking project by project funding, negotiating access with healthcare systems, conducting study, writing up (usually out of funding period) • Opportunities for formal and informal linkages of mutual advantage to research team and healthcare system • More explicitly recognise relatives roles and responsibilities of research team and healthcare system Developing the field of implementation science
  • 156. Developing the field of implementation science
  • 157. Developing the field of implementation science
  • 158. Developing the field of implementation science www.implementationscience.com
  • 159. Summary • Implementation science is a relatively new field of health services research • Rapid progress has been made but substantial challenges remain • Opportunities to foster linkages between implementation service departments and implementation researchers to form implementation science laboratories and address I2 challenge
  • 160. Discussion • Based on the workshop today, what are your current views on: – Value of behavioural approaches to implementing evidence based guidance? – What would be needed to adopt these approaches in practice? – Are there any additional approaches that might complement behavioural approaches?
  • 161. Closing remarks Professor Paula Whitty Director of NEQOS & Acting NENC AHSN Knowledge & Information Programme lead Dr. Jackie Gray Medical Epidemiologist, NEQOS
  • 162. Get involved in the Work Programme • Sign up at the registration desk (in main foyer) or • Email Dr Jackie Gray jackie.gray5@nhs.net
  • 163. Keep up to date with developments: • Sign up for the e- bulletin at the registration desk (if you haven’t already) Resources will be available on: You Tube - video will be uploaded (link included in next e- bulletin) Slide Share - slide deck will be uploaded (link included in next e-bulletin) AHSN web site www.ahsn-nenc.org.uk NEQOS web site www.neqos.nhs.uk/ Twitter - @AHSN_NENC
  • 164. Thank you