The document discusses a blended learning programme called the Diabetes Academy that was piloted in four countries to improve diabetes care among primary care physicians. It identified four key gaps through needs assessments and used a communities of practice framework. A formative evaluation found communities starting to form and some self-reported performance improvements. The programme demonstrated the complexity of educational interventions for performance improvement and the need for evaluation and participant motivation strategies.

1. Learning and Performance Improvement Dr Dean Jenkins Medical Education Consultant, BMJ Learning

2. Declarations of interest Employed by BMJ learning Does have services and products to sell in relation to this talk

3. An example project Blended learning programme in diabetes called the ‘Diabetes Academy’ Broad needs assessment in four countries Communities of Practice theoretical framework Formative evaluation of the project

4. Diabetes Academy Pilot project in 2010 4 collaborators BMJ Learning (UK), AxDev (Canada), EIMSED (Austria), QUAIME (Germany) Supported by Merck

5. Diabetes Academy - purpose Diabetes in primary care General / family practitioners in the four countries (60 in total) Designed as a pilot project to evaluate the processes and feasibility of this approach

6. Diabetes Academy - gaps Analysis of performance gaps / barriers Review of English and German literature Qualitative and quantitative evaluation data from national data of four collaborators e.g. included outcomes from DNAT Diabetes Needs Assessment Tool. An online knowledge assessment tool in diabetes designed to prioritise reading for primary care physicians. Schroter S, Jenkins D, Playle R, et al. Evaluation of an online Diabetes Needs Assessment Tool (DNAT) for health professionals: a randomised controlled trial. Trials. 2009;10:63.

7. Diabetes Academy – PI Rx Collaborators met and prioritised gaps for intervention. Criteria: common to at least three countries amenable to the intervention proposed 4 priority gaps / barriers identified Assessment of treatment targets Lack of knowledge of diagnostic criteria Drugs’ mode of action Suboptimal team integration

8. Diabetes Academy - CoP “ Communities of Practice” based intervention designed. Communities of Practice are groups of individuals who share and interest, a concern, a problem or a passion. They interact on an ongoing basis. They develop knowledge and expertise in their particular practice. Chosen primarily because of the complexity of the gaps and barriers identified and that it had a robust theoretical basis that could help frame the evaluation.

10. Diabetes Academy - the blend Face to face orientation meeting Case-based discussion on-line Diabetes knowledge testing on-line Reading materials on-line Virtual classrooms on-line Face to face seminars Diabetes experts supporting many of the above interventions

11. Diabetes Academy - outcomes Formative evaluation of the pilot Despite short timescale (3 months) Communities of Practice were starting to form Varied use of interventions – many lessons learnt on ‘pull through’ of participants Some examples of self-reported performance improvement Currently being prepared for submission for publication

12. Diabetes Academy - perspective Performance improvement interventions are complex Cost is high Expertise is required for the design of educational interventions Motivation of participants and ‘pull through’ tactics need to be considered Evaluation of outcomes is challenging Community of Practice is a useful theoretical framework

13. Why learn? How to learn and link to performance? Key factors

14. Educational Interventions – why? Teaching stuff (so they know) < --- > improving (so they do their work better) Teach individuals < --- > empower individuals and teams Train health professionals < --- > improve patient health

15. Measure performance Consider performance improvement as the specific target of educational interventions. PI-CME Education / learning associated with Clinical Audit, Clinical Governance, Staff development Provide learners with: Data, data, data Feedback, feedback, feedback

16. Learning outcomes Moore’s “Expanded Outcomes Framework”. JCEHP 2009;29(1):1-15

17. e.g. Measuring performance (level 5) outcomes Objective: Observation of performance in patient care setting; patient charts; administrative databases. Subjective: self-report of performance

18. Health Professional education

19. PI-CME / clinical governance

20. Summary Performance and Quality Improvement should be the target of educational interventions. Aim high. Numbers through a training programme is not as important as better patient health Education needs formative evaluation (to improve) as well as summative evaluation (to prove it works). Education should be learner-centred, social and work-based.

21. E- learning Dr Helen Morant BMedSci, MBBS, MSc (Science Media Production) Editor, Online Learning, BMJ Learning

22. Outline The book is closed on whether “e-learning” works Definitions restrict We know the principles of education that are effective How technology can serve those?

23. Motivations for learning Getting the job Professional qualifications Keeping the job Certification Appraisal Development Helping people Curiosity

24. The book is closed on whether e-learning works It is “as good as traditional methods” The really important research questions are about Medium Context Objectives Cost effectiveness We lived on farms, then we lived in cities, and now we're going to live on the internet!

25. Define: definition Computer? Social network? Podcast? The Cloud? Think broadly to innovate…

26. We know what principles of education are effective: Deep Mixed Personal Active Repetitive Competitive

27. Why do we use computers? Linking Structuring Storing Delivering Repurposing

28. Applications Blogs Wikis Microblogs Social networks Bookmarking Videos Games 2 nd life

30. Who’s driving this car? “ we don’t even know what this thing is yet” Defining how people should use Twitter is a losing battle - conventions and application (either discrete technology of functionalities) grow organically from the users. This is good. “ he didn’t need to tell us. Our litigators are capable of doing a Google search” In 10 years time, you will not be able to protect intellectual property rights. Discuss. You will neither able to claim credit for nor be liable for what you put the internet. Discuss.

31. Tell me a story…. Case studies Interaction with patients in real life Video and audio - Patient Voices Branched learning - virtual patients

32. Life is messy

33. Time and movement Technology allows things that previously had to be synchronous to be asynchronous and vice versa Synchronous: lectures Asynchronous: online lectures Asynchronous: letters to the editor Synchronous: instant messaging Portability

34. References and reading Virtual Patient design (RVC slides) http://www.slideshare.net/ctrace/virtual-patients-novice-2011-7103730 Instructional design variations in internet-based learning for health professions education: a systematic review and meta-analysis. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. Acad Med. 2010 May;85(5):909-22. Exploring the Value of Social Media and Medical Meetings http://cmeadvocate.com/2011/03/24/social-tuesday-at-7th-meetings-management-forum.aspx Internet-based learning in the health professions: a meta-analysis. Cook DA, Levinson AJ, Garside S, Dupras DM, Erwin PJ, Montori VM. JAMA. 2008 Sep 10;300(10):1181-96 The failure of e-learning research to inform educational practice, and what we can do about it David A. Cook Medical Teacher 2009, Vol. 31, No. 2 : Pages 158-162 Internet-based medical education: a realist review of what works, for whom and in what circumstances Geoff Wong, Trisha Greenhalgh, and Ray Pawson. BMC Med Educ. 2010; 10: 12. The impact of E-learning in medical education. Ruiz JG, Mintzer MJ, Leipzig RM. Acad Med. 2006 Mar;81(3):207-12. Learning Effectiveness: What the Research Tells Us - Karen Swan in Elements of Quality Online Education: Practice and Direction Edited by John Bourne & Janet C. Moore Copyright ©2003 by Sloan-C™

35. Outline Review evidence for new technologies in quality improvement A “Communities of practice” example Performance improvement learning How technology drives and challenges learning Questions