Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

PNB 4D09 - Thesis Final - Kaitlyn Gonsalves - April 13 Submission

107 views

Published on

  • Be the first to comment

  • Be the first to like this

PNB 4D09 - Thesis Final - Kaitlyn Gonsalves - April 13 Submission

  1. 1. Running head: LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION INVESTIGATING LEARNING METHODS FOR SURGICAL PROCEDURES IN ORTHOPAEDIC RESIDENCY EDUCATION BY KAITLYN GONSALVES A Thesis Submitted to the Department of Psychology, Neuroscience & Behaviour In Partial Fulfillment of the Requirements for the Honours Bachelor of Science Degree McMaster University April 2016
  2. 2. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION ii Descriptive Note HONOURS BACHELOR OF SCIENCE (2016). MCMASTER UNIVERSITY Hamilton, Ontario. TITLE: Investigating learning methods for surgical procedures in orthopaedic residency education Author: Kaitlyn Gonsalves Supervisor: Dr. Ranil Sonnadara Number of pages: vii, 50
  3. 3. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION iii Abstract Surgical residents often use different forms of studying to understand complex material. Common forms of studying include writing or typing notes, reviewing a textbook, verbalizing or explaining information to another individual, watching videos, or listening to podcasts to learn complex material. Surgical residents are required to learn and perform a wide range of complex surgical procedures. Recent changes in the healthcare system and the transition to competency- based medical education (CBME) have resulted in medical educators seeking alternative teaching methods as a supplement to direct operating room (OR) observation. Our study compared textbook reading to a computer-based instructional video (CBVI) tool on surgical procedures to examine the effectiveness of video-based learning tools. We studied two procedures, ankle fracture and shoulder arthroplasty. Orthopaedic residents were split into two groups, where each group received a baseline quiz. Residents independently studied either the reading materials or the CBVI for a procedure, where order effects and mode of presentation were controlled for in the design of the methodology. Both groups wrote an identical knowledge- based quiz following the study period. One month after studying the procedures, residents received an online retention test based on content from both procedures. A repeated measures Analysis of Variance (ANOVA) was used to analyze the scores from the baseline, knowledge, and retention quizzes. There was no significant difference between the quiz scores for participants who studied via textbook or video modes for a procedure. By examining the effectiveness of video-based learning tools for surgical procedures, it is our hope that CBVI tools support the need for alternative teaching methods, which can be incorporated into modern competency-based medical education for the 21st century.
  4. 4. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION iv Acknowledgements I would like to thank all the individuals in the Sonnadara lab and in the MultiSensory Perception Lab, especially my fellow thesis students, for their guidance and support throughout this long journey. To my family and close friends, thank you for your unwavering support and patience while I worked away on my thesis throughout this past year, thank you for the motivation to keep me going, and thank you for being a light through all my struggles. A special thank you to Dr. Ranil Sonnadara, Dr. David Shore, Natalie Wagner, and Brendan Stanley for being pillars of support. Thank you for your expertise, your kindness, and your wisdom throughout this journey. I am infinitely thankful and grateful. They have all undoubtedly encouraged and supported me through this project. Thank you for guiding me through this tough year, without you, I wouldn’t be where I am today.
  5. 5. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION v TABLE OF CONTENTS DESCRIPTIVE NOTE………………………………………………………………………..…ii ABSTRACT……………………………………………………………………………..………iii ACKNOWLEDGMENTS……………………………………………………………………....iv INTRODUCTION…………………………………………………………………….…………8 The traditional medical education curriculum…………………………………………….9 Problems with the traditional model highlight a need for reform………………………..11 Competency-based medical education curriculum (CBME)………………….…………14 Psychological foundations for learning and video-based learning………..……………..18 Using video instruction as an alternate teaching method within CBME curriculum…....21 METHODS……………………………………………………………..………………….……25 Participants……………………………………………………………..……………...…25 Procedure ……………………………………………………………..…………………26 RESULTS………………………………………………………..……………………………...29 DISCUSSION………………………………………………………..…………………….……39 Limitations………………………………………………………..………………..…….43 CONCLUSION………………………………………………………….……………..….……46 REFERENCES………………………………………………..………………………………...47
  6. 6. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION vi Table Caption Table 1. Methodology of data collection……………………………………………………..…28 Table 2. Day 2 demographics and descriptive statistics…………………………………………30 Table 3. Day 1 demographics and descriptive statistics…………………………………………32 Table 4. ANOVA analysis: within-subject factors and between-subject factors……..…………33 Table 5. ANOVA analysis: descriptive statistics…………………..……………………………34 Table 6. ANOVA analysis: tests of within-subjects effects……..………………………………38 Table 7. Descriptive statistics on demographic questionnaire from previous experience…...….40
  7. 7. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION vii Figure Caption Figure 1. Ankle Fracture Procedure showing the scores across time………………..………….35 Figure 2. Shoulder Arthroplasty Procedure showing the scores across time………..…………..36 Figure 3. Scores for ankle fracture and shoulder arthroplasty procedure ……......………...…...37
  8. 8. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 8 Introduction In medicine, there are often numerous long hours of studying material, reviewing case studies, preparing for evaluations, and clinical procedures, which creates a demanding environment for the medical student. On the path to becoming a medical practitioner, there is about a 10-year timeline consisting of numerous hours of studying, practicing clinical skills, and completing evaluations. After a minimum of three years of university, students start medical school, and will spend three to four years learning knowledge, skills, and professional attitudes, while applying these skills in the clinical setting as part of a health care team. Within Canada and the United States, students graduate from medical school with a Doctor of Medicine (MD), and begin their post-graduate work as residents who train in a specific speciality (for example, paediatrics, family medicine, orthopaedics). Residents will do a variety of clinical rotations in various hospitals and health care facilities under supervision of their residency program. Surgical residents spend approximately three to seven (or more) years working in the hospital learning the different specialities and focusing on the speciality of their choice (Hodges, 2010). They spend extraordinarily long hours in the hospital setting taking care of patients, interpreting test results, and reviewing case studies, in addition to learning clinical skills and understanding how to perform surgeries in the operating room (OR) (Sonnadara et al., 2014). The traditional medical education curriculum has prepared medical students for over a hundred years (Irby, Cooke, & O'Brien, 2010), yet the current curriculum does not provide a flexible approach to teaching modern physicians. There is a need for more flexible and effective approaches to prepare future medical practitioners for modern medicine.
  9. 9. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 9 The traditional medical education curriculum Prior to 1910, medical education lacked a rigorous and standardized approach. Students were taught by unqualified faculty members who were local doctors teaching to supplement their income (Irby, Cooke, & O'Brien, 2010). Faculty members gave passive lectures, which did not include opportunities to apply the knowledge to patient care and there was limited interaction with patients (Irby, Cooke, & O'Brien, 2010). Abraham Flexner set the standard for the medical school curriculum in the 1900s since it is primarily based on his recommendations from Flexner’s classic 1910 report on educating physicians (Irby, Cooke, & O'Brien, 2010). He created recommendations for medical education to be scientifically grounded within a university atmosphere and a teaching hospital (Irby, Cooke, & O'Brien, 2010). Flexner’s 1910 recommendations transformed medical education to a more rigorous scientific standard for North American medical schools. This revolutionary change is well known as a ‘Flexner revolution’ and it stood as the first extensive and large-scale reform in American and Canadian medical schools in the 1920s (Hodges, 2010; Irby, Cooke, & O'Brien, 2010). Flexner’s recommendations in the 1900s set the standard for the traditional medical education curriculum. The traditional medical education curriculum is known as the time-spent model of medical education curriculum, as the underlying assumption is that students will become competent medical practitioners by being immersed in the clinical setting within a fixed interval of time (Hodges, 2010). The rigid timelines that make up the traditional medical education curriculum is the same model used for postgraduate surgical residency programs (Irby, Cooke, & O'Brien, 2010). The postgraduate surgical education curriculum used for residency programs requires residents to spend a fixed time period in a clinical setting during the program (Hodges, 2010; Sonnadara et al., 2014). During a fixed interval of time, residents are expected to work on patient
  10. 10. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 10 cases, attend formal teaching in classroom settings and classroom-based learning activities, observe and practice surgeries, as well as master clinical skills, all while learning under the supervision of experienced clinical staff members (Irby, Cooke, & O'Brien, 2010; Sonnadara et al., 2014). This supervision relies on the master-apprentice approach, where senior residents, attending physicians, and senior physicians (for example, the “masters”) teach and train junior residents (the “apprentice”) in the hospital (Dawson & Kaufman, 1998). The master-apprentice approach has shown to be inefficient as residents have to train for years to be exposed to a full range of surgical procedures (Dawson & Kaufman, 1998). The time-spent model has shown to be resistant to change over time, as there have been few modifications over the past 100 years (Hodges, 2010). Notable modifications to the model include early clinical exposure and the addition of problem-based learning (Hodges, 2010). The common assumption is that the fixed length of time for a residency program is sufficient for a resident trainee to develop competency (where they must successfully showcase the appropriate abilities for a task) (Hodges, 2010). This is not always the case. There is a lack of evidence surrounding the link between length of time spent in a training program and developing competence (Hodges, 2010; Sonnadara et al., 2014). The sole factor determining graduation is the length of time spent in the residency training program; although other assessments take place during the program, it usually does not interfere with one’s progress in the program (Hodges, 2010). Under extreme circumstances, if a resident is clinically incompetent and fails multiple assessments, they will not be able to graduate from the residency program. Currently, the traditional medical education curriculum offers many concerns, as it is outdated and it is grounded in rigid program guidelines making the curriculum inflexible (Irby, Cooke, & O'Brien, 2010).
  11. 11. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 11 Problems with the traditional model highlight a need for reform The traditional medical education curriculum or time-spent model offers numerous problems and concerns regarding the competency of trainees in the program. The following will discuss current concerns and issues within the traditional curriculum. Surgical residents and surgical residency program directors have both expressed concerns regarding the level of preparedness of residents to practice independently upon graduation (Bell et al., 2009). In a study by Bell et al. (2009) graduating general surgery residents reported an average experience of completing nine essential surgeries approximately 20 times during their residency. Approximately 121 essential surgical procedures were chosen by residency program directors as essential for residents to practicing general surgery, yet only nine surgeries were reported by graduating residents. It is clear the operative experience of surgical residents was not at the level of basic competency, yet the program directors believe residents should be able to perform these essential procedures independently upon graduation (Bell et al., 2009). The traditional curriculum does a poor job of ensuring that surgical residents are fit for independent practice, as residents have less experience in completing independent surgeries (Bell, Banker, Rhodes, Biester, & Lewis, 2007). Additionally, it is possible that attending surgeons are making decisions on behalf of the residents, in comparison to letting residents independently make their own decisions (Bell et al., 2007), therefore the current model does a poor job of ensuring residents are competent in these areas and inadequately prepares residents for independent surgical practice beyond graduation. The traditional curriculum has a number of internal limitations within its current structure. Currently, residents are working shorter weeks in teaching hospitals due to a reduction in their work hours; a previous maximum of 100 hours per week is currently reduced to 80 hours per
  12. 12. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 12 week (Bell et al., 2007; Whang, Mello, Ashley & Zinner, 2003). A reduction in work hours limits the amount of exposure to valuable teaching time in the hospital (Sonnadara et al., 2014; Reznick & MacRae, 2006). Residents learn through a variety of methods, such as: viewing and performing surgeries in the operating room (OR), and working on clinical cases under the supervision of their senior supervisor. The time-spent model uses a fixed interval of time and assumes overall competency in the speciality upon graduation, yet the current structure limits the amount of time residents can learn through valuable methods, such as viewing and performing surgeries, as well as learning under the supervision of their supervisor. Residents have a limited amount of time to learn new surgeries in the OR during their work week, resulting in residents reporting less exposure to view and perform surgeries, while program directors often require more exposure to procedures to demonstrate competence (Bell et al., 2009; Sonnadara et al., 2014). In addition, there is a high demand for surgeries in the OR to become more efficient. Here, efficiency in the OR negatively impacts residents as they do not have sufficient time to directly view and practice performing surgeries on a human patient (Sonnadara et al., 2014; Reznick & MacRae, 2006; Van Eaton et al., 2011). A limited amount of direct OR observations forces residents to practice clinical skills through stimulation, due to an increased need for patient safety, and less time working in real clinical situations with human patients. Residents should gain more exposure to direct OR surgeries, as it is assumed they have developed competency through actually performing enough clinical cases during the time spent in the residency program (Sonnadara et al., 2014; Reznick & MacRae, 2006). The fixed period of time for a residency program has not shown to help individuals develop competency. In addition, evaluating a resident’s competence with clear outcomes has never been clearly defined within the traditional curriculum (Long, 2000). Successful completion of residency programs is widely
  13. 13. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 13 based upon the time spent on clinical rotations, without taking into account the abilities acquired, and disregarding competency (Carraccio, Wolfsthal, Englander, Ferentz & Martin, 2002). Thus, without clear and defined outcomes that can be assessed, there are no valid or reliable measures for assessing competency in residents (Hodges, 2010). Residents are expected to master clinical procedures, while their supervisors have less time to focus on teaching residents important clinical skills and surgical procedures, due to an increasing demand for clinical supervisors’ time with additional administrative work (Irby, Cooke, & O'Brien, 2010; Ruiz, Mintzer, & Leipzig, 2006). Overall, a reduction in resident working hours, less exposure to surgeries, and clinical staff who have limited time to teach residents, results in fewer opportunities for residents to learn and presents serious obstacles for traditional or time-spent residency programs to overcome (Irby, Cooke, & O'Brien, 2010; Sonnadara et al., 2014). Residents must seek alternative strategies to develop competence in performing surgeries, as today’s patients have more advanced and complex clinical cases, and there is a greater emphasis on optimal performance with minimal errors (Reznick & MacRae, 2006). There must be alternative methods that provide surgical residents with adequate experience to learn and perform surgical procedures, in order to ensure they become competent surgeons. A potential solution proposed to extend the length of residency programs (Sonnadara et al., 2014). It is thought the additional time would allow residents to gain enough exposure to surgical procedures and experience adequate teaching time. Yet, the length of residency programs is already long and extending the time-spent residency program does not address the problematic issues within its flawed and inflexible structure (Irby, Cooke, & O'Brien, 2010). The time-spent curriculum does not provide flexibility for the trainee to learn at their own pace and the curriculum does not prioritize a learner-centered approach. Individualized learning provides greater flexibility to
  14. 14. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 14 center teaching around the trainee, by providing an individualized and learner-centered approach, the trainee gains a more beneficial learning experience that can benefit them long-term—yet this is currently not the case with the time-spent model (Irby, Cooke, & O'Brien, 2010). Lastly, the time-spent model does not incorporate modern and innovative practices in teaching and learning. Through decades of research, theories in teaching and learning have evolved tremendously to become more established. We have collectively gained a better understanding of how we can learn material for the long-term retention, and we can work towards using best practices for teaching and learning clinical knowledge to improve patient- based care. New, effective, and innovative ways to teach students to learn have not been incorporated into the traditional medical education curriculum. The current issues arising from the traditional model highlight the need for a medical education reform to a more flexible design that includes effective learning methods. To account for problems with the traditional medical education curriculum, we can find solutions by using modern, flexible, and outcome-based approaches to training residents, such as the competency-based medical education curriculum (CBME) (Sonnadara et al., 2014). In the past 20 years there has been a shift towards transforming the traditional curriculum into a more modern competency-based approach (Frank & Danoff, 2007). Competency-based medical education curriculum (CBME) There has been a movement to reform medical education in Canada and the United States. Medical education is shifting towards a CBME curriculum for residency programs. CBME can be defined in a variety of ways, and Frank et al. (2010) produced a definition based on analyzing 173 definitions of competency-based education and identifying common
  15. 15. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 15 themes among them (Sonnadara et al., 2014). CBME can be defined as a medical education approach based on graduate outcome abilities and competencies that have been derived from society and patient needs (Frank et al., 2010). CBME promotes a flexible style of teaching that is centered around the learner and accountability, with less focus on the time spent in the program (Frank et al., 2010). Competency-based frameworks in medicine have aided in transforming the time-spent model into a modern competency-based education (Iobst et al., 2010). In the 1990s, groundwork was laid for the Canadian Medical Education Directions for Specialists or “CanMEDS” initiative that was started by the Royal College of Physicians and Surgeons of Canada. It quickly became one of the most important and influential competency-based medical education frameworks in medicine (Frank & Danoff, 2007; Sonnadara et al., 2014). The CanMEDS initiative analyzed the demands of modern medical practitioners: who need to be able to meet the diverse needs of patients, their communities, and their societies they interact with to provide the best health care (Frank & Danoff, 2007). From an analysis of patient and societal needs, the CanMEDS initiative defined key outcome-based competencies, and grouped the competencies together into seven clear roles of a physician to meet society’s needs (Frank & Danoff, 2007; Sonnadara et al., 2014). The CanMEDS initiative developed an influential competency-based framework (Frank & Danoff, 2007). The competency-based approach aims to prepare residents for clinical practice by focusing on successful completion of specific graduate outcome abilities and competencies (Iobst et al., 2010). Residents can demonstrate successful abilities and competency through outcome-based assessments (Frank & Danoff, 2007; Sonnadara et al., 2014). Residents must be able to demonstrate they are competent on all aspects of their residency training (Iobst et al.,
  16. 16. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 16 2010). The older CanMEDS 2005 framework has been revised to include changes such as providing simple and clear language, and minimizing overlap between roles. The revisions are included in the CanMEDS 2015 framework. The CanMEDS 2015 framework provides seven key roles of a physician: medical expert, communicator, collaborator, leader, health advocate, scholar, and professional (Frank, Snell, & Sherbino, 2015). Each of these roles has a set of specific competencies related to the role (Frank & Danoff, 2007). Each of the competencies within each role should be taught and assessed by residency programs. Residents should be able to demonstrate these competencies upon graduation from the residency program (Sonnadara et al., 2014). The Royal College of Physicians and Surgeons of Canada have incorporated the CanMEDS initiatives as an essential part of Canadian medical residency education (Frank & Danoff, 2007). Establishing a competency-based framework can provide medical educators with the tools to shift towards adopting a competency-based model for medical education and surgical residency programs. A model of CBME provides a wide range of benefits for residents in training. The CBME for residency programs focuses on accomplishing competencies based upon abilities and de-emphasizes the time spent in the residency program (Iobst et al., 2010). With recent changes to the healthcare system and previous issues from the traditional model, implementing CBME will provide greater accountability for residents and clinical staff by ensuring residents are able to successfully complete outcome-based competencies through frequent assessments. Frequent assessments provide residents with more opportunities to learn, as residents must demonstrate competency on assessments and tests to successfully move forward within the residency program, thus providing supervisors with more confidence in the residents’ capabilities (Sonnadara et al., 2014). It is key to assess and evaluate residents’ abilities
  17. 17. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 17 in providing direct patient care, as it is one of their main responsibilities as a resident (Iobst et al., 2010). Thus, the CBME framework for residency programs is centered around individualized learning and focused on a learner-centered approach (Irby, Cooke, & O'Brien, 2010). As CBME provides a more flexible framework with a learner-centered approach, it can incorporate instructional methods that use the most effective strategies for residents to understand concepts and learn clinical skills. The CBME framework offers greater flexibility for residents to learn the curriculum, as residents can move through the program at a quicker or slower pace, depending on how long it takes them to acquire the necessary skills to demonstrate competency on assessments (Holmboe et al., 2010). Residents will acquire competency for skills at different rates, as competency is based on individual progress (Carraccio et al., 2002). Frequently practicing important skills and performing them can help residents to develop competency, so it is important for residents to successfully complete frequent assessments and gain feedback on their performance. The CBME model emphasizes continuous, complete, and detailed assessments incorporated with frequent feedback to assess the resident throughout the program (Holmboe et al., 2010). In comparison to the traditional curriculum, the CBME curriculum will benefit residents who have gaps in specific areas of clinical knowledge, skills, and professional attitudes. By providing continuous feedback and frequent assessments, residents will be able to see the gaps in their knowledge, well in advance of major assessments. Thus, residents and clinical staff members can work towards an action plan for help residents gain competency in areas of weakness (Holmboe et al., 2010). The described benefits of the CBME framework provide solutions to the current problems within the traditional time-spent curriculum.
  18. 18. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 18 The CBME framework can provide flexible and learner-centered solutions to the problems with the traditional model, such as limited work hours, reduced direct OR teaching time, and limited time of clinical staff to reach residents. The CBME model will allow for a flexible learning environment where residents will be able to learn at their own pace (Irby, Cooke, & O'Brien, 2010). Since the competency-based model of medical education offers a flexible approach centered around the learner, we should look to using innovative and alternate methods of learning that are efficient, including methods that offer the best retention for learning complex clinical skills and procedures. Understanding the foundations of how learning works is crucial to seeking alternative methods for residents to learn complex surgeries and important skills. In the context of alternative methods for teaching and learning, we must first understand teaching and learning in the field of health education. Psychological foundations for learning and video-based learning Medical education should be informed by research based theories that understand how students learn, and use effective instructional teaching methods guided by evidence-based principles (Mayer, 2010). Learning is often described as a change in the learner’s knowledge due to experience (Mayer, 2008; Mayer, 2009; Mayer, 2011). Learning in medical education involves multimedia learning, which is the combination of learning from words and pictures (Mayer, 2010). A well-established theory based on learning from words and pictures is known as the cognitive theory of multimedia learning (Mayer, 2005; Mayer 2009). This theory is based on key cognitive science principles, which emphasize that we have two different information processing systems: auditory-verbal channel and visual-pictorial channel; both hold a limited amount of information (Mayer, 2010). The cognitive theory of multimedia learning proposes three memory
  19. 19. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 19 systems: a sensory memory that holds an exact copy of the presentation, working memory that holds a few items at a given time, and long-term memory that holds knowledge for longer periods of time (Mayer, 2010). Both the sensory and long-term memories have an unlimited capacity, yet working memory has a limited amount of information that it can hold (Mayer, 2010). The capacities of the memory systems come into play when information is being processed. Spoken words and pictures are processed separately as they enter the sensory memory, and move into working memory (Mayer, 2010). It is within working memory that the two channels (verbal and pictorial) are combined to create a holistic interpretation, which is integrated with previous knowledge from long-term memory (Mayer, 2010). Learning takes place through active cognitive processes, such as selecting, organizing, and integrating words and pictures within each channel, respectively (Mayer, 2010). Processing information through two different channels (which process stimuli-specific information) helps to reduce cognitive load and makes it easier to integrate stimuli-specific information leading to understanding complex material (Mayer, 2010). By understanding how the learner processes information, we can shape medical education to include teaching methods that align with how we process complex multimedia information. Successful teaching methods initiate a change within the learner’s knowledge allowing for learning to occur (Mayer, 2010). In order to initiate a change in the learner, there must be a clear objective stating what is being taught, what level of expertise must the learner achieve, and how will the learner be assessed (Mayer, 2010). Without these clear objectives, it is difficult for the learner to understand the content, level of expertise expected, and what is expected of them in assessment—these are clear issues within the traditional time-spent curriculum. With clear objectives, one can design instructional materials that aid the learner in processing complex
  20. 20. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 20 information. Well-designed instructional materials and teaching methods should include three main goals: use only necessary content and minimize inessential information to avoid extraneous (or unnecessary) processing, a level of complexity where the learner has enough cognitive capacity to process information, and the learner has motivation to understand the given content (Mayer, 2010). Mayer (2010) discussed the underlying principles that govern the accomplishment of these goals. Firstly, to reduce extraneous processing: instructors can eliminate unnecessary material, highlight critical concepts, and place words near their respective image. Secondly, to manage complexity: instructors should teach the key concepts in advance, separate lessons into multiple parts, and use words in verbal form. Thirdly, to manage motivation: instructors should present both pictures and words together, use conversation-style to present words and use a human voice (compared to a machine-generated voice) (Mayer, 2010). These principles help to guide multimedia learning to be an effective form of learning that coincides with how humans process complex information. Mayer (2010) has described in-depth the importance of well- designed teaching methods and instructional materials. Well-designed teaching methods and instructional materials will benefit novice and experienced trainees, as with practice, they can develop expertise and combine concepts into more complex ideas with ease (Van Merriënboer & Sweller, 2010). Using well-designed multimedia instructional tools can benefit learners to understand complex concepts, such as surgical procedures. Using video instruction as an alternate teaching method within CBME curriculum Well-designed multimedia instruction provides the learner with a video or pictures, and spoken or printed instructions—these factors reflect real life actions and situations. In addition,
  21. 21. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 21 video instruction enhances learning to go far beyond verbal explanation or printed text (Greenhalgh, 2001). Using multimedia and video instruction provides a level of clarity that can explain complex material in a more interactive manner using video and text, as compared to passively reading the same content in printed text. The field of computer technology, multimedia learning, and video instruction support various ways of learning content, which provides versatility and flexibility when teaching content (Ruiz et al., 2006). Video instruction can allow the learner to fast forward through the video, skip parts, and re-watch parts of the video to review if they needed clarification. These options can be used to different extents by learners as they progress at their own pace. Residents can use these options to help them learn a variety of concepts. Advantages of multimedia learning include greater accessibility to content as learners can adjust the pace and the time necessary to understand the material (Ruiz et al., 2006). Clinical faculty and learners both agree that multimedia learning can enhance teaching and learning (Ruiz et al., 2006). In addition, multimedia content can also include assessments throughout a video or presentation to evaluate if the student has understood the material (Ruiz et al., 2006) and it can provide a check-in regarding what they have learned. Multimedia learning provides solutions to the previously stated issues with the traditional time-spent medical education curriculum, such as the reduced resident work hours and limited teaching time from clinical staff. Multimedia learning provides a flexible approach that can be incorporated into the CBME curriculum. Learning using computer technology provides a convenient, flexible and personalized learning experience for students to absorb material at their own pace (Greenhalgh, 2001). In addition, multimedia content can be updated to match and reflect changing attitudes, new research findings, and additional skills. Multimedia is a convenient, flexible, and efficient learning
  22. 22. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 22 experience for residents, allowing residents anywhere in the world to review the same material while still gaining a personalized learning experience. Multimedia, computer and internet technologies, and computer-based video instruction (CBVI) have been widely used in the context of medical education (Larvin, 2009). Learning via computer and internet technologies has made medical e-learning an important priority in the UK Department of Health. The UK Academy of Medical Royal Colleges have created e-learning programs for various health care services (Larvin, 2009). The UK Academy also recommended collaborating on e-learning ideas and sharing e-learning resources across health care professions (Larvin, 2009). In addition, the UK Academy also recommended online assessments that are directly related to expected learning outcomes and common competencies, in hopes of making online assessments more reliable than one-time examinations (Larvin, 2009). Using technology for learning or e-learning creates a huge potential for educating residents in surgical training, compared to any other medical specialities (Larvin, 2009). Multimedia learning benefits medical students learning surgical techniques. In a study by Dubrowski & Xeroulis (2005), with 21 medical students, the authors investigated self-directed learning skills by giving students a CBVI tool of the procedure (which was optional and they were encouraged to use it for learning) but it was not necessary to use the CBVI to complete the task. The students had to complete a 1-hour session on how to close wounds with suturing instruments and knot tying techniques. The CBVI contained two versions of the video: 1) presentation video involved slow speed and narration from an expert surgeon discussing proper use of tools, tips on good techniques to use, common errors, and the overall performance of the wound closure with suturing and knot tying techniques, 2) presentation involved a real-time video with narration by an expert surgeon. Results indicated that medical students viewed
  23. 23. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 23 sections of the slow presentation more, in comparison to the real-time video. The slow video was used extensively during the practice sessions (Dubrowski & Xeroulis, 2005). Both videos were shown to be useful as medical students learned surgical techniques, yet the slow presentation highlighted section that were more beneficial to new learners, while the real-time presentation would benefit experts (Dubrowski & Xeroulis, 2005). The study by Dubrowski and Xeroulis (2005) highlights the added advantage of using CBVI, as compared to learning without using CBVI. With the additional use of CBVI, the entire video or sections of the video can be re- watched and re-played while the learner practices the task. In an academic environment, individuals who used CBVI or video-based tools learned more efficiently and exhibited better retention, as compared to more traditional teaching strategies (Larvin, 2009). The Royal College of Surgeons of England valued e-learning to the extent of revising their Surgical Education and Training Programme (STEP) in 2001 and including an e-learning component (Larvin, 2009). Computer-based video instruction should not replace traditional methods, but it should complement current teaching methods in the CBME curriculum (Ruiz et al., 2006). Multimedia and video-based instruction provides the material in a flexible format where one can learn from any place with access to the video. Multimedia instruction provides an added advantage for surgical residents who work multiple shifts and a considerable number of hours in a day. As the time with clinical supervisors is limited and the work hours in a week are reduced, alternate methods for learning complex material must be sought, such as using computer-based video instruction and e-learning for complex concepts, especially when a considerable amount of studying must be done off duty (Larvin, 2009). Instructional video-based learning offers experiences similar to real life practice, which contrasts the act of passively reading textbook material. Residents have studied textbook content
  24. 24. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 24 for numerous years on their path to becoming a medical practitioner. Yet, studying complex material from a textbook takes much longer than watching a multimedia video. It is possible using video-based instruction could provide an alternative and effective learning method for residents, especially given the time constraints with the traditional curriculum. Based on the need for alternative methods for teaching residents complex surgical procedures, we will compare two study methods for residents, instructional videos of surgical procedures with voiceover instructions and traditional textbook readings to determine which method provides better knowledge retention. We can use retention tests to assess how well the learner retains information over time, based on information that was previously presented to them (Mayer, 2010). The purpose for this study is to investigate the most effective method for studying complete and complex surgical procedures, by comparing if textbook material or video- based material is more successful.
  25. 25. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 25 Methods We will use four surgical procedures in this study: Anterior Cruciate Ligament (ACL) repair, shoulder arthroplasty, elbow arthroscopy, and ankle fracture. Participants will study the procedures through different resources, either studying a procedure using textbook material, or studying a procedure using the 10 to 20 minute surgical video with instructional voiceover from a staff surgeon. Participants The participants were 18 orthopaedic surgical residents across all post-graduate years (PGY) 1 through 5 from the McMaster Orthopaedic Program. From the 18 orthopaedic residents, all were male, while the mean PGY was 2.88 years (SD 1.32 years). The residents were recruited with support from the program director and program coordinator of the McMaster Orthopaedic Program. On Day 1 of data collection, the 18 residents who showed up were randomly assigned into two groups: Group A and Group B. Group A and B were separated from each other to ensure participants completed the study individually. Two participants were excluded from the data. One participant left halfway through the study period, and another participant only completed one procedure on each data collection day, instead of the required two surgeries due to being late on both days. For these reasons, the two participants were excluded from the results. Both individuals were previously assigned to Group A. After excluding the two participants from the results, Group A had 8 residents and Group B had 10 residents. There were two days of data collection: Day 1 and Day 2.
  26. 26. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 26 Procedure All participants completed a consent and demographic questionnaire. All participants had 10 minutes to complete a baseline quiz on ankle fracture and shoulder arthroplasty procedures, in order to gauge their previous knowledge prior to completing the study. On Day 1 of data collection, all participants studied two procedures: ankle fracture and shoulder arthroplasty. Group A learned ankle fracture by reading the text material, while Group B learned ankle fracture by watching the video with voice instruction. All participants had 15 minutes for the study period. Immediately after the study period, participants had 5 minutes to complete a short- answer knowledge based quiz to test if they gained any knowledge from the material. Next, Group A learned shoulder arthroplasty by watching the video with voice instruction, while Group B learned shoulder arthroplasty by reading the text material. All participants had 20 minutes for the study period, and they had 5 minutes to complete a short-answer knowledge based quiz. On Day 2 of data collection, all participants studied the final two procedures: elbow arthroscopy and ACL repair. All participants had 10 minutes to complete a baseline quiz on elbow arthroscopy and ACL repair procedures, in order to gauge their previous knowledge prior to completing the study. Group A learned elbow arthroscopy by watching the video with voice instruction and completing a quiz, while Group B learned elbow arthroscopy by reading the text material and completing a quiz. The study periods for elbow arthroscopy and ACL repair were both 15 minutes with 5 minutes to complete the knowledge quiz. Next, Group A learned ACL repair by reading the text material and completing a quiz, while Group B learned ACL repair by watching the video with voice instruction and completing a quiz. On each data collection day, both Group A and Group B had one hour to complete the two baseline quizzes, study two
  27. 27. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 27 procedures and complete the procedure-specific knowledge quizzes. By the end of data collection Day 2, both groups have learned four surgical procedures in total, and each group would have only learned a procedure either by text or video material. The methodology is best described visually, as seen in Table 1. One-month post-data collection, an online retention test was administered to all participants via the McMaster LimeSurvey platform. The goal of the retention test was to see if participants retained the knowledge they previously studied on the two data collection days encompassing all four procedures. The retention quiz included six to seven questions per procedure. The participants received a personalized invitation via an e-mail with a link to the closed survey. Participants had approximately five days to complete it. Participants received reminder e-mails approximately 12 hours before the midnight deadline. All factors have been taken into consideration and they have been controlled for in the development of the methodology to ensure they do not influence the results. The text reading material, including questions and answers for baseline, knowledge, and the retention quizzes were created by the senior orthopaedic staff surgeon and orthopaedic resident on this project. Each procedure specific quiz had approximately six or seven questions. The quizzes were all in the form of short answer as to be fair to both groups who studied using either textbook or video material. If the quizzes were multiple choice, participants who studied a surgery via textbook material might be able to recognize the written correct answer. Recognizing words you were previously exposed to leads to recognition memory and the repetition effect, which would be far greater for words that were studied and later tested (Goldinger, 1996; Hintzman, Block, & Inskeep, 1972). Thus, if the quizzes were multiple choice, participants who studied via textbook material would gain an unfair advantage leading to skewed results.
  28. 28. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 28 Table 1. Methodology of data collection Data Collection Group 1 (A) Time Allocated Group 2 (B) Time Allocated Day 1 (February 3rd 2016) Baseline Quiz 10 mins Baseline Quiz 10 mins Ankle Fracture (text) Study - 15 mins Ankle Fracture (12 min video) Study - 15 mins Quiz - 5 mins Quiz - 5 mins Shoulder Arthroplasty (18 min video) Study - 20 mins Shoulder Arthroplasty (text) Study - 20 mins Quiz - 5 mins Quiz - 5 mins Day 2 (February 17th 2016) Baseline Quiz 10 mins Baseline Quiz 10 mins Elbow Arthroscopy (12 min video) Study – 15 mins Elbow Arthroscopy (text) Study – 15 mins Quiz – 5 mins Quiz – 5 mins ACL Repair (text) Study – 15 mins ACL Repair (14 min video) Study – 15 mins Quiz – 5 mins Quiz – 5 mins Day 3 Retention test (March 8th –12th 2016) LimeSurvey Quiz: Procedure 1 Procedure 2 Procedure 3 Procedure 4 LimeSurvey Quiz: Procedure 1 Procedure 2 Procedure 3 Procedure 4
  29. 29. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 29 Results Since one group studied a procedure via text, and the other group studied the same procedure via video, we investigated if there are differences in the information retained by the study method. We compared baseline quiz scores, knowledge quiz scores immediately after learning, and retention quiz scores one month later, across both groups and all procedures. We will use this as a link to understand which method of studying (text or video) was the most effective for long-term retention, which we hope will benefit surgical residents when learning complete surgeries. Data from Day 2 of data collection was excluded from data analysis, as a random assortment of participants returned for the second part of the study, resulting in an imbalance between groups on Day 2. As Group A had two participants, and Group B had seven participants, it created an imbalance, resulting in insufficient data to run further analysis (see Table 2 for descriptive statistics). The research question considers if computer-based video instruction (CBVI) is an effective tool for teaching complete and complex surgeries in surgical residency programs. To assess the primary research question, data were analyzed using a repeated measures Analysis of Variance (ANOVA) design. It was chosen to compare which of the two study methods is more effective over time, as all residents participated in learning both surgical procedures. The dependent measures were the quiz scores for the baseline, knowledge, and retention quizzes. The within-subject factors were: time point (with three levels: baseline, immediate knowledge test, and one month later for the retention test), and procedure (with two levels: ankle fracture and shoulder arthroplasty). The only between-subjects factor was group (with two levels: group A and group B). This aided in understanding the differences among groups receiving either
  30. 30. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 30 Table 2. Day 2 demographics and descriptive statistics Group A Group B Number of participants 2 7 Mean PGY (years) 2.5 2.71 SD of PGY (years) 2.12 1.50 Median PGY (years) 2.5 3 Gender All male All male
  31. 31. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 31 textbook or video material for a procedure. Statistical significance at p<0.05 was considered significant. The statistical software used was SPSS IBM version 23. Demographics on Day 1 from both groups indicated that Group A had 8 participants with a mean PGY of 3.1 years (SD of PGY was 1.36 years), and a median PGY of 4 years. Group B had 10 participants with a mean PGY of 2.8 years (SD of PGY was 1.32 years), and a median PGY of 3 years (see Table 3). Refer to Table 2 for descriptive statistics from Day 2 participants. The ANOVA looked at the scores of participants who completed the baseline, knowledge, and retention quizzes for ankle fracture and shoulder arthroplasty procedures. There was a total of n = 11 participants (see Table 4). The descriptive statistics including the mean and standard deviation of the quiz scores for ankle fracture and shoulder arthroplasty procedures can be found in Table 5 (to see a graphic representation, see Figure 1 and 2). Across three time points for both surgeries, participants had statistically significant changes in their scores showing a main effect of time F(2,18) = 10.30, p = 0.001 (see Figure 3). All other factors were not significant and outliers have been removed (see Table 6).
  32. 32. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 32 Table 3. Day 1 demographics and descriptive statistics Group A Group B Number of participants 8 10 Mean PGY (years) 3.1 2.8 SD of PGY (years) 1.36 1.32 Median PGY (years) 4 3 Gender All male All male
  33. 33. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 33 Table 4. ANOVA analysis: within-subject factors and between-subject factors Within-subject factors Dependent variable Between-subject factor Procedure Time Group A Group B 1 – Ankle Fracture 1 Ankle fracture baseline quiz score N = 4 N = 7 2 Ankle fracture knowledge quiz score 3 Ankle fracture retention quiz score 2 – Shoulder Arthroplasty 1 Shoulder arthroplasty baseline quiz score 2 Shoulder arthroplasty knowledge quiz score 3 Shoulder arthroplasty retention quiz score
  34. 34. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 34 Table 5. ANOVA analysis: descriptive statistics Dependent variable Group: A or B Mean Standard deviation N Ankle fracture baseline quiz score A 46.88% 21.35% 4 B 42.86% 6.68% 7 Ankle fracture knowledge quiz score A 67.50% 22.17% 4 B 80.00% 17.32% 7 Ankle fracture retention quiz score A 62.07% 6.56% 4 B 46.88% 26.93% 7 Shoulder arthroplasty baseline quiz score A 56.25% 12.50% 4 B 39.29% 9.27% 7 Shoulder arthroplasty knowledge quiz score A 67.86% 21.43% 4 B 71.43% 27.36% 7 Shoulder arthroplasty retention quiz score A 67.71% 13.68% 4 B 63.20% 24.52% 7
  35. 35. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 35 Figure 1. Ankle Fracture Procedure showing the scores across time. Participant scores were collapsed within Group A and within Group B. 47% 68% 62% 43% 80% 47% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Baseline Quiz Knowledge Quiz Retention Quiz QuizScore Time Ankle Fracture Procedure Group A (text) Group B (video)
  36. 36. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 36 Figure 2. Shoulder Arthroplasty Procedure showing the scores across time. Participant scores were collapsed within Group A and within Group B. 56% 68% 68% 39% 71% 63% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Baseline Quiz Knowledge Quiz Retention Quiz QuizScore Time Shoulder Arthroplasty Procedure Group A (video) Group B (text)
  37. 37. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 37 Figure 3. Scores for ankle fracture and shoulder arthroplasty procedure are collapsed together across group, which highlights the main effect of time. 46% 72% 60% 0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100% Baseline Quiz Knowledge Quiz Retention Quiz QUIZSCORE TIME Ankle Fracture and Shoulder Arthroplasty Quiz Scores
  38. 38. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 38 Table 6. ANOVA analysis: tests of within-subjects effects Measure Degrees of freedom and F value Significance, p value Time F(2,18) = 10.30 0.001 Procedure F(1,9) = 0.54 0.48 Procedure*Group F(1,9) = 0.18 0.67 Time*Group F(2,18) = 1.77 0.20 Procedure*Time F(2,18) = 1.36 0.28 Procedure*Time*Group F(2,18) = 0.95 0.41 Note. Only the main effect of time was significant, all other factors were not significant.
  39. 39. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 39 Discussion Results show that across time, participants have statistically different scores, this indicates the textbook material and video material influenced their scores across the three time points. There was a non-significant difference across the 3-way interaction of time, group, and procedure. This indicates that textbook material and CBVI show similar learning outcomes. The main effect of time being significant from the ANOVA highlight that across time, participants had significant differences between their scores on the baseline, knowledge, and retention test across procedures (refer to Figure 3). This indicates that participants did better on the knowledge quizzes after reviewing either the textbook or video material; yet one study mode was not superior to the other, as they were roughly equal in providing the learner with knowledge to complete the quizzes. Prior experience and PGY were taken into account through the demographic questionnaire at the beginning of the study, and on the retention test. The demographic questionnaire asked about prior experience: how many times they have completed any of the four surgeries, and how many times have they watched (and not completed) any of the surgeries. The retention test questionnaire asked participants if they had completed or witnessed any of four surgeries during the 4-week period between the last study period and the retention test. This information provided valuable insight into the previous knowledge and experiences of the residents that they bring with them as they complete this study and prior to completing the retention test. Descriptive statistics on the demographic questionnaire and participants’ previous experience can be found in Table 7 and it is discussed below. Regarding the demographic questionnaire, participants who completed or witnessed a procedure in the range of 20-50 times, did relatively better on the respective procedure and
  40. 40. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 40 Table 7. Descriptive statistics on demographic questionnaire from previous experience Measure How many times have you actively participated in the following: How many times have you witnessed (and not actively participated) in the following: ACL Repair Shoulder Arthroplasty Ankle Fracture Repair Elbow Arthroscopy ACL Repair Shoulder Arthroplasty Ankle Fracture Repair Elbow Arthroscopy Mean 10.94 5.5 19.53 0.93 17.81 7.19 23.06 0.81 SD of mean 8.94 5.98 17.60 1.831 19.12 8.09 22.16 1.72 Median 10 4 10 0 8.5 5 11 0 Note. Participants entered in numbers 0–50 to quantify their previous experience for a procedure.
  41. 41. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 41 scored in the range of 70-80% on the respective procedure’s knowledge and retention test (example: a participant in PGY 4 had viewed the ankle fracture procedure approximately 50 times, scored 50% on the ankle fracture baseline quiz, scored 90% on the knowledge quiz, and 71% on the retention test after studying the ankle fracture video). Residents who have viewed more procedures tend to be in PGY 4 or 5, which is towards the end of their residency and they have much more experience than first or second year residents. Their vast amount of experience in completing and viewing specific surgeries over the years attributed to their higher scores on the quizzes for the respective surgeries. Regarding the retention test, approximately 11 participants completed the retention test at the time of analysis, yet of the 11 participants, only five of them had completed or witnessed one or more of the surgeries during the four-week period. It is thought that these five participants would score better on the retention test, considering they completed or witnessed these surgeries during the four-week period between the last study period and the retention test. About 3 of the 5 participants scored 70% and above on the retention test related to the surgeries they had completed or witnessed. The other two participants did somewhat worse (they scored 57% and 14% respectively) on the retention test related to the surgeries they had completed or witnessed. It is possible that our retention test answer key was specific to the teachings from one or two orthopaedic staff surgeons, so it was not inclusive to all possible alternate answers from other staff surgeons. The way that residents learn from experienced staff surgeons can differ, as it depends on the staff surgeon whom is supervising the resident, so residents may have learned the same surgery in slightly different ways. This can contribute to their different answers on the baseline, knowledge, and retention quizzes, and thus affected their scores on our dependent measure.
  42. 42. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 42 Participants in both Group A and Group B had variable scores after watching the video of a procedure compared with the textbook material—sometimes they scored much better and sometimes they scored much worse. In some cases, the video of the procedure helped the participant to score better on the knowledge quiz, yet it did not improve their score on the retention test a month later. One participant on Day 2, stated “watching the video of the ACL repair was so much easier.” However, this participant scored 70% on the baseline quiz for ankle fracture, 73% on the knowledge quiz, and 29% on the retention test. Through anecdotal conversations with participants, they seemed to enjoy and prefer the video of the procedure, yet they did not always improve on the respective knowledge and retention quizzes. Participants could enjoy multimedia and CBVI tools due to the convenience of replaying or stopping the video, yet it could also be less engaging, as they could prefer other forms of learning, such as live lectures, which should be further explored (Schreiber, Fukuta, & Gordon, 2010). In other cases, participants who watched the video of the procedure did relatively the same as their baseline, or worse. These instances where participants did worse on the knowledge and retention test could be explained by a few environmental and individual factors. Both data collection days were an hour earlier than the residents’ scheduled grand rounds, and participants could have been tired from an earlier on-call night (in one case). As well, there was no real motivation or drive for residents to participate in a research study as there were no direct benefits or compensation for their participation. This can be seen through participation on the retention test approximately two weeks after the deadline and analysis had to accommodate the late responses. These factors could all have negatively attributed to participants score on the quizzes.
  43. 43. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 43 Limitations Limitations of the study include a variety of factors, such as participant recruitment, and marking the data from the quizzes. The study was completed in three phases that needed all participants to return to all three phases. Recruiting participants to come in on time (as it was early in the morning) and to return on subsequent days of the study was a challenge. Only half the participants came back on Day 2 to study the final two procedures, greatly limiting the amount of full data sets we would have for analysis. As well, participants were randomly assigned to Group A or Group B, and they had to stay within their groups to receive the proper study mode for the given procedure. The groups on Day 2 were imbalanced, as only two participants came back from Group A, and seven participants came back from Group B. Recruiting participants for the retention test was another challenge, as only eight participants completed the retention test by the original deadline. Two weeks after the deadline, approximately seven participants completed the retention test. The majority of participants who completed the retention test did not complete data collection on Day 2, thus we did not have a full data set to compare their pre- and post-answers. Participants who did not return to complete all three phases of the study affected data analysis and it greatly contributed to the exclusion of Day 2 from the results. Our strong relationship with the orthopaedic program director aided us greatly in recruiting participants, yet it still remained a challenge to get the same residents to come out to all three data collection days to create complete participant data sets across all three phases. To resolve this, it would be crucial to advertise the study as three phases, where attendance at all three phases are required to participate, and include a compensation in monetary value or gift prize to be won at the end of the study.
  44. 44. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 44 A considerable barrier was individually marking the baseline, knowledge, and retention quizzes. The researcher marked all short answer quizzes for Day 1, Day 2, and the retention test. The answers from participants compared to the answer key highlighted only partial matches, and it was discovered that participants answered quite differently than the correct answer we were looking for on the answer key. This was a huge barrier in accessing correct scores for each quiz, as each alternate answer for each question on every quiz had to be double-checked by an orthopaedic staff surgeon. All alternate answers for each question on every quiz had to be recorded, and verified to be either correct or incorrect. This barrier directly affected the scores on the dependent variable, as personal judgment had to be used regarding each incorrect or correct response. It is possible some answers could have been marked incorrectly, due to using personal judgment and advice from the orthopaedic staff surgeon. This could be relieved through two or three individuals simultaneously marking all the quizzes in one session, and one of them would be the orthopaedic staff surgeon or resident on the project to advise on possible alternate answers. Future research and directions from this project could take on many forms. A secondary study from this research could compare textbook material and video material in combination, compared to text material only, and video material only. Other suggestions could be to compare how residents perform on live versions of these surgeries, and residents could be assessed in real- time using competency-based assessments from a staff surgeon. Since residents practice procedures in the OR during their time spent in residency, it would be worthwhile to investigate how residents in PGY 1 through 5 perform during live surgeries, while an orthopaedic staff surgeon assesses their competency using valid and reliable assessment tools, such as filling out a checklist and assessment during the live procedure. It would be interesting to see if
  45. 45. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 45 implementing feedback and not implementing any feedback from the staff surgeon plays a role in how residents perform during live procedures. A future study could compare the scores on the assessment tools to further understand how residents learn while in the OR, and information from this study can be compared to studying CBVI tools and textbook material for the same procedures.
  46. 46. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 46 Conclusion Although this project cannot definitively state that video materials are superior to text materials for studying complete and complex surgical procedures; surgical residents can continue to use traditional text material and articles for reference, and supplement their learning with additional instructional videos that can provide an engaging learning experience similar to completing the surgery live. The videos created for this project can be used by the residents in the McMaster Orthopaedic program to supplement their learning when they are not in the OR or in the hospital. The reading materials and videos can be available to them no matter where they are, especially for when they cannot be in the OR. Medical education has taken huge strides in the past few decades. The addition of ongoing research investigating best practices, optimal learning methods, and techniques for teaching surgical procedures to residents indicate that modern medicine is becoming more flexible to encompass the best practices for learning. The Royal College of Physicians and Surgeons of Canada and medical educators need continue to learn from one another to implement competency-based frameworks throughout all medical education programs and work together to provide the best possible learning environments for future medical practitioners.
  47. 47. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 47 References Bell, R. H., Banker, M. B., Rhodes, R. S., Biester, T. W., & Lewis, F. R. (2007). Graduate medical education in surgery in the United States. Surgical Clinics of North America, 87(4), 811–823. Bell Jr, R. H., Biester, T. W., Tabuenca, A., Rhodes, R. S., Cofer, J. B., Britt, L. D., & Lewis Jr, F. R. (2009). Operative experience of residents in US general surgery programs: a gap between expectation and experience. Annals of surgery, 249(5), 719–724. Carraccio, C., Wolfsthal, S. D., Englander, R., Ferentz, K., & Martin, C. (2002). Shifting paradigms: from Flexner to competencies. Academic Medicine, 77(5), 361–367. Dawson, S. L., & Kaufman, J. (1998). The imperative for medical simulation. Proceedings of the IEEE, 86(3), 479–483. Dubrowski, A., & Xeroulis, G. (2005). Computer-based Video Instructions for Acquisition of Technical Skills. Journal of Visual Communication in Medicine, 28(4), 150–155. Ericsson, K. A. (2004). Deliberate practice and the acquisition and maintenance of expert performance in medicine and related domains. Academic medicine, 79(10), S70–S81. Frank, J. R., & Danoff, D. (2007). The CanMEDS initiative: implementing an outcomes-based framework of physician competencies. Medical teacher, 29(7), 642–647.
  48. 48. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 48 Frank, J. R., Mungroo, R., Ahmad, Y., Wang, M., De Rossi, S., & Horsley, T. (2010). Toward a definition of competency-based education in medicine: a systematic review of published definitions. Medical teacher, 32(8), 631–637. Frank, J.R., Snell, L., Sherbino, J. Editors. (2015). CanMEDS 2015 Physician Competency Framework. Ottawa: Royal College of Physicians and Surgeons of Canada; 2015. Goldinger, S. D. (1996). Words and voices: episodic traces in spoken word identification and recognition memory. Journal of experimental psychology: Learning, memory, and cognition, 22(5), 1166. Greenhalgh, T. (2001). Computer assisted learning in undergraduate medical education. BMJ, 322(7277), 40–44. Hintzman, D. L., Block, R. A., & Inskeep, N. R. (1972). Memory for mode of input. Journal of Verbal Learning and Verbal Behavior, 11(6), 741–749. Hodges, B. (2010). A Tea-Steeping or i-Doc Model for Medical Education? Academic Medicine, 85, S34–S44. Holmboe, E. S., Sherbino, J., Long, D. M., Swing, S. R., & Frank, J. R. (2010). The role of assessment in competency-based medical education. Medical Teacher, 32(8), 676–682.
  49. 49. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 49 Iobst, W. F., Sherbino, J., Cate, O. T., Richardson, D. L., Dath, D., Swing, S. R., ... & Frank, J. R. (2010). Competency-based medical education in postgraduate medical education. Medical teacher, 32(8), 651–656. Irby, D. M., Cooke, M., & O'Brien, B. C. (2010). Calls for reform of medical education by the Carnegie Foundation for the Advancement of Teaching: 1910 and 2010. Academic Medicine, 85(2), 220–227. Larvin, M. (2009). E-Learning in surgical education and training. ANZ journal of surgery, 79(3), 133–137. Long, D. M. (2000). Competency-based residency training: the next advance in graduate medical education. Academic Medicine, 75(12), 1178–1183. Mayer, R. E. (Ed.). (2005). The Cambridge handbook of multimedia learning. Cambridge University Press. Mayer, R.E. (2008). Learning and Instruction, 2nd edn. Upper Saddle River, NJ: Merrill Prentice Hall Pearson. Mayer, R. E. (2009). Multimedia learning. Cambridge university press. Mayer, R. E. (2010). Applying the science of learning to medical education. Medical education, 44(6), 543–549.
  50. 50. LEARNING METHODS IN ORTHOPAEDIC RESIDENCY EDUCATION 50 Mayer, R.E. (2011). Applying the Science of Learning. Boston, MA: Pearson. Reznick, R. K., & MacRae, H. (2006). Teaching Surgical Skills — Changes in the Wind. New England Journal of Medicine, 355(25), 2664–2669. Ruiz, J. G., Mintzer, M. J., & Leipzig, R. M. (2006). The Impact of E-Learning in Medical Education. Academic Medicine March 2006, 81(3), 207–212. Sonnadara, R. R., Mui, C., McQueen, S., Mironova, P., Nousiainen, M., Safir, O., … Reznick, R. (2014). Reflections on competency-based education and training for surgical residents. Journal of Surgical Education, 71(1), 151–158. Van Eaton, E. G., Tarpley, J. L., Solorzano, C. C., Cho, C. S., Weber, S. M., & Termuhlen, P. M. (2011). Resident education in 2011: three key challenges on the road ahead. Surgery, 149(4), 465–473. Van Merriënboer, J. J., & Sweller, J. (2010). Cognitive load theory in health professional education: design principles and strategies. Medical education,44(1), 85–93. Whang, E. E., Mello, M. M., Ashley, S. W., & Zinner, M. J. (2003). Implementing resident work hour limitations: lessons from the New York State experience. Annals of surgery, 237(4), 449–455.

×