Assessment of the use of the Virtual Orthodontic Case Studies in an Undergraduate Teaching Program Dr Jean –Marc Retrouvey VOCS: An Interactive Teaching Tool in Orthodontics
What is CAL?Computer Assisted Learning Any type of pedagogical material that is delivered through a computer software. Can be very simple (PPTs) or very complex (Flight simulators) Most effective is the interactive model where students participate to the learning process
Validity of CAL Rosenberg study demonstrated that CAL usage was at least as effective in orthodontic undergraduate education (diagnosis) as standard classroom lectures One of the few orthodontically oriented pedagogical tools build with instructional designers Rosenberg, H., et al., Evaluation of computer-aided learning in orthodontics. American Journal of Orthodontics and Dentofacial Orthopedics, 2010. 138(4): p. 410-419.
Interactive Seminars are coming to Orthodontics…. Interactive Distance Learning in Orthodontic Residency Programs: Problems and Potential Solutions Katherine P. Klein, D.M.D., M.S.; Wallace H. Hannum, Ph.D.; Henry W. Fields, D.D.S., M.S., M.S.D.; William R. Proffit, D.D.S., Ph.D. (2012) Sharing resources through distance education has been proposed as one way to deal with a lack of full-time faculty members and maintain high-quality content in orthodontic residency programs. To keep distance education for orthodontic residents cost-effective while retaining interaction, a blended approach was developed that combines observation of web-based seminars with live post-seminar discussions.
Rationale for VOCS Database of carefully selected patients used for pedagogical reasons (searchable) Study and present cases anywhere ( home, school, lectures) Avoid model storage and manipulations (paperless records) Fast loading time on any computer (No 3D ) Capability to test students electronically
Objectives of this study First objective: To evaluate students’ perceptions concerning a series of aspects of the VOCS learning tool including quality and quantity of information provided and simplicity of use.
Objectives of the studySecond Objective: To determine if there was a difference in the results of data collected from “conventional “ examination and measurements on study casts and the VOCS software. Question : Is VOCS a reliable tool to study orthodontic cases?
Study Design: part 1 21 students Each student studied one case using the VOCS online system questionnaire was filled by same student to assess his/her perception about the online method’s quality and likability. evaluating ‘general material’, ‘patient history’, ‘facial images’, ‘intra-oral images’, ‘dental cast’, ‘panoramic’, ‘cephalometrics’, ‘quiz’, and ‘overall feedback’ were presented. Each item had a 4 category scale ranging from ‘strongly disagree’ to ‘strongly agree’.
– Virtual Orthodontic Case Studies (VOCS) Date: ___________________ Instructions: Evaluations provide us with important feedback regarding our work. Please take a couple of minutes to answer the following questions candidly. We thank you for your time and input which will assist in preparing for future work developed for the VOCS. If you require additional room for your comments, please feel free to add an extra sheet of paper. Strongly Disagree Agree Strongly Disagree AgreeInteractive Materials/Activities1. Module objectives were clear. 2. Module materials were well written. 3. Module materials were well organized. 4. Materials/activities were helpful in learning new skills 5. Materials/activities were relevant and interesting 6. The module provided new information and/or sufficient depth to challenge me.Program Outcomes7. The outlined module objectives were met. 8. I learned something I can immediately apply. 9. My time was well invested in this module. 10. I would recommend this module to other individuals.
Evaluation – VOCS Strongly Disagree Agree Strongly Disagree AgreeOverall Usability of Website11. The website was easy to navigate. 12. It was simple to find information. 13. The radiographs were clear. 14. The photographs were clear. 15. The website models were sufficient for diagnosis. Website as a Teaching Method16. The website is a good teaching method. 17. The cases on the website are accessible anywhere/time. 18. The cases on the website can be reviewed easily. 19. The case material is easy to refer to during the quiz. Website Compared to other Teaching Methods20. The website is a better teaching method than the student presentations.21. The website is a better teaching method than the hands-on teacher-to-class technique.
Evaluation – VOCSYou Tell Us22. In your opinion, what were the advantages of the website over other teaching methods?23. In your opinion, what were the disadvantages of the website over other teaching methods?24. Additional comments (please feel free to add further remarks on another sheet).
Diagnostic Capability The website models were sufficient for diagnosis All students either100 90 80 agreed that this 2 D 70 method was sufficient to 60 diagnose simple to 50 moderately complex 40 malocclusions. 30 20 10 0 Strongly Agree Agree Disagree Strongly Disagree
Overall Feedback My time was well invested in this module I would recommend this module to other individuals100 100 90 90 80 80 70 70 60 60 50 50 40 40 30 30 20 20 10 10 0 0 Strongly Agree Agree Disagree Strongly Strongly Agree Agree Disagree Strongly Disagree Disagree
Comments for part 1 95% of the students responded ‘agree’ or ‘strongly agree’ to the questionnaire’s items (36 and 59%, respectively). Twenty of the 46 items had 100% of the students responding either ‘agree’ or ‘strongly agree’. The only negatively phrased item (‘some questions are redundant’) had 65% of responses at ‘agree’ or ‘strongly agree’ and the remaining 35% at ‘disagree’ or ‘strongly disagree’.(quality of the quizz)
Biggest problem: The item concerning ‘intra-oral images being clear’ had 2 people (10%) responding ‘strongly disagree’ . Midline determination is challenging
Reliability of the VOCS measurements in relationto conventional measurements Duplicated the methodology used in dental schools for an initial orthodontic case study (undergraduate level) Assess if the students measured the same things and obtain the same diagnosis ( correctly or incorrectly).
Methods 21 students participated 13 completed the exercise correctly Analyzed conventional records and measured study casts (test reliability and accuracy of the virtual tools) Teachers (3) corrections or evaluations were deemed be worth 100% of the score (total agreement on data analysis)
Table 1: Individual students percentage score in theorthodontic analysis evaluation using either models or theVOCS. Students Percentage score Model VOCS 1 50,0 59,1 2 68,2 50,0 3 72,7 72,7 4 45,5 68,2 5 45,5 50,0 6 81,8 31,8 7 81,8 81,8 8 45,5 63,6 9 77,3 68,2 10 63,6 68,2 11 86,4 81,8 12 77,3 72,7 13 59,1 54,5
Table 2: Test statistics on the differences in average scoresbetween groups using model or VOCS to complete theorthodontics analysis evaluation. n Mean score Standard Deviation p Model 13 65,7 15,2 0,63 VOCS 13 63,2 14,1
Results of the analyses from students The results show that their was no significant difference between the analyses of the students if they used either the standard study casts, boley gages and rulers or if they used the virtual dental casts and tools (P=0.63) The students who got the right or wrong diagnosis using either methods (except for one student 80 -30)
Discussion and Conclusions Case studies can be made available 24/7 on any LMS (learning management system) Quizzes and cases studies can be packaged by the instructor or the administrator Repository of searchable cases Easy to load on any computer system Reliable and test at the same level any case analysis
Self paced teaching tool used now at McGill Canadian National Dental Board has used 4 of the VOCS cases for its examination process Very positive response to the Virtual environment. All students from any group can be tested on the same case
Better teaching tool?More than 50% still preferred the one on one hands-on teaching technique(gold standard). Perfectly normal as feed back and assistance are not readilyavailable with the on line method.It was preferred to the students presentations .100 The website is a better teaching method 100 90 The website is a better teaching than the student presentations 90 method than the hands-on 80 80 teacher-to-class technique 70 70 60 60 50 50 40 40 30 30 20 20 10 10 0 0 Strongly Agree Disagree Strongly Strongly Agree Disagree Strongly Agree Disagree Agree Disagree