- Medical students receive little training in otoscopy, averaging only 57 hours. General practitioners also demonstrate limited skills. - A new web-based otoscopy simulator called OtoTrain was developed and its face validity, content validity, and usefulness for medical education were evaluated through an expert survey. - The survey found that OtoTrain demonstrated good face validity and content validity for teaching otoscopy skills. Experts also indicated it was generally superior to traditional teaching methods like textbooks and lectures.

1. • 50% of diagnoses for ear infections are incorrect.2
• Only 5% of senior medical students feel competent in their
ability to perform otoscopy.3
• Medical students average 57 hours of total otoscopic training.4
• General Practitioners demonstrate comparable skills to most
medical students.5
Introduction
Methods
Results
Conclusion
References
Jordan Lewis1
, Brandon Wickens1,2
, Sumit K. Agrawal1,2
, Hanif M. Ladak1
1
Western University, London, ON, Canada
2
London Health Sciences Centre, London, ON, Canada
Face and Content Validity of a Novel Web-Based Otoscopy
Simulator for Medical Education
Presentation by: Jordan Lewis, BMSc 2015
Research Supervisor: Dr. Hanif Ladak, PhD, PEng
Domain
Mean Likert Scale
Ranking (SD)
Face validity
Realistic representation of normal/abnormal external auditory canal anatomy 2.7 (1.5)
Realistic representation of normal/abnormal eardrum anatomy 2.3 (1.3)
Content validity
Teaches importance of assessing external auditory canal 2.4 (0.8)
Teaches importance of assessing entire eardrum 2.2 (1.3)
Provides adequate breadth of pathologies 2.1 (1.4)
Is a useful introduction to otoscopy 1.5 (0.8)
Applicability to medical training
Useful for training of medical students/audiology students 1.5 (0.8)
Useful for training of non-Otolaryngology residents (Family Medicine,
Paediatrics, Emergency Medicine, etc.)
1.7 (1.0)
Useful for training of Otolaryngology residents 2.5 (1.3)
Recommend for use in continuing medical education 1.8 (1.0)
User-friendly 1.7 (1.1)
Table 1: Questionnaire results (n=11). Likert scale rankings are from 1 (strongly
agree) to 7 (strongly disagree), with 4 being neutral.
Statement
Online
Simulator
Superior
Equivalent
Atlas or
Lecture
Superior
Unsure
Learning eardrum anatomy 45.4% 45.5% 0.0% 9.1%
Learning external auditory canal anatomy 63.6% 36.4% 0.0% 0.0%
Learning otoscopy technique 63.6% 27.3% 0.0% 9.1%
Availability of training technique 54.5% 9.1% 27.3% 9.1%
Otoscopy
Rationale
• Visual examination of the
eardrum using an otoscope
• Diagnosis of ear pathologies
• Challenges: small speculum,
poor lighting, narrow/curved
ear canals, earwax, hair, young
patients (children)
Current Training Methods
2) Atlases 3) Physical Models
• Commonly used
• Provide clear, wide angle images
of eardrum – not partial views
typical in otoscopy
• Not widely adopted
• Expensive (limited #)
• Require lab space (scheduling
issues)
• Single ear canal (no diversity)
• No automatic feedback
1) Clinical Skills Course
• 2nd year medical students
• Independent study (atlases)
• Very little clinical skills training - tend to practice on each other
OtoTrain:
Low cost: No special equipment required
Accessibility: Can reach remote users at any location
Simultaneous training: Numerous users at any time
Extensibility: Can add unlimited new cases
Quantitative feedback: Training & examination mechanisms
A web-based simulation software for otoscopy training
developed at Western University
• To conduct a survey of experts in the field of otology and
audiology education to:
1) Assess the face and content validity of the simulator
(OtoTrain™) to evaluate its use as a training platform
2) Compare OtoTrain™ with standard medical education
techniques
Objectives Definitions
Face
Validity
The degree to which a
simulator appears to be a
realistic
representation
of actual ears and associated
pathologies
Content
Validity
Evaluation of the
subject matter
test items
in the context of the
curriculum
Figure 4: Otoscopy simulator tutorial module. Figure 5: Otoscopy module demonstrating acute otitis media in study mode .
Figure 6: Performance feedback following module completion. Figure 7: Pathology descriptions can be accessed following each question in
study mode.
Table 2: Comparison of training methods.
• Experts (n=11) from around the world were recruited to participate in the study and complete a survey. A web-based tutorial and link to the online
simulator were distributed, along with a link to a web-based questionnaire:
• The questionnaire consisted of 3 parts:
Ø Assessment of face validity, content validity, and applicability to medical training curricula. Questions were answered using a 7-point Likert
scale ranging from 1 (strongly agree) to 7 (strongly disagree).
Ø Comparison to other training methods. Experts ranked if OtoTrain™ is superior, equivalent, or inferior to the traditional atlas- or lecture-based
method of otoscopy education.
Ø Qualitative feedback on areas for improvement was obtained through written comments.
Face validity, content validity, and applicability to medical training
• All mean scores were ≤ 3 (falling between “Agree” and “Mostly Agree”)
suggesting good face and content validity (Table 1).
Comparison of training methods
• OtoTrain™ was deemed superior to the atlas/lecture-based method,
with the exception of learning eardrum anatomy (equal percentage of
experts considered superior or equivalent) (Table 2).
Qualitative feedback
• Suggested areas for improvement (currently being implemented)
include: (1) Need for haptic feedback; (2) Greater emphasis on
importance of pneumatic otoscopy, and; (3) Need for more (rare)
eardrum pathologies.
OtoTrain™ is a novel, web-based otoscopy simulator that can be
easily distributed and used by students on a variety of platforms.
Face validity, content validity, and applicability to the medical
training curriculum were positive. A skills transference study is
planned following further modifications to the simulator.
Figure 2: Example of otoscopy
atlas used to study ear
pathologies.6
Figure 3: Example of a physical
model used for otoscopy training.7
Figure 1: Otoscopic examination
using an otoscope.1
1. A.D.A.M. Health Reference (2010) Otoscope examination. http://upmc.adam.com/graphics/images/en/8771.jpg.
Accessed March 20, 2014.
2. Pichichero ME. Diagnostic accuracy, tympanocentesis training performance, and antibiotic selection by pediatric
residents in management of otitis media. Pediatrics. 2002;110:1064-70.
3. Jones WS, Johnson CH, Logacre JL. How well are we teaching otoscopy: medical students’ perspectives. Pediatr.
Res. 2003;53(4):544A.
4. Lund VJ. Otolaryngology in the curriculum – 10 years on. J R Soc Med. 199;83:377379.
5. Fisher EW, Pfleiderer AG. Assessment of otoscopic skills of general practitioners and medical students: is there
room for improvement? Br J Gen Pract. 1992;42: 65-67.
6. Sanna, M., Alessandra Russo, and Giuseppe de Donato. Color Atlas of Otoscopy: From Diagnosis to Surgery.
Stuttgart: Thieme, 1999.
7. Health Edco (2015) Ear Examination Simulator. http://www.healthedco.com/index.php/ear-examination-
simulator.html. Accessed March 16, 2015.