Dmitriy Babichenko, Jonathan Velez, University of Pittsburgh This presentation was given at the 2017 Serious Play Conference, hosted by the George Mason University - Virginia Serious Play Institute. ScopingSim is an interactive alternate-controller-based serious game that uses off-the-shelf open-source components that can be plugged into virtually any computer and is designed to leverage engaging gaming elements to motivate learners to practice both mechanical and diagnostic aspects of scoping procedures. This presentation will address a number of challenges that we had to overcome in order to develop a useful working prototype, including collecting requirements, underestimating costs, dealing with student developers and continuity of support, setting up experiments to identify models of expertise and feedback mechanisms, and making decisions on whether or not to use VR technologies. Physical medical simulators (mannequins) are widely used for training medical students and medical personnel to perform specialized procedures, hone diagnostic techniques, and improve clinical decision-making skills in critical situations. Such mannequin simulators, however, are often extremely expensive, require development of complex teaching scenarios, support of technical staff, and presence of a clinical expert for debriefing and feedback. To address these issues we began to develop ScopingSim - an interactive alternate-controller-based serious game that uses off-the-shelf open-source components, can be plugged into virtually any computer, and leverages engaging gaming elements to motivate learners to practice both mechanical and diagnostic aspects of scoping procedures. This presentation will address a number of challenges that we had to overcome in order to develop a useful working prototype, including collecting requirements, underestimating costs, dealing with student developers and continuity of support, setting up experiments to identify models of expertise and feedback mechanisms, and making decisions on whether or not to use VR technologies.

1. To Scope or Not
To Scope
Challenges of Gamifying Clinical
Procedures Training

2. Who are we?
● Dmitriy Babichenko
● Jonathan Velez
● Kailani Bailey
● William O’Toole
● Ravi Patel

3. While we take full responsibility for any errors and shortcomings of this presentation, we would like to thank the
following people for their constant support of this project, for offering their medical, educational, and gamification
expertise, and for providing indispensable advice on all aspects of design and implementation.
● Lorin Grieve, PharmD, Instructor, Pharmacy and Therapeutics, University of Pittsburgh School of
Pharmacy
● John Lutz, Director of Information Technology and Co-Director of Research, Peter M. Winter Institute for
Simulation, Education, and Research (WISER)
● Phillip Lamberty, MD, Director, Medical Intensive Care Unit UPMC Presbyterian Hospital, Director,
Pulmonary and Critical Care Ultrasonography, Medical Director, Select Specialty Hospital Pittsburgh
● Deborah Farkas, PhD, Director of Educational Development, Peter M. Winter Institute for Simulation,
Education, and Research (WISER)
● Timothy Meehan, Student, University of Pittsburgh School of Information Sciences
● Taylor Winn, Student, iSchool Inclusion Institute (I3)
● Morgan Freeman, Student, iSchool Inclusion Institute (I3)
● Michael Depew, Director, iSchool Inclusion Institute (I3)
Acknowledgements

4. Who are you?

5. Challenges (should you choose to accept them)
1. Structure an in-house game or simulation development project
2. Connect learning objects to system requirements, game
mechanics, and technologies
3. Establish a framework that facilitates learner evaluation

6. ScopingSim

7. What is bronchoscopy?
(Medline - https://medlineplus.gov/ency/images/ency/fullsize/23232.jpg)

8. Motivation #1 - Cost
Medical simulators & task trainers are EXPENSIVE (and
creepy)
1. High cost of purchase
2. High cost of maintenance
3. Low simulator to student
ratio

9. Motivation #2 - Feedback
● High-end simulators & task trainers provide
limited feedback based on anatomy &
physiology
● Difficult to determine whether a learner
performed well because of skill / knowledge
or just luck
● No user model / learner model

10. Motivation #3 - Debriefing
● Students cannot practice on their own
without supervision of domain expert
● Domain experts’ time is expensive
● Low domain expert to student ratio
● Without debriefing complex procedure
simulations are virtually pointless Image source:
https://cphp.org/critical-incident-debriefing-services/

11. Motivation #4 - Generalizability
● Common procedure(s) -
bronchoscopy, endoscopy,
colonoscopy
● Good generalizability
● Number of interested
stakeholders
Laerdal SimMan (http://www.laerdal.com/us/doc/86/SimMan)

12. Let’s Roleplay
IT Expert Medical Expert Education/Game Expert

13. Mission Impossible I:
Define Requirements

14. What did the stakeholders want?
● Something that doesn’t require an expensive mannikin
& easily distributable
● Provides real-time feedback & just-in-time learning
● Debriefs learner without the help of a domain expert

15. ● Something that doesn’t require an expensive mannikin
& easily distributable
Deliberate Practice
● Provides real-time feedback & just-in-time learning -
Reinforcement Learning
● Debriefs learner without the help of a domain expert -
Reflective Learning
What did the stakeholders want? (Learning Goals)

16. Mission Impossible II Organize Your Ideas

17. Affinity Diagram
● Organizes a large number of ideas into their
natural relationships
● Taps a team’s creativity and intuition
● When to use
○ You are confronted with many facts or
ideas in apparent chaos
○ Issues seem too large and complex to
grasp
○ Group consensus is necessary
http://asq.org/learn-about-quality/idea-creation-tools/overview/affinity.html

18. Affinity Diagram Procedure
● Look for ideas that seem to be related in
some way. Place them side by side.
● Form notes into “ idea” clusters
● Discuss any patterns, especially reasons
for moving controversial notes, and
potentially split or merge clusters
● Repeat
http://asq.org/learn-about-quality/idea-creation-tools/overview/affinity.html

19. Now it’s your turn
Affinity Diagram Procedure
http://asq.org/learn-about-quality/idea-creation-tools/overview/affinity.html

20. Mission Impossible III:
Priorities.. Priorities..

21. Make the Hard Decisions
● Recognize that each idea in the pool may
be competing ideas
○ How will committing to one idea influence the
feasibility of other ideas in its cluster?
○ How will committing to one idea influence the
feasibility of ideas in other clusters?
● Analysis & Design
○ Select and prioritize requirements
○ Requirements should support learning objectives
○ Be agile..! Requirements and feasibility can
change at any moment

22. ScopingSim with Scope Video Version 1
Setareh Sarachi, Pavitraa, Faris Obaid Alotibi, Abhishek Mukherjee, Dimple Varma

23. ScopingSim with Scope Video Version 2
Siying Zhang, Yue Su, Hanwei Cheng, Zihan Xie

27. Failed Controllers

30. Ambu
Scope
Flex
Sensor
Accelerometer /
Gyroscope
Arduino
Uno

32. Mission Impossible Ghost Protocol:
User Modeling

33. Building the User Model
We have no idea what that might look like at this point...

34. Mission Impossible - Let’s talk about models
● How do we define performance?
● How do we classify “expertise”?
● When and how do we give feedback?

35. How do we define performance?
● User model constrained by
system design
● System design constrained by
user model
Raw measures:
● x, y, z acceleration
● Time
● Spatial position in virtual environment
● Completed objectives
● Number of failures
● ...

36. How do we classify “expertise” ?
● Ask the experts!
○ How is expertise determined in the
domain?
○ Pedagogies for knowledge/skill acquisition
● Qualify expertise from quantified
performance
○ Heuristics vs Analytics
Choose your pedagogy:
● Fitt and Posner’s Theory of Motor
Acquisition (1967)
○ Cognitive → Integrative → Autonomous
Parameterize your raw data:
● Changes in direction (avg and std dev)
● Time in motion (avg and std dev)
● Time at rest (avg and std dev)
● Ratio of time at rest to time in motion
● Velocity of motion (avg and std dev)
● Accomplishment of game objectives
● …

37. When and how do we give feedback?
● Feedback elements constrained by system design
○ Environmental cues (e.g., visual, audio, haptic)
○ Implementation examples: mini-map, flash or buzz on
environment collision events, servomotor resistance
● Scaffold learning based on the model of expertise
○ Suggestion mechanisms (helpful or distracting?)
○ Implementation examples: next-step model
● Contextualize deviations in performance to
encourage reflective learning
○ Facilitate debriefing; ask “how” and “why”
○ Detect where learners’ performance can improve and
provide enough info for open-ended self-evaluation

38. Mission Impossible:
Where are we today?

39. Dmitriy Babichenko - dmb72@pitt.edu
To Scope or Not
To Scope
Thank you!
Questions/Answers?

40. To Scope or Not
To Scope
Thank you!
Questions/Answers?
The Floor is Failure
Dmitriy Babichenko - dmb72@pitt.edu

42. To Scope or Not
To Scope
Thank you!
Questions/Answers?
Dmitriy Babichenko - dmb72@pitt.edu