The document discusses how games can be used for assessment by providing feedback, measuring outcomes, and allowing for formative learning. It provides examples of games like the Standardized Patient Studio that assess medical interview skills and TraumaSim that study trauma referral behaviors. The document also describes the NBCOT Navigator, a platform featuring games and simulations that provides continuing competency assessment for occupational therapists.

1. Using Games for Assessment: Why? How? And Examples
Jenn McNamara
BreakAway Games
Serious Play 2016 Chapel Hill, NC

2. Before we start we must remember…
Games aren’t real
– images from “If games were real” cracked.com

3. Topics
■ A little bit about assessment
■ A little bit about games’ qualities to support assessment
■ Examples demonstrating their intersection
GamesAssessment

4. Assessments Serve To…
■ Measure outcomes or knowledge levels
 Attainment of specific levels of proficiency
■ Provide insight into the learning / thought processes
■ Provide immediate feedback
 Specific gaps
 Potential areas or direction for further instruction
■ Provide a record of behavior
■ Certify proficiency

5. Types of Assessment
■ Formative Assessment: formal and informal assessment procedures
employed by teachers during the learning process in order to modify
teaching and learning activities to improve student attainment.
Typically qualitative feedback that focuses on the details of content
and performance.
■ Summative Assessment: seeks to monitor educational outcomes, often
for purposes of external accountability.
■ Standardized Assessment: questions, conditions for administering,
scoring procedures, and interpretations are consistent and are
administered and scored in a predetermined, standard manner.
Wikipedia

6. Engagement
Fundamental components of the gameplay experience: Analysing immersion Ermi & Mayra

7. Support for Games for Learning
■ Well designed games can enhance motivation and learning
 a sense of accomplishment: Bandura; Gist et al.
 informative feedback: Bransford et al.; Salas & Cannon Bowers
 challenge /competition: Epstein & Harackiewicz; Reeve & Deci
■ Provide successive tasks to help players make progress towards concrete, specific and
timely goals
 goal setting: Locke et al.; Locke & Latham; Schunk & Ertmer
■ Provide repeated interactive experiences in task based environment
 expertise: Glaser; Chi et al.; Bransford et al
 anchored instruction: Bransford et al.; CGTV
 active learning: Rothman; Chi; Mayer; Vogel et al.
Entertainment Software Association 2012; Hussain et al 2009

8. Games and Formative vs. Summative vs. Standardized
Assessment
■ Games play nicely for formative assessment and learning
 Fundamental teaching element in support of formative assessment they
provide constant, clear feedback that allows for correction and learning
■ Relationship between standardized or summative assessment and
games is potentially problematic
■ An ideal standardized or summative assessment purposely avoids
teaching in the interest of maintaining validity – consider simulation
vs. game or redefine assessment

9. USC Standard Patient Studio
9
In the US, adequate
diagnostic interview &
physical exam are the
#1 and #2 sources of
indemnity payout.
PIAA Data Sharing Project
DSP2009-2013; 2014
Edition.
Desire standardized, repeatable,
virtual, emotionally expressive,
standard patient encounters for
medical diagnostic skills and
interview training without expenses
of live actors and recorded video
evaluation.
Funded by the Defense Medical
Research Development Program.
USC Standard Patient Studio
-Simulated patient interactions with
performance measurement /
formative guidance.
-Allows spoken or typed
interviews, branching encounters,
diagnostic tests and assessment.
- Reduced authoring from 6
months to < 1 day with no coding.
VALIDATION
- >68% case performance gains after
20 minutes of use
- 92% recognition & 96% accuracy
medical interview natural language
- automatic medical performance
assessment accurate within 5%
RECOGNITION
2015 Gold Serious Play Award
2015 IMSH SGVE Challenge Winner

10. Standardized Patient Studio Feedback
10

11. Standardized Patient Studio Feedback
11

12. TraumaSim / RAMPART
12
University of Pittsburgh
physician / researcher
studying trauma referral
behaviors of ED
physicians couldn’t
simulate real world
performance in the
laboratory using traditional
written cases.
Pitt researchers validated that
using TraumaSim:
1. physicians made decisions
consistent with actual practice
2. that cognitive load could be
manipulated
3. load increased use of
heuristics consistent with
cognitive theory
RECOGNITION
2015 Serious Play Bronze Award
Mohan, D., et al. Assessing the
Validity of Using Serious Game
Technology to Analyze Physician
Decision Making (2014)
http://dx.doi.org/10.1371/journal.po
ne.0105445
Created a simple 2D environment:
- present multiple patients
- manage common distractions
with complimentary goals:
- study physician decision making
- evaluate construct validity of
using a serious game for this
purpose

13. TraumaSim / RAMPART
13

14. TraumaSim / RAMPART
14

15. The NBCOT Navigator™
15
INSTITUTE OF
MEDICINE REPORT
Concept for new tools
derived from IOM Report:
Health Professions
Education: A Bridge to
Quality
PRACTICE ANALYSIS STUDY
input from 2,500 certificants asking
for interactive experiences
VALIDATED 6 DOMAINS
• Client-Centered Care
• Work in Interprofessional Teams
• Utilize Informatics
• Employ Evidence-Based Practice
• Apply Quality Improvement
• Professional Responsibility
VIRTUAL CONTINUING
COMPETENCY ASSESSMENT
PLATFORM
Created with BreakAway Games to
offer accessible, engaging and
dynamic tools to provide:
• Feedback on current skills
• Career enhancement / growth
• Evidence-based resources
PILOT TESTED AND LAUNCHED
Total # of users: 512 & 13,227
Total # of tools : 6,561 & 31,141
(9 week pilot & first 38 weeks live)
RECOGNITION
Judge’s Choice Most Innovative
Product Demo - Association of
Test Publishers Conference 2016
Silver Serious Play Award 2016

16. NBCOT Navigator™:
A Portal + 4 Types of Assessments
16
PORTAL AND
CERTIFICANT
DASHBOARD
SELF REFLECTIVE
ASSESSMENT
AND
MINI PRACTICE
QUIZES
ANIMATED CASE
SIMULATIONS
GAMES

17. Contact Information
Jenn McNamara
Vice President Serious Games and Strategic Partnerships
BreakAway Games
610 420-8812
jmcnamara@breakawayltd.com