Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.

Jenn McNamara - Using Games for Assessment: Why? How? And Examples

257 views

Published on

Jenn McNamara, VP Serious Games and Strategic Partners, BreakAway Games

This presentation was given at the 2016 Serious Play Conference, hosted by the UNC Kenan-Flagler Business School.

Previously I presented on the potential of games for assessment. Now, organizations are using games for training, researchers are exploring game strategies for assessment, and early adopters have begun leveraging games for assessment. This session will provide lessons learned through presented case study examples relevant to anyone considering game-inspired assessments.

Published in: Education
  • Be the first to comment

Jenn McNamara - Using Games for Assessment: Why? How? And Examples

  1. 1. Using Games for Assessment: Why? How? And Examples Jenn McNamara BreakAway Games Serious Play 2016 Chapel Hill, NC
  2. 2. Before we start we must remember… Games aren’t real – images from “If games were real” cracked.com
  3. 3. Topics ■ A little bit about assessment ■ A little bit about games’ qualities to support assessment ■ Examples demonstrating their intersection GamesAssessment
  4. 4. Assessments Serve To… ■ Measure outcomes or knowledge levels  Attainment of specific levels of proficiency ■ Provide insight into the learning / thought processes ■ Provide immediate feedback  Specific gaps  Potential areas or direction for further instruction ■ Provide a record of behavior ■ Certify proficiency
  5. 5. Types of Assessment ■ Formative Assessment: formal and informal assessment procedures employed by teachers during the learning process in order to modify teaching and learning activities to improve student attainment. Typically qualitative feedback that focuses on the details of content and performance. ■ Summative Assessment: seeks to monitor educational outcomes, often for purposes of external accountability. ■ Standardized Assessment: questions, conditions for administering, scoring procedures, and interpretations are consistent and are administered and scored in a predetermined, standard manner. Wikipedia
  6. 6. Engagement Fundamental components of the gameplay experience: Analysing immersion Ermi & Mayra
  7. 7. Support for Games for Learning ■ Well designed games can enhance motivation and learning  a sense of accomplishment: Bandura; Gist et al.  informative feedback: Bransford et al.; Salas & Cannon Bowers  challenge /competition: Epstein & Harackiewicz; Reeve & Deci ■ Provide successive tasks to help players make progress towards concrete, specific and timely goals  goal setting: Locke et al.; Locke & Latham; Schunk & Ertmer ■ Provide repeated interactive experiences in task based environment  expertise: Glaser; Chi et al.; Bransford et al  anchored instruction: Bransford et al.; CGTV  active learning: Rothman; Chi; Mayer; Vogel et al. Entertainment Software Association 2012; Hussain et al 2009
  8. 8. Games and Formative vs. Summative vs. Standardized Assessment ■ Games play nicely for formative assessment and learning  Fundamental teaching element in support of formative assessment they provide constant, clear feedback that allows for correction and learning ■ Relationship between standardized or summative assessment and games is potentially problematic ■ An ideal standardized or summative assessment purposely avoids teaching in the interest of maintaining validity – consider simulation vs. game or redefine assessment
  9. 9. USC Standard Patient Studio 9 In the US, adequate diagnostic interview & physical exam are the #1 and #2 sources of indemnity payout. PIAA Data Sharing Project DSP2009-2013; 2014 Edition. Desire standardized, repeatable, virtual, emotionally expressive, standard patient encounters for medical diagnostic skills and interview training without expenses of live actors and recorded video evaluation. Funded by the Defense Medical Research Development Program. USC Standard Patient Studio -Simulated patient interactions with performance measurement / formative guidance. -Allows spoken or typed interviews, branching encounters, diagnostic tests and assessment. - Reduced authoring from 6 months to < 1 day with no coding. VALIDATION - >68% case performance gains after 20 minutes of use - 92% recognition & 96% accuracy medical interview natural language - automatic medical performance assessment accurate within 5% RECOGNITION 2015 Gold Serious Play Award 2015 IMSH SGVE Challenge Winner
  10. 10. Standardized Patient Studio Feedback 10
  11. 11. Standardized Patient Studio Feedback 11
  12. 12. TraumaSim / RAMPART 12 University of Pittsburgh physician / researcher studying trauma referral behaviors of ED physicians couldn’t simulate real world performance in the laboratory using traditional written cases. Pitt researchers validated that using TraumaSim: 1. physicians made decisions consistent with actual practice 2. that cognitive load could be manipulated 3. load increased use of heuristics consistent with cognitive theory RECOGNITION 2015 Serious Play Bronze Award Mohan, D., et al. Assessing the Validity of Using Serious Game Technology to Analyze Physician Decision Making (2014) http://dx.doi.org/10.1371/journal.po ne.0105445 Created a simple 2D environment: - present multiple patients - manage common distractions with complimentary goals: - study physician decision making - evaluate construct validity of using a serious game for this purpose
  13. 13. TraumaSim / RAMPART 13
  14. 14. TraumaSim / RAMPART 14
  15. 15. The NBCOT Navigator™ 15 INSTITUTE OF MEDICINE REPORT Concept for new tools derived from IOM Report: Health Professions Education: A Bridge to Quality PRACTICE ANALYSIS STUDY input from 2,500 certificants asking for interactive experiences VALIDATED 6 DOMAINS • Client-Centered Care • Work in Interprofessional Teams • Utilize Informatics • Employ Evidence-Based Practice • Apply Quality Improvement • Professional Responsibility VIRTUAL CONTINUING COMPETENCY ASSESSMENT PLATFORM Created with BreakAway Games to offer accessible, engaging and dynamic tools to provide: • Feedback on current skills • Career enhancement / growth • Evidence-based resources PILOT TESTED AND LAUNCHED Total # of users: 512 & 13,227 Total # of tools : 6,561 & 31,141 (9 week pilot & first 38 weeks live) RECOGNITION Judge’s Choice Most Innovative Product Demo - Association of Test Publishers Conference 2016 Silver Serious Play Award 2016
  16. 16. NBCOT Navigator™: A Portal + 4 Types of Assessments 16 PORTAL AND CERTIFICANT DASHBOARD SELF REFLECTIVE ASSESSMENT AND MINI PRACTICE QUIZES ANIMATED CASE SIMULATIONS GAMES
  17. 17. Contact Information Jenn McNamara Vice President Serious Games and Strategic Partnerships BreakAway Games 610 420-8812 jmcnamara@breakawayltd.com

×