• Share
  • Email
  • Embed
  • Like
  • Save
  • Private Content
A Research Approach to Develop Measurable Competencies and Skills: Videogames or Real-World Training
 

A Research Approach to Develop Measurable Competencies and Skills: Videogames or Real-World Training

on

  • 1,140 views

 

Statistics

Views

Total Views
1,140
Views on SlideShare
998
Embed Views
142

Actions

Likes
1
Downloads
17
Comments
0

6 Embeds 142

http://lessonsfromgames.blogspot.com 133
http://lessonsfromgames.blogspot.com.br 4
http://www.lessonsfromgames.blogspot.com 2
http://lessonsfromgames.blogspot.fr 1
http://lessonsfromgames.blogspot.de 1
http://lessonsfromgames.blogspot.co.uk 1

Accessibility

Upload Details

Uploaded via as Microsoft PowerPoint

Usage Rights

© All Rights Reserved

Report content

Flagged as inappropriate Flag as inappropriate
Flag as inappropriate

Select your reason for flagging this presentation as inappropriate.

Cancel
  • Full Name Full Name Comment goes here.
    Are you sure you want to
    Your message goes here
    Processing…
Post Comment
Edit your comment

    A Research Approach to Develop Measurable Competencies and Skills: Videogames or Real-World Training A Research Approach to Develop Measurable Competencies and Skills: Videogames or Real-World Training Presentation Transcript

    • A Research Approach to Develop Measurable Competencies and Skills: Videogames or Real-World Training K. Anders Ericsson Department of Psychology Florida State University by
    • One of the crucial issues of training interventions is to be able to specify their goals so one can measure and evaluate objective outcomes. I believe that videogames and simulators can provide us with the tools to specify the goal and targets of education for everyday and professional life.
    • Getting away from paper-and-pencil or multiple-choice tests of the type of Scholastic Aptitude Tests (SAT) Define outcomes with video games and simulations of everyday situations and tasks
    • Traditional Instruction On-the-job experience Target Performance “ Retain skills and knowledge” “ learning transfer” “ generalization”
    • Traditional Instruction On-the-job experience Target Performance “ Retain skills and knowledge” “ learning transfer” “ generalization” Alternatives , such as Simulations Videogames
    • Outline of My Presentation
      • If the desired competency concerned a domain ,
      • like Chess, Music, or Sports, how
      • would expert performance be attained
      II. Objective measurement of performance III. Mechanisms, skills, and their acquisition through deliberate practice IV. Acquisition of control and complex mental representations— Beyond automaticity V. Training and the fidelity assumption
    • I. If the desired competency concerned a domain, like Chess, How is Expert Performance Attained How did chess players attain their performance? Benjamin Franklin learned to play chess from an acquaintance in 1733 He was one of the first chess players in America. Although he was an avid player he reached a modest level of chess skill
    • Our Knowledge about Learning and Skilled Performance is Limited How good is a chess player or an individual in a domain of expertise? -- Self ratings
    • Professors Drivers Doctors (94% above average) (80% above average) (Davies et al., 2006) Self-Assessments on Relative Performance How would you rate your own performance? bottom 10% below average average above average top 10%
    • Professors Drivers Doctors (94% above average) (80% above average) (Davies et al., 2006) Self-Assessments on Relative Performance How would you rate your own performance? bottom 10% below average average above average top 10%
    • How could a chess player in the chess club be able to keep improving? Playing games Watching better players play Reading books Improving One’s Chess Performance or Performance in Tennis, Golf or SCRABBLE
    • Rated Relevance for Improvement from Doing X on a scale from 1 to 7 (Charness, Krampe, & Mayr, 1996) Active participation No in chess tournaments 6.1 correlation Playing chess games Negative outside of chess 3.6 correlation tournaments Serious analysis of Positive positions alone 5.9 correlation
    • Rated Relevance for Improvement from Doing X on a scale from 1 to 7 (Charness, Krampe, & Mayr, 1996) Active participation No in chess tournaments 6.1 correlation Playing chess games Negative outside of chess 3.6 correlation tournaments Serious analysis of Positive positions alone 5.9 correlation
    • Effects and Dose-Response Relations to Practice Activities Professional teachers and coaches * Monitor students’ development * design training tasks for individual students Expert Performance
    • Rated Relevance for Improvement from Doing X on a scale from 1 to 7 (Charness, Krampe, & Mayr, 1996) Active participation No in chess tournaments 6.1 correlation Playing chess games Negative outside of chess 3.6 correlation tournaments Serious analysis of Positive positions alone 5.9 correlation
    • Simulated Play Against World Class Players Study published games by chess masters Make predictions for each next move Check if your prediction was correct, if not, study the chess position until you understand why the correct move was played Black on move Position B
    •  
    • On-the-job supervised experience Classroom Knowledge Acquisition Measurable Outcomes Part II. Objective Measurement of Performance Independent ability to increase patients’ health Kirkpatrick’s Four Level Model of Training Evaluation
    • The most respected experts (“expensive” stockbrokers) are not markedly better in picking stocks on New York Stock Exchange than a random process, such as a monkey throwing darts (c.f. Wall Street Journal study)
    • Superior Performance Chess winning chess games 50 games ( 100-250 hours ) for a rating Psychotherapy superior ability to increase mental patients’ health (post-pre) 50-100 patients (400-800 hours) Teaching superior ability to increase students’ performance (post - pre) 500-1000 contact hours
    • Minimal Effects of Traditional Indicators of Expertise Long Education Extended experience as a professional Teaching Psychotherapy
    • Identify challenging and difficult situations, where experts are supposed to excel. Adriaan de Groot (1914–2006)
    • Recreate the situation and task in laboratory Actions and thoughts of novices and experts can be directly compared
    • Select the best chess move for this position Type as much of the presented text as possible within one minute Play the same piece of music twice in same manner Chess Typing Music Domain Presented Information Task Selection Speed Control
    • Reproducible Soccer Performance
    • From cover story in Time Magazine (March 10, 2008) on The Science of Experience Response to Critical Unexpected Problems
    • Salchow Part III. Mechanisms, Skills, and their Acquisition through Deliberate Practice
    • The problem with learning during work and play Andy Murray (Scotland) in 2010
    • The problem with learning during work and play
    • Immediacy of Informative Feedback Sport training  performance (1-2 years) Investing  value (10-20 years) Medicine  patient outcomes (12-36 months)
    • Increase in Complexity and Control as a Function of Years of Piano Training Years of piano training
    • “ individualized training activities especially designed by a coach or teacher to improve specific aspects of an individual's performance through repetition and successive refinement. To receive maximal benefit from feedback, individuals have to monitor their training with full concentration, which is effortful and limits the duration of daily training”. (Ericson & Lehmann, 1996, pp. 278-279) Deliberate Practice
    • Design and Sequencing of Training Activities Professional teachers and coaches * Monitor students’ development * design training tasks for individual students Expert Performance
    • Accumulated Amount of Practice Alone During the Development of Amateurs and Expert Musicians
    • The Body’s Resilience to Change The Protection of Equilibrium
    • Increase of Fitness
    • Deliberate Practice Optimal
    • How many consecutive push-ups can someone perform without pause? (H. H. Hart, 1974) Range of modifiability with extended Deliberate Practice
    • Charles Linster Non-stop push-up records http://www.recordholders.org/en/list/ulysses.html 6,006 Charles Linster (USA) 05-Oct-1965 … 10,507 Minoru Yoshida (JAP) Oct-1980 Fu Bingli, a kung fu master from China 12 press ups with right finger http://www.telegraph.co.uk/news/newstopics/howaboutthat/6385592/Man-proves-he-has-worlds-strongest-fingers.html# Accessed on October 23 rd , 2009
    • Part IV. Acquisition of Control and Complex Mental Representations—Beyond Automaticity
    • The Development of Increased Control Actively Avoiding Automating Control of Critical Aspects of Performance
    • Reproducibly Superior Performance is not automatic but is associated with informative verbalizations of thoughts
    • Black on move Position B * What aspect was overlooked * When could this aspect have been discovered * How to avoid similar mistakes in the future * Develop new skills by deliberate practice Learning from Making an Incorrect Move
    • Alekhine beat most of the 30 skilled players while playing them simultaneously under blindfold conditions Ref P16 A http://images.google.com/imgres?imgurl=http://www.chessbase.com/news/2007/bilbao04.jpg&imgrefurl=http://www.chessbase.com/newsdetail.asp%3Fnewsid%3D4189&h=318&w=480&sz=36&hl = en&start=1&usg=__JwOiqXRmMEJcHCssYGF1N_5BqU4=&tbnid=DC9VzEbhAUofSM:&tbnh=85&tbnw=129&prev=/images%3Fq%3Dblindfold%2Bchess%26gbv%3D2%26hl%3Den%26sa%3DG Accessed on August 26, 2008 Blindfold Chess Exceptional abilities
    • The Importance of Acquired Mental Representations
    • Expert Performers are Better Able to Represent and Analyze Situations Slowing Down (Moulton et al.,2010) (Ward & Williams, 2003) (Tuffiash, Roring, & Ericsson, 2007) (Kesting, et al., 2010)
    • V. Training and the Fidelity Assumption It is frequently assumed that the best learning of skills occurs in the natural environment . Hence, simulation should therefore reproduce the experience in the natural environment What do we know about natural learning? How optimal is that type of natural learning?
    • Perceptual Performance on Cardiac Auscultation Instruction and Experience 0-9 years 10-20 years Over 20 years General Practitioners Student Based on Butterworth & Reppert (1960) Reviews (Choudhry, Fletcher, & Soumerai, 2005; Ericsson, 2004; Ericsson, Whyte, & Ward, 2007) Performance No Correlations between Amount of Experience and Performance after the First Year
    • Benefits of Provided Experience on Performance Use of Cockpits with Visual and Kinesthetic Simulation Modest transfer of performance— savings of training in real airplanes (except new procedures) (Allerton, 2000; Roessingh, 2005; Rantanen & Talleur, 2005) Focus on Fidelity and Safety A airline captain needs 3,500 hours of flying plus 60 hours of simulation (Parker, Johns, & Hellige, 2007)
    • Identifying the Crucial Elements of Effective Simulator Training - Deliberate Practice A review of 109 studies of high-fidelity medical simulation showed that the characteristics of Deliberate Practice accounted for improvements (Issenberg et al., 2005) A review of 31 of those studies that examined repetitive practice, and “ found a strong association ( η2 = .46 ) between hours of practice on high-fidelity medical simulators and standardized learning outcomes .” (Issenberg et al., 2006, p. 792)
    • The Use of Simulators for Deliberate Practice with Performers at Different Levels of Expertise Allows exposure to challenging situations without danger Allows exposure to difficult situations under conditions optimal to learning and performance - Individuals being focused and ready - Immediate feedback (and opportunity for repetition) - Presentation of related cases to facilitate discrimination Allows the presentation of rare emergency situations
    • Deliberate Practice with Simulators Authentic test conditions with actors (Kneebone et al., 2005) Reinstating actual scenarios in anesthesiology (Liu et al, 2009)
    • Measurement and Training Capturing the essence of expert performance Measuring it with representative tasks Measuring Identification current level of  of optimal performance training tasks
    • Traditional Instruction On-the-job experience Target Performance “ Retain skills and knowledge” “ learning transfer” “ generalization” Alternatives , such as Simulations Videogames
    • Recommendations Identify skills and analyze domains of activity where performance can be measured with objective methods that capture on-the-job performance with large individual differences in attained performance where increases in performance motivate major investment Develop libraries of representative situations with appropriate feed-back about correct/appropriate actions with scaled difficulty to maximize optimal training effects Develop cognitive structures to support Deliberate Practice assessment of representations for thinking, planning, and evaluations in Long-term working-memory (LTWM) essential for “Retain[ing] skills and knowledge” “ learning transfer” “ generalization”
    • The Complex Process of Acquisition of of Expert Performance
    •  
    •  
    •  
    •