Your SlideShare is downloading. ×
0
A Research Approach to Develop Measurable Competencies and Skills: Videogames or Real-World Training K. Anders Ericsson De...
One of the crucial issues of training interventions is to be able to specify  their goals so one can measure and evaluate ...
Getting away from paper-and-pencil  or multiple-choice tests of the type of  Scholastic Aptitude Tests (SAT) Define outcom...
Traditional Instruction On-the-job experience Target Performance “ Retain skills and knowledge” “ learning transfer” “ gen...
Traditional Instruction On-the-job experience Target Performance “ Retain skills and knowledge” “ learning transfer” “ gen...
Outline of My Presentation <ul><li>If   the desired competency concerned a domain ,  </li></ul><ul><li>  like Chess, Music...
I.  If the desired competency concerned a domain,  like Chess, How is Expert Performance Attained How  did  chess players ...
Our Knowledge about Learning and  Skilled Performance is Limited How good is a chess player or  an individual in a domain ...
Professors  Drivers  Doctors (94% above average)  (80% above average)  (Davies et al., 2006) Self-Assessments on Relative ...
Professors  Drivers  Doctors (94% above average)  (80% above average)  (Davies et al., 2006) Self-Assessments on Relative ...
How could a chess player  in the chess club be able  to keep improving?  Playing games Watching better players play Readin...
Rated Relevance for Improvement from Doing X on a scale from 1 to 7  (Charness, Krampe, & Mayr, 1996) Active participation...
Rated Relevance for Improvement from Doing X on a scale from 1 to 7  (Charness, Krampe, & Mayr, 1996) Active participation...
Effects and Dose-Response Relations to Practice Activities Professional teachers and coaches * Monitor students’ developme...
Rated Relevance for Improvement from Doing X on a scale from 1 to 7  (Charness, Krampe, & Mayr, 1996) Active participation...
Simulated Play Against World Class Players Study published games by chess  masters Make predictions for each next move Che...
 
  On-the-job   supervised   experience Classroom  Knowledge  Acquisition  Measurable Outcomes Part II. Objective Measureme...
The most respected experts (“expensive” stockbrokers)  are not markedly better in picking  stocks on New York Stock Exchan...
Superior   Performance Chess winning chess games 50 games ( 100-250 hours ) for a rating Psychotherapy superior ability to...
Minimal Effects of Traditional  Indicators of Expertise Long  Education Extended experience as a professional Teaching Psy...
Identify challenging and difficult situations,  where experts are supposed to excel. Adriaan de Groot  (1914–2006)
Recreate  the situation  and task in laboratory Actions and  thoughts of novices and experts can be directly  compared
Select the best chess move for this position Type as much of the presented text as possible within one minute Play the sam...
Reproducible Soccer Performance
From cover story in  Time Magazine   (March 10, 2008) on   The Science of Experience Response to Critical Unexpected Probl...
Salchow Part III. Mechanisms, Skills, and their Acquisition  through Deliberate Practice
The problem with learning  during  work and play Andy Murray (Scotland) in 2010
The problem with learning  during  work and play
Immediacy of Informative Feedback Sport training    performance (1-2 years) Investing    value  (10-20 years) Medicine  ...
Increase in Complexity and Control  as a Function of Years of Piano Training Years of piano training
“ individualized training activities especially designed by a coach or teacher  to improve specific aspects of  an individ...
Design and Sequencing of    Training Activities Professional teachers and coaches * Monitor students’ development * design...
Accumulated Amount of Practice Alone  During the Development of Amateurs and Expert Musicians
The Body’s Resilience to Change   The Protection of Equilibrium
Increase of Fitness
Deliberate Practice Optimal
How many consecutive push-ups  can someone perform without pause? (H. H. Hart, 1974) Range of  modifiability  with extende...
Charles Linster   Non-stop  push-up records http://www.recordholders.org/en/list/ulysses.html 6,006 Charles Linster (USA) ...
Part IV. Acquisition of Control and Complex  Mental Representations—Beyond Automaticity
The Development of Increased Control Actively Avoiding Automating Control of Critical Aspects of Performance
Reproducibly Superior Performance  is not automatic but is associated  with informative verbalizations of thoughts
Black on move Position B *  What aspect was overlooked * When could this aspect have been discovered * How to avoid simila...
Alekhine beat most of the 30 skilled players while playing  them simultaneously under blindfold conditions   Ref P16 A htt...
The Importance of Acquired Mental Representations
Expert Performers are Better Able to Represent and Analyze Situations Slowing Down (Moulton et al.,2010) (Ward & Williams,...
V.  Training and the Fidelity Assumption It is frequently assumed that  the best learning of skills occurs  in the natural...
Perceptual Performance on Cardiac Auscultation Instruction and Experience 0-9 years 10-20 years Over 20 years General Prac...
Benefits of Provided Experience on Performance Use of Cockpits with Visual and Kinesthetic Simulation   Modest transfer of...
Identifying the Crucial Elements of  Effective Simulator Training - Deliberate Practice A review of 109 studies of high-fi...
The  Use of Simulators for Deliberate Practice with Performers at Different Levels of Expertise Allows exposure to challen...
Deliberate Practice with Simulators Authentic test conditions  with actors (Kneebone et al., 2005) Reinstating actual scen...
  Measurement and Training Capturing the essence of expert performance Measuring it with representative tasks Measuring   ...
Traditional Instruction On-the-job experience Target Performance “ Retain skills and knowledge” “ learning transfer” “ gen...
Recommendations Identify skills and analyze domains of activity   where performance can be measured with  objective method...
The Complex  Process of Acquisition of of Expert  Performance
 
 
 
 
Upcoming SlideShare
Loading in...5
×

A Research Approach to Develop Measurable Competencies and Skills: Videogames or Real-World Training

950

Published on

0 Comments
1 Like
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
950
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
19
Comments
0
Likes
1
Embeds 0
No embeds

No notes for slide

Transcript of "A Research Approach to Develop Measurable Competencies and Skills: Videogames or Real-World Training"

  1. 1. A Research Approach to Develop Measurable Competencies and Skills: Videogames or Real-World Training K. Anders Ericsson Department of Psychology Florida State University by
  2. 2. One of the crucial issues of training interventions is to be able to specify their goals so one can measure and evaluate objective outcomes. I believe that videogames and simulators can provide us with the tools to specify the goal and targets of education for everyday and professional life.
  3. 3. Getting away from paper-and-pencil or multiple-choice tests of the type of Scholastic Aptitude Tests (SAT) Define outcomes with video games and simulations of everyday situations and tasks
  4. 4. Traditional Instruction On-the-job experience Target Performance “ Retain skills and knowledge” “ learning transfer” “ generalization”
  5. 5. Traditional Instruction On-the-job experience Target Performance “ Retain skills and knowledge” “ learning transfer” “ generalization” Alternatives , such as Simulations Videogames
  6. 6. Outline of My Presentation <ul><li>If the desired competency concerned a domain , </li></ul><ul><li> like Chess, Music, or Sports, how </li></ul><ul><li> would expert performance be attained </li></ul>II. Objective measurement of performance III. Mechanisms, skills, and their acquisition through deliberate practice IV. Acquisition of control and complex mental representations— Beyond automaticity V. Training and the fidelity assumption
  7. 7. I. If the desired competency concerned a domain, like Chess, How is Expert Performance Attained How did chess players attain their performance? Benjamin Franklin learned to play chess from an acquaintance in 1733 He was one of the first chess players in America. Although he was an avid player he reached a modest level of chess skill
  8. 8. Our Knowledge about Learning and Skilled Performance is Limited How good is a chess player or an individual in a domain of expertise? -- Self ratings
  9. 9. Professors Drivers Doctors (94% above average) (80% above average) (Davies et al., 2006) Self-Assessments on Relative Performance How would you rate your own performance? bottom 10% below average average above average top 10%
  10. 10. Professors Drivers Doctors (94% above average) (80% above average) (Davies et al., 2006) Self-Assessments on Relative Performance How would you rate your own performance? bottom 10% below average average above average top 10%
  11. 11. How could a chess player in the chess club be able to keep improving? Playing games Watching better players play Reading books Improving One’s Chess Performance or Performance in Tennis, Golf or SCRABBLE
  12. 12. Rated Relevance for Improvement from Doing X on a scale from 1 to 7 (Charness, Krampe, & Mayr, 1996) Active participation No in chess tournaments 6.1 correlation Playing chess games Negative outside of chess 3.6 correlation tournaments Serious analysis of Positive positions alone 5.9 correlation
  13. 13. Rated Relevance for Improvement from Doing X on a scale from 1 to 7 (Charness, Krampe, & Mayr, 1996) Active participation No in chess tournaments 6.1 correlation Playing chess games Negative outside of chess 3.6 correlation tournaments Serious analysis of Positive positions alone 5.9 correlation
  14. 14. Effects and Dose-Response Relations to Practice Activities Professional teachers and coaches * Monitor students’ development * design training tasks for individual students Expert Performance
  15. 15. Rated Relevance for Improvement from Doing X on a scale from 1 to 7 (Charness, Krampe, & Mayr, 1996) Active participation No in chess tournaments 6.1 correlation Playing chess games Negative outside of chess 3.6 correlation tournaments Serious analysis of Positive positions alone 5.9 correlation
  16. 16. Simulated Play Against World Class Players Study published games by chess masters Make predictions for each next move Check if your prediction was correct, if not, study the chess position until you understand why the correct move was played Black on move Position B
  17. 18. On-the-job supervised experience Classroom Knowledge Acquisition Measurable Outcomes Part II. Objective Measurement of Performance Independent ability to increase patients’ health Kirkpatrick’s Four Level Model of Training Evaluation
  18. 19. The most respected experts (“expensive” stockbrokers) are not markedly better in picking stocks on New York Stock Exchange than a random process, such as a monkey throwing darts (c.f. Wall Street Journal study)
  19. 20. Superior Performance Chess winning chess games 50 games ( 100-250 hours ) for a rating Psychotherapy superior ability to increase mental patients’ health (post-pre) 50-100 patients (400-800 hours) Teaching superior ability to increase students’ performance (post - pre) 500-1000 contact hours
  20. 21. Minimal Effects of Traditional Indicators of Expertise Long Education Extended experience as a professional Teaching Psychotherapy
  21. 22. Identify challenging and difficult situations, where experts are supposed to excel. Adriaan de Groot (1914–2006)
  22. 23. Recreate the situation and task in laboratory Actions and thoughts of novices and experts can be directly compared
  23. 24. Select the best chess move for this position Type as much of the presented text as possible within one minute Play the same piece of music twice in same manner Chess Typing Music Domain Presented Information Task Selection Speed Control
  24. 25. Reproducible Soccer Performance
  25. 26. From cover story in Time Magazine (March 10, 2008) on The Science of Experience Response to Critical Unexpected Problems
  26. 27. Salchow Part III. Mechanisms, Skills, and their Acquisition through Deliberate Practice
  27. 28. The problem with learning during work and play Andy Murray (Scotland) in 2010
  28. 29. The problem with learning during work and play
  29. 30. Immediacy of Informative Feedback Sport training  performance (1-2 years) Investing  value (10-20 years) Medicine  patient outcomes (12-36 months)
  30. 31. Increase in Complexity and Control as a Function of Years of Piano Training Years of piano training
  31. 32. “ individualized training activities especially designed by a coach or teacher to improve specific aspects of an individual's performance through repetition and successive refinement. To receive maximal benefit from feedback, individuals have to monitor their training with full concentration, which is effortful and limits the duration of daily training”. (Ericson & Lehmann, 1996, pp. 278-279) Deliberate Practice
  32. 33. Design and Sequencing of Training Activities Professional teachers and coaches * Monitor students’ development * design training tasks for individual students Expert Performance
  33. 34. Accumulated Amount of Practice Alone During the Development of Amateurs and Expert Musicians
  34. 35. The Body’s Resilience to Change The Protection of Equilibrium
  35. 36. Increase of Fitness
  36. 37. Deliberate Practice Optimal
  37. 38. How many consecutive push-ups can someone perform without pause? (H. H. Hart, 1974) Range of modifiability with extended Deliberate Practice
  38. 39. Charles Linster Non-stop push-up records http://www.recordholders.org/en/list/ulysses.html 6,006 Charles Linster (USA) 05-Oct-1965 … 10,507 Minoru Yoshida (JAP) Oct-1980 Fu Bingli, a kung fu master from China 12 press ups with right finger http://www.telegraph.co.uk/news/newstopics/howaboutthat/6385592/Man-proves-he-has-worlds-strongest-fingers.html# Accessed on October 23 rd , 2009
  39. 40. Part IV. Acquisition of Control and Complex Mental Representations—Beyond Automaticity
  40. 41. The Development of Increased Control Actively Avoiding Automating Control of Critical Aspects of Performance
  41. 42. Reproducibly Superior Performance is not automatic but is associated with informative verbalizations of thoughts
  42. 43. Black on move Position B * What aspect was overlooked * When could this aspect have been discovered * How to avoid similar mistakes in the future * Develop new skills by deliberate practice Learning from Making an Incorrect Move
  43. 44. Alekhine beat most of the 30 skilled players while playing them simultaneously under blindfold conditions Ref P16 A http://images.google.com/imgres?imgurl=http://www.chessbase.com/news/2007/bilbao04.jpg&imgrefurl=http://www.chessbase.com/newsdetail.asp%3Fnewsid%3D4189&h=318&w=480&sz=36&hl = en&start=1&usg=__JwOiqXRmMEJcHCssYGF1N_5BqU4=&tbnid=DC9VzEbhAUofSM:&tbnh=85&tbnw=129&prev=/images%3Fq%3Dblindfold%2Bchess%26gbv%3D2%26hl%3Den%26sa%3DG Accessed on August 26, 2008 Blindfold Chess Exceptional abilities
  44. 45. The Importance of Acquired Mental Representations
  45. 46. Expert Performers are Better Able to Represent and Analyze Situations Slowing Down (Moulton et al.,2010) (Ward & Williams, 2003) (Tuffiash, Roring, & Ericsson, 2007) (Kesting, et al., 2010)
  46. 47. V. Training and the Fidelity Assumption It is frequently assumed that the best learning of skills occurs in the natural environment . Hence, simulation should therefore reproduce the experience in the natural environment What do we know about natural learning? How optimal is that type of natural learning?
  47. 48. Perceptual Performance on Cardiac Auscultation Instruction and Experience 0-9 years 10-20 years Over 20 years General Practitioners Student Based on Butterworth & Reppert (1960) Reviews (Choudhry, Fletcher, & Soumerai, 2005; Ericsson, 2004; Ericsson, Whyte, & Ward, 2007) Performance No Correlations between Amount of Experience and Performance after the First Year
  48. 49. Benefits of Provided Experience on Performance Use of Cockpits with Visual and Kinesthetic Simulation Modest transfer of performance— savings of training in real airplanes (except new procedures) (Allerton, 2000; Roessingh, 2005; Rantanen & Talleur, 2005) Focus on Fidelity and Safety A airline captain needs 3,500 hours of flying plus 60 hours of simulation (Parker, Johns, & Hellige, 2007)
  49. 50. Identifying the Crucial Elements of Effective Simulator Training - Deliberate Practice A review of 109 studies of high-fidelity medical simulation showed that the characteristics of Deliberate Practice accounted for improvements (Issenberg et al., 2005) A review of 31 of those studies that examined repetitive practice, and “ found a strong association ( η2 = .46 ) between hours of practice on high-fidelity medical simulators and standardized learning outcomes .” (Issenberg et al., 2006, p. 792)
  50. 51. The Use of Simulators for Deliberate Practice with Performers at Different Levels of Expertise Allows exposure to challenging situations without danger Allows exposure to difficult situations under conditions optimal to learning and performance - Individuals being focused and ready - Immediate feedback (and opportunity for repetition) - Presentation of related cases to facilitate discrimination Allows the presentation of rare emergency situations
  51. 52. Deliberate Practice with Simulators Authentic test conditions with actors (Kneebone et al., 2005) Reinstating actual scenarios in anesthesiology (Liu et al, 2009)
  52. 53. Measurement and Training Capturing the essence of expert performance Measuring it with representative tasks Measuring Identification current level of  of optimal performance training tasks
  53. 54. Traditional Instruction On-the-job experience Target Performance “ Retain skills and knowledge” “ learning transfer” “ generalization” Alternatives , such as Simulations Videogames
  54. 55. Recommendations Identify skills and analyze domains of activity where performance can be measured with objective methods that capture on-the-job performance with large individual differences in attained performance where increases in performance motivate major investment Develop libraries of representative situations with appropriate feed-back about correct/appropriate actions with scaled difficulty to maximize optimal training effects Develop cognitive structures to support Deliberate Practice assessment of representations for thinking, planning, and evaluations in Long-term working-memory (LTWM) essential for “Retain[ing] skills and knowledge” “ learning transfer” “ generalization”
  55. 56. The Complex Process of Acquisition of of Expert Performance
  1. A particular slide catching your eye?

    Clipping is a handy way to collect important slides you want to go back to later.

×