TRUTHS ABOUTTRAININGEVALUATION          David Kelly, CPLP, CRP
A LITTLE ABOUT ME   Training Director for over 10 Years   Director, Center for Learning at ACLD   Certified Professiona...
TODAY’S DISCUSSION A Brief History of Measurement in Learning  and Development Programs What do Trainers Measure? Does W...
WHAT CAME BEFORE…               Why are we               talking about               ‘evaluation’ of               trainin...
A BRIEF HISTORY OF TRAINING EVALUATIONDon Kirkpatrick – The Four Levels of Evaluation  1.   Reaction  2.   Learning  3.   ...
A BRIEF HISTORY OF TRAINING EVALUATION
CAPITAL               NON-CAPITALINVESTMENTS           INVESTMENTS   Buildings            Marketing   Equipment        ...
CAPITAL               NON-CAPITALINVESTMENTS           INVESTMENTS   Buildings            Marketing   Equipment        ...
SATISFACTION WITH MEASURES OF SUCCESSOF LEARNING AND DEVELOPMENTTo what extent are you satisfied with themeasures (value) ...
WHAT METRICS ARE  YOU USING?
COMMON LEARNING METRICS Cost per Employee  or Hour                       Do these metrics Course Completions            ...
MEANINGFUL PERFORMANCE METRICS Decreased Time to  Performance Reduced  Performance Cycle  Times Quality  Improvements ...
HERE’S MY TWO CENTS…              Measurement     for               measurement’s sake               is a waste of time. ...
THE CREDIBILITY ISSUEThe truth is most training evaluation data is either    irrelevant or does not hold up to scrutiny   ...
THE CREDIBILITY ISSUETraining: “Sales increased 40% after Training.”Marketing: “That wasn’t training, we ranadvertisements...
THE CREDIBILITY ISSUE   Source: http://elearningindustry.com/subjects/free-elearning-resources/item/256-free-elearning-roi-
THE BOTTOM LINEROI and the metrics used for TrainingEvaluation are driven by companies andconsultants with a vested intere...
HERE’S MY TWO CENTS…              It’s not about               Learning; It’s about               Performance            ...
HERE’S MY TWO CENTS…              Most  executives that               ask for an ROI, DO               NOT WANT AN       ...
MAYBE IT’S NOT ABOUT EVALUATION…                    What other ways                    can we                    determine...
WHO DECIDES WHAT SUCCESS MEANS?The Program Stakeholders Executives Management Approving Financing of  Training Learning...
HERE’S MY TWO CENTS…              The  time to think               about how you will               define success is    ...
WHAT’S YOUR STORY?            A strong testimonial            from a participant or            executive on how           ...
A STRUCTURED APPROACH
THE SUCCESS CASE METHOD           Step 1.           Identify targeted business           goals and impact           expect...
THE SUCCESS CASE METHOD           Step 2.           Survey a large           representative sample of all           partic...
THE SUCCESS CASE METHOD           Step 3.           Analyze the survey data to           identify:              a small g...
THE SUCCESS CASE METHOD           Step 4.           Conduct interviews with the           two selected groups to:         ...
THE SUCCESS CASE METHOD           Step 5.           Document and disseminate           the story:              Report imp...
IT’S NOT ABOUT HOW MANY SEATS ARE FILLED               It’s not about the Training, it’s               about the Performan...
IT’S NOT ABOUT HOW MANY SEATS ARE FILLED                Most learning and                 performance improvement        ...
SUMMARY Data is critical to show the  value and effectiveness of  Learning and Performance  Programs Performance is what...
SUMMARY Success starts with  identifying desired impacts  during the needs  assessment Managers of program  participants...
QUESTIONS?David Kelly E-Mail:  LnDDave@gmail.com Phone: (516) 474-1852 Twitter: @LnDDave Blog: http://davidkelly.me A...
Upcoming SlideShare
Loading in...5
×

Truths About Training Evaluation

1,467
-1

Published on

These slides were used during m

Published in: Education
0 Comments
2 Likes
Statistics
Notes
  • Be the first to comment

No Downloads
Views
Total Views
1,467
On Slideshare
0
From Embeds
0
Number of Embeds
3
Actions
Shares
0
Downloads
47
Comments
0
Likes
2
Embeds 0
No embeds

No notes for slide
  • Start with the story of me finding the binders of program evaluations
  • Truths About Training Evaluation

    1. 1. TRUTHS ABOUTTRAININGEVALUATION David Kelly, CPLP, CRP
    2. 2. A LITTLE ABOUT ME Training Director for over 10 Years Director, Center for Learning at ACLD Certified Professional in Learning and Performance (CPLP) Certified ROI Professional (CRP) Board Member of ASTD Long Island Member of ASTD’s National Advisors for Chapters Member of the eLearn Magazine Editorial Board
    3. 3. TODAY’S DISCUSSION A Brief History of Measurement in Learning and Development Programs What do Trainers Measure? Does What We Measure Even Matter? The Question of Credibility A Better Definition for Training Success
    4. 4. WHAT CAME BEFORE… Why are we talking about ‘evaluation’ of training in the first place?
    5. 5. A BRIEF HISTORY OF TRAINING EVALUATIONDon Kirkpatrick – The Four Levels of Evaluation 1. Reaction 2. Learning 3. Behavior 4. ResultsJack Phillips – ROI Methodology 5. ROI (Benefits / Costs)
    6. 6. A BRIEF HISTORY OF TRAINING EVALUATION
    7. 7. CAPITAL NON-CAPITALINVESTMENTS INVESTMENTS Buildings  Marketing Equipment  Human Resources Tools  Quality Vehicles  Some Technology Companies  Staff Support Some Technology  Processes ROI is usually ROI is not typically measured measuredWHY MEASURE TRAINING?
    8. 8. CAPITAL NON-CAPITALINVESTMENTS INVESTMENTS Buildings  Marketing Equipment  Human Resources Tools  Quality Vehicles  Some Technology Companies  Staff Support Some Technology  Processes ~15% of Total ~85% of Total Expenditures ExpendituresWHY MEASURE TRAINING?
    9. 9. SATISFACTION WITH MEASURES OF SUCCESSOF LEARNING AND DEVELOPMENTTo what extent are you satisfied with themeasures (value) of learning anddevelopment?1. Very Dissatisfied – 8%2. Dissatisfied – 45%3. Satisfied – 41%4. Very Satisfied – 6%*CEO Survey—Fortune 500 and LargePrivate Companies, ROI Institute
    10. 10. WHAT METRICS ARE YOU USING?
    11. 11. COMMON LEARNING METRICS Cost per Employee or Hour Do these metrics Course Completions REALLY matter? Learner Satisfaction Attendance Test Scores
    12. 12. MEANINGFUL PERFORMANCE METRICS Decreased Time to Performance Reduced Performance Cycle Times Quality Improvements Increased Customer Satisfaction Increased Sales
    13. 13. HERE’S MY TWO CENTS…  Measurement for measurement’s sake is a waste of time. Ask yourself… “Will I be doing anything with the data we’re collecting?”
    14. 14. THE CREDIBILITY ISSUEThe truth is most training evaluation data is either irrelevant or does not hold up to scrutiny *Bozarth-Ferguson Magic Formula used with permission
    15. 15. THE CREDIBILITY ISSUETraining: “Sales increased 40% after Training.”Marketing: “That wasn’t training, we ranadvertisements during that period.”Training: “Turnover has decreased 25% as aresult of our new orientation programs.”Human Resources: “That wasn’t training, weupdated hiring criteria.”
    16. 16. THE CREDIBILITY ISSUE Source: http://elearningindustry.com/subjects/free-elearning-resources/item/256-free-elearning-roi-
    17. 17. THE BOTTOM LINEROI and the metrics used for TrainingEvaluation are driven by companies andconsultants with a vested interest inpeople believing these metrics matter.In most cases, the data collected isn’treally relevant to business outcomes.
    18. 18. HERE’S MY TWO CENTS…  It’s not about Learning; It’s about Performance  It’s not about what people KNOW; it’s about what people DO.
    19. 19. HERE’S MY TWO CENTS…  Most executives that ask for an ROI, DO NOT WANT AN ACTUAL ROI. They want to know that the efforts had value.
    20. 20. MAYBE IT’S NOT ABOUT EVALUATION… What other ways can we determine success besides traditional training evaluation?
    21. 21. WHO DECIDES WHAT SUCCESS MEANS?The Program Stakeholders Executives Management Approving Financing of Training Learning and Development Professionals Program Participants Managers of Program Participants
    22. 22. HERE’S MY TWO CENTS…  The time to think about how you will define success is during the DESIGN of a program, not after it’s completion.
    23. 23. WHAT’S YOUR STORY? A strong testimonial from a participant or executive on how training helped workers perform better is often more powerful than any number on a spreadsheet.
    24. 24. A STRUCTURED APPROACH
    25. 25. THE SUCCESS CASE METHOD Step 1. Identify targeted business goals and impact expectations.
    26. 26. THE SUCCESS CASE METHOD Step 2. Survey a large representative sample of all participants in a program to identify high impact and low impact cases.
    27. 27. THE SUCCESS CASE METHOD Step 3. Analyze the survey data to identify:  a small group of successful participants  a small group unsuccessful participants
    28. 28. THE SUCCESS CASE METHOD Step 4. Conduct interviews with the two selected groups to:  Document the nature and business value of their application of learning  Identify the performance factors that supported learning application and obstacles that prevented it.
    29. 29. THE SUCCESS CASE METHOD Step 5. Document and disseminate the story:  Report impact  Applaud successes  Use data to educate managers and organization
    30. 30. IT’S NOT ABOUT HOW MANY SEATS ARE FILLED It’s not about the Training, it’s about the Performance Improvement. It’s about an ongoing process, not an event. “Evaluating training is like evaluating the wedding instead of the marriage.” - Robert Brinkhoff
    31. 31. IT’S NOT ABOUT HOW MANY SEATS ARE FILLED  Most learning and performance improvement takes place informally, and on the job.  The success of a training program is not determined via a test; it is determined by how effectively trainees use the training when they return to their work.
    32. 32. SUMMARY Data is critical to show the value and effectiveness of Learning and Performance Programs Performance is what happens after a Learning Program, so the most important data pertains to how work has changed
    33. 33. SUMMARY Success starts with identifying desired impacts during the needs assessment Managers of program participants are the key stakeholders for driving successful learning and performance programs
    34. 34. QUESTIONS?David Kelly E-Mail: LnDDave@gmail.com Phone: (516) 474-1852 Twitter: @LnDDave Blog: http://davidkelly.me Also connect via LinkedIn or Facebook
    1. A particular slide catching your eye?

      Clipping is a handy way to collect important slides you want to go back to later.

    ×